aboutsummaryrefslogtreecommitdiff
path: root/src/main/java/dev/morling/onebrc
Commit message (Collapse)AuthorAgeFilesLines
* Thank you for the challange!! (#599)Jaime Polidura2024-01-281-0/+398
| | | | | | | | | | | | | | | | | | | | | | | | | * added code * Fixed pointers bugs * removed my own benchmark * added comment on how I handle hash collisions * executed mwvn clean verify * made scripts executable & fixed rounding issues * Fixed way of dealing with hash collisions * changed method name sameNameBytes to isSameNameBytes * changes script from sh to bash * fixed chunking bug * Fixed bug in chunking when file size is too small * added Runtime.getRuntime().availableProcessors
* Solution without unsafe using vector API (#602)giovannicuccu2024-01-282-177/+222
| | | | | | | | | | | | | * Solution without unsafe * Solution without unsafe * Solution without unsafe, remove the usage of bytebuffer, passes the create_measurements3 test * bug fix for 10k test, update also the CreateMeasurements3.java to use '\n' as newline instead of the os value (if it runs on windows it uses crlf and "breaks" the file format ) --------- Co-authored-by: Giovanni Cuccu <gcuccu@imolainformatica.it>
* serkan-ozal's 3rd submission with some minor improvements: (#615)Serkan ÖZAL2024-01-281-16/+38
| | | | | | - faster merge by ignoring empty entries in the map - enable CDS for faster startup (added `prepare_serkan-ozal.sh` to generate CDS archive in advance) - some tweaks with JVM options - optimized result printing
* anestoruk second attempt (#625)Andrzej Nestoruk2024-01-281-59/+61
| | | | | | | * initial implementation * few improvements and a cleanup (down to ~12s) * use array instead of hashmap for collecting partial results
* processing three at once (#626)Artsiom Korzun2024-01-281-68/+94
|
* shipilev: Amendments to version 4 (#627)Aleksey Shipilëv2024-01-281-60/+96
| | | | | * Amendments * One more locality touchup: no need to carry the entire name array
* jerrinot - running out of ideas (#631)Jaromir Hamala2024-01-281-19/+14
| | | | | * another shameless copycat from thomas: less safepoints * I have no idea what I am doing
* Reduce preferred vector size (#622)Dr Ian Preston2024-01-281-5/+5
| | | Co-authored-by: Ian Preston <ianopolous@protonmail.com>
* Version 4 (#183)Aleksey Shipilëv2024-01-281-0/+649
|
* Mahadev virtual thread 1brc (#611)Mahadev K2024-01-281-0/+152
| | | | | | | | | | | | | * Read file with multiple virtual threads and process chunks of file data in parallel. * Updated logic to bucket every chunk of aggs into a vector and merge them into a TreeMap for printing. * Virtual Thread / File Channels Impl. * Renamed files with GHUsername. * Added statement to get vals before updating. * Added executable permission to the files.
* Dkarampi solution (#614)Dimitris Karampinas2024-01-281-0/+260
| | | | | | | | | * Simple multi-threaded version * Format code * Formatted code * More formatting
* Use native type, remove lots of type conversions (#618)Van Phu DO2024-01-281-80/+99
| | | | | | | * less type conversion, less string cast * adjust some comments * fixed format issue
* Bytesfellow initial submittion (#619)Aleksei2024-01-281-0/+557
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Latest snapshot (#1) preparing initial version * Improved performance to 20seconds (-9seconds from the previous version) (#2) improved performance a bit * Improved performance to 14 seconds (-6 seconds) (#3) improved performance to 14 seconds * sync branches (#4) * initial commit * some refactoring of methods * some fixes for partitioning * some fixes for partitioning * fixed hacky getcode for utf8 bytes * simplified getcode for partitioning * temp solution with syncing * temp solution with syncing * new stream processing * new stream processing * some improvements * cleaned stuff * run configuration * round buffer for the stream to pages * not using compute since it's slower than straightforward get/put. using own byte array equals. * using parallel gc * avoid copying bytes when creating a station object * formatting * Copy less arrays. Improved performance to 12.7 seconds (-2 seconds) (#5) * initial commit * some refactoring of methods * some fixes for partitioning * some fixes for partitioning * fixed hacky getcode for utf8 bytes * simplified getcode for partitioning * temp solution with syncing * temp solution with syncing * new stream processing * new stream processing * some improvements * cleaned stuff * run configuration * round buffer for the stream to pages * not using compute since it's slower than straightforward get/put. using own byte array equals. * using parallel gc * avoid copying bytes when creating a station object * formatting * some tuning to increase performance * some tuning to increase performance * avoid copying data; fast hashCode with slightly more collisions * avoid copying data; fast hashCode with slightly more collisions * cleanup (#6) * tidy up
* Some fine tuning for thomaswue (#606)Thomas Wuerthinger2024-01-281-152/+239
| | | | | | * Some fine tuning. * Process 2MB segments to make all threads finish at the same time. Process with 3 scanners in parallel in the same thread.
* anestoruk submission (#617)Andrzej Nestoruk2024-01-281-0/+193
| | | | | * initial implementation * few improvements and a cleanup (down to ~12s)
* use long for string equals (#613)John Ziamos2024-01-281-8/+11
| | | use more generic hashcode
* Initial submission for jonathan_aotearoa. (#586)Jonathan Wright2024-01-281-0/+587
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Initial submission for jonathan_aotearoa * Fixing typos * Adding hyphens to prepare and calculate shell scripts so that they're aligned with my GitHub username. * Making chunk processing more robust in attempt to fix the cause of the build error. * Fixing typo. * Fixed the handling of files less than 8 bytes in length. * Additional assertion, comment improvements. * Refactoring to improve testability. Additional assertion and comments. * Updating collision checking to include checking if the station name is equal. * Minor refactoring to make param ordering consistent. * Adding a custom toString method for the results map. * Fixing collision checking bug * Fixing rounding bug. * Fixing collision bug. --------- Co-authored-by: jonathan <jonathan@example.com>
* serkan-ozal's 2nd submission with some minor improvements: (#612)Serkan ÖZAL2024-01-281-55/+72
| | | | - use shared memory arena and region between worker threads - reduce number of instructions slightly while processing file region
* jerrinot's improvement (#607)Jaromir Hamala2024-01-281-111/+167
| | | | | | | | | | * some random changes with minimal, if any, effect * use munmap() trick credit: thomaswue * some smaller tweaks * use native image
* CalculateAverage_pdrakatos (#515)PanosDR2024-01-281-0/+244
| | | | | | | | | | | | | | | | | | | | | * CalculateAverage_pdrakatos * Rename to be valid with rules * CalculateAverage_pdrakatos * Rename to be valid with rules * Changes on scripts execution * Fixing bugs causing scripts not to be executed * Changes on prepare make it compatible * Fixing passing all tests * Increase direct memory allocation buffer * Fixing memory problem causes heap space exception
* Second version by albertoventurini (#609)Alberto Venturini2024-01-281-30/+59
| | | | | | | | | * Contribution by albertoventurini * Use byte arrays of size 2^20 --------- Co-authored-by: Alberto Venturini <alberto.venturini@accso.de>
* serkan-ozal: Initial impl (#553)Serkan ÖZAL2024-01-281-0/+674
| | | | | | | | | | | * Initial impl * Fix bad file descriptor error in the `calculate_average_serkan-ozal.sh` * Disable Epsilon GC and rely on default GC. Because apparently, JIT and Epsilon GC don't play well together in the eval machine for short lived Vector API's `ByteVector` objects * Take care of byte order before processing key length with bit shift operators * Fix key equality check for long keys
* Simplify dedupeStation() (#589)Dr Ian Preston2024-01-271-16/+10
| | | | | 13.8s locally now. Co-authored-by: Ian Preston <ianopolous@protonmail.com>
* Use LinkedBlockingQueue to process results - based on thomaswue (#603)tivrfoa2024-01-271-0/+431
| | | | | | | | | | | | | | | | | | | | | /** * Solution based on thomaswue solution, commit: * commit d0a28599c293d3afe3291fc3cf169a7b25ae9ae6 * Author: Thomas Wuerthinger <thomas.wuerthinger@oracle.com> * Date: Sun Jan 21 20:13:48 2024 +0100 * * Changes: * 1) Use LinkedBlockingQueue to store partial results, that * will then be merged into the final map later. * As different chunks finish at different times, this allows * to process them as they finish, instead of joining the * threads sequentially. * This change seems more useful for the 10k dataset, as the * runtime difference of each chunk is greater. * 2) Use only 4 threads if the file is >= 14GB. * This showed much better results on my local test, but I only * run with 200 million rows (because of limited RAM), and I have * no idea how it will perform on the 1brc HW. */
* (new submission) melgenek: ~top 15 on 10k. Buffered IO, VarHandles, vectors, ↵Yevhenii Melnyk2024-01-271-0/+549
| | | | | | | custom hashtable (#600) * melgenek: ~top 15 on 10k. Buffered IO, VarHandles, vectors, custom hashtable * Calculate the required heap size dynamically
* Fix hash code collisions (#605)Jairo Graterón2024-01-271-79/+77
| | | | | | | | | | | | | * fix test rounding, pass 10K station names * improved integer conversion, delayed string creation. * new algorithm hash, use ConcurrentHashMap * fix rounding test * added the length of the string in the hash initialization. * fix hash code collisions
* Reading 1B row file using Java NIO lib. (#601)Manish Garg2024-01-271-0/+169
|
* Larger heap, small tweaks (#593)Roy van Rijn2024-01-271-34/+50
| | | More small tweaks, perf from 775~ to 738~
* 1BRC gigiblender (#595)Florin Blanaru2024-01-271-0/+501
| | | | | * Dirty implementation gigiblender * Final impl gigiblender
* Next version (#596)Roman Musin2024-01-271-29/+49
| | | | | | | | | | | | | | | | | | | * cleanup prepare script * native image options * fix quardaric probing (no change to perf) * mask to get the last chunk of the name * extract hash functions * tweak the probing loop (-100ms) * fiddle with native image options * Reorder conditions in hope it makes branch predictor happier * extracted constant
* improve hard disk access locality, another 8% (#591)Van Phu DO2024-01-271-155/+172
| | | | | | | * improve hard disk access locality, another 8% * add some comments & credit * fixed format
* First attempt with Java-managed concurrency (#590)Hieu Dao Quang2024-01-271-0/+117
| | | Co-authored-by: Quang Hieu Dao <hieu_dq@flinters.vn>
* Initial submission (#588)rcasteltrione2024-01-271-0/+309
| | | | | * Initial submission * fixed not executable scripts
* Fix hundredwatt's entry on 10k dataset (#558)Jason Nochlin2024-01-261-8/+13
| | | | | | | | | | | | | * Improve hash function * remove limit on number of cores * fix calculation of boundaries between chunks * fix IOOBE --------- Co-authored-by: Jason Nochlin <hundredwatt@users.noreply.github.com>
* CalculateAverage_gonix update (#579)gonix2024-01-251-23/+21
| | | Minor updates here and there, shaves off ~5% of execution time on my machine.
* Contribution by albertoventurini (#578)Alberto Venturini2024-01-251-0/+299
| | | | | | | | | | | | | | | * Contribution by albertoventurini * Shave off a couple of hundreds of milliseconds, by making an assumption on temperature readings * Parse reading without loop, inspired by other solutions * Use all cores * Small improvements, only allocate 247 positions instead of 256 --------- Co-authored-by: Alberto Venturini <alberto.venturini@accso.de>
* Updates for gamlerhart: Simpler & Faster (#580)Roman Stoffel2024-01-251-131/+86
| | | | | | | | | | | | | | | * Update with Rounding Bugfix * Simplification of Merging Results * More Plain Java Code for Value Storage * Improve Performance by Stupid Hash Drop around 3 seconds on my machine by simplifying the hash to be ridicules stupid, but faster. * Fix outdated comment
* Second submission to keep a bit of dignity (#581)Dmitry Bufistov2024-01-251-200/+182
| | | | | | | | | * Dmitry challenge * Dmitry submit 2. Use MemorySegment of FileChannle and Unsafe to read bytes from disk. 4 seconds speedup in local test from 20s to 16s.
* tonivade improved solution (#582)Antonio Muñoz2024-01-251-128/+146
| | | | | | | | | | | | | | | | | | | * tonivade improved not using HashMap * use java 21.0.2 * same hash same station * remove unused parameter in sameSation * use length too * refactor parallelization * use parallel GC * refactor * refactor
* Down to 14s locally (#583)Dr Ian Preston2024-01-251-49/+69
| | | | | | Use flat array for stats. Use simd for line termination Co-authored-by: Ian Preston <ianopolous@protonmail.com>
* armandino: minimise hash collisions + other improvements (#585)Arman Sharif2024-01-251-50/+46
|
* Simplify Node class with less field, improve hash mix speed (#584)Van Phu DO2024-01-251-66/+60
| | | | | | | * Simplify Node class with less field, improve hash mix speed * remove some ops, a bit faster * more inline, little bit faster but not sure
* gabrielfoo's first attempt (#556)gabrielfoo2024-01-251-0/+180
| | | | | | | | | * first attempt * formatting fix --------- Co-authored-by: Gabriel <gabriel@gabriel>
* C style code. Should be ~4secs or lower based on local testing. (#559)Vemana2024-01-251-0/+1654
| | | | | | 1. Use Unsafe 2. Fit hashtable in L2 cache. 3. If we can find a good hash function, it can fit in L1 cache even. 4. Improve temperature parsing by using a lookup table
* Laake Scates-Gervasi first submission (justplainlaake) [2.5s execution, ↵Laake Scates-Gervasi2024-01-231-0/+459
| | | | | | | | | | | | | | | | | | | | | | | | | locally similar time to top 5] (#431) * Init Push * organize imports * Add OpenMap * Best outcome yet * Create prepare script and calculate script for native image, also add comments on calculation * Remove extra hashing, and need for the set array * Commit formatting changes from build * Remove unneeded device information * Make shell scripts executable, add hash collision double check for equality * Add hash collision double check for equality * Skip multithreading for small files to improve small file performance
* Rewrote to always read 16 bytes, this has less instructions on perf. (#562)Roy van Rijn2024-01-231-181/+116
| | | It doesn't make a lot of sense since quite some code can be written shorter, but this is what gives the best numbers.
* CalculateAverage_3j5a off-the-shelf Java components + ArraysSupport (#566)3j5a2024-01-231-0/+277
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * off the shell Java components, curious about official runtime results. thnx my laptop results are around 12 seconds, e.g: 87.66user 1.32system 0:12.11elapsed 734%CPU (0avgtext+0avgdata 13980924maxresident)k Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Address sizes: 39 bits physical, 48 bits virtual Byte Order: Little Endian CPU(s): 8 On-line CPU(s) list: 0-7 Vendor ID: GenuineIntel Model name: Intel(R) Core(TM) i5-8400H CPU @ 2.50GHz * off-the-shelf Java components... curious about official runtime results. thnx laptop results are around 11 seconds, e.g: ./calculate_average_3j5a.sh 81.46s user 1.36s system 758% cpu 10.917 total Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Address sizes: 39 bits physical, 48 bits virtual Byte Order: Little Endian CPU(s): 8 On-line CPU(s) list: 0-7 Vendor ID: GenuineIntel Model name: Intel(R) Core(TM) i5-8400H CPU @ 2.50GHz * off-the-shelf Java components + ArraysSupport.. laptop results are around 10.2 seconds, e.g: ./calculate_average_3j5a.sh 75.02s user 1.31s system 750% cpu 10.175 total Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Address sizes: 39 bits physical, 48 bits virtual Byte Order: Little Endian CPU(s): 8 On-line CPU(s) list: 0-7 Vendor ID: GenuineIntel Model name: Intel(R) Core(TM) i5-8400H CPU @ 2.50GHz * method handle... * full buffer read attempt * MH * MH cleanup
* 1brc contribution from mattiz (first attempt) (#567)Mathias Bjerke2024-01-231-0/+324
| | | | | * Contribution from mattiz * Formatted code
* fine tuning performance further (#526)karthikeyan972024-01-231-55/+46
| | | | | | | | | | | | | | | * final comit changing using mappedbytebuffer changes before using unsafe address using unsafe * using graalvm,correct unsafe mem implementation --------- Co-authored-by: Karthikeyans <karthikeyan.sn@zohocorp.com>
* Native image + a few smaller optimisations (#564)Roman Musin2024-01-231-133/+114
| | | | | | | | | * Inline parsing name and station to avoid constantly updating the offset field (-100ms) * Remove Worker class, inline the logic into lambda * Accumulate results in an int matrix instead of using result row (-50ms) * Use native image
generated by cgit v1.2.3 (git 2.25.1) at 2026-05-13 19:56:32 +0000