aboutsummaryrefslogtreecommitdiff
path: root/src/main/java/dev/morling/onebrc
Commit message (Collapse)AuthorAgeFilesLines
* First version - roman_r_m (#193)Roman Musin2024-01-111-0/+280
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * initial commit * - use loop - use mutable object to store results * get rid of regex * Do not allocate measurement objects * MMap + custom double parsing ~ 1:30 (down from ~ 2:05) * HashMap for accumulation and only sort at the end - 1:05 * MMap the whole file * Use graal * no GC * Store results in an array list to avoid double map lookup * Adjust max buf size * Manual parsing number to long * Add --enable-preview * remove buffer size check (has no effect on performance) * fix min & max initialization * do not check for \r * Revert "do not check for \r" This reverts commit 9da1f574bf6261ea49c353488d3b4673cad3ce6e. * Optimise parsing. Now completes in 31 sec down from ~43 * trying to parse numbers faster * use open address hash table instead of the standard HashMap * formatting * Rename the script to match github username (change underscores to slashes) Enable transparent huge pages, seems to improve by ~2 sec * Revert "formatting" This reverts commit 4e90797d2a729ed7385c9000c85cc7e87d935f96. * Revert "use open address hash table instead of the standard HashMap" This reverts commit c784b55f61e48f548b2623e5c8958c9b283cae14. * add prepare_roman-r-m.sh * SWAR tricks to find semicolon (-2 seconds ro run time) * remove time call * fix test * Parallel version (~6.5 seconds)
* Add multithreaded generation of measurements file with Gaussian distribution ↵Wladimir Schmidt2024-01-111-0/+537
| | | | | | | | | | (M2 AIR took 24 seconds for 1B items) (#175) * Add multithreaded variant to generate measurements * Add removing existing measurements.txt file in case exists (for usability reasons) Fix bug for number of lines generated * Fix also for less than assumed chunk size (10M entries) per thread
* First attempt from MeanderingProgrammerMeanderingProgrammer2024-01-111-0/+250
| | | | | | | | | | | | | #### Check List: - [x] Tests pass (`./test.sh MeanderingProgrammer` shows no differences between expected and actual outputs) - [x] All formatting changes by the build are committed - [x] Your launch script is named `calculate_average_MeanderingProgrammer.sh` (make sure to match casing of your GH user name) and is executable - [x] Output matches that of `calculate_average_baseline.sh` * Execution time: `00:04.668` * Execution time of reference implementation: `02:40.597` * System: Apple M2 Max, 12 cores, 64 GB
* Basic Implementation with Memory Mapped File, SIMD SearchRoman Stoffel2024-01-111-0/+270
| | | | By gamlerhart
* create fork hundredwatt (#279)Jason Nochlin2024-01-111-0/+339
| | | Co-authored-by: Jason Nochlin <hundredwatt@users.noreply.github.com>
* Add code by maeda6uiuimaeda6uiui2024-01-111-0/+485
|
* Fixing the off-by-one error and updating to native, redone layout of code. ↵Roy van Rijn2024-01-111-202/+166
| | | | (#307)
* Leaderboard, formattingGunnar Morling2024-01-111-23/+23
|
* Update davecom entry use ByteBuffer instead of String for Names and graal (#245)David Kopec2024-01-111-20/+28
| | | | | | | | | * my implementation about 19.5 seconds on M1 MacBook Air * final version * use ByteBuffer instead of String; use graal * add prepare_davecom script
* Charlibot submission (#249)Charlie Evans2024-01-111-0/+332
| | | | | | | * committing my ugly code * fulfil the brief wrt number of characters and the number of cities * prepare submission
* CalculateAverage_kumarsaurav123.java-> RAM 16 GB CPU 32 Core (#247)kumarsaurav1232024-01-111-0/+256
|
* Simple, slow but very short. :-) (#240)Aleš Justin2024-01-111-0/+46
| | | | | * Simple, slow but very short. :-) * Fix sync on DSS::accept.
* improved artsiomkorzun solution (#176)Artsiom Korzun2024-01-111-195/+249
| | | | | | | improved artsiomkorzun solution improved artsiomkorzun solution Co-authored-by: Artsiom Korzun <akorzun@deltixlab.com>
* Implementation CalculateAverage_japplis of 1BRC from Anthony Goubard (#271)Anthony Goubard2024-01-101-0/+244
| | | | | | | | | | * Implementation CalculateAverage_japplis of 1BRC from Anthony Goubard (japplis). Local performance (7 years old desktop i7-6700K - 8 cores - 16GB) 26 seconds. For reference, Jamie Stansfield (isolgpus) is 23 seconds on my machine and 11s in your results. I've added the nbactions.xml to the .gitignore file. When you add in NetBeans options like --enable-preview to actions like debug file or run file, it creates this file. * Implementation CalculateAverage_japplis of 1BRC from Anthony Goubard (japplis). Local performance (7 years old desktop i7-6700K - 8 cores - 16GB) 26 seconds. For reference, Jamie Stansfield (isolgpus) is 23 seconds on my machine and 11s in your results. I've added the nbactions.xml to the .gitignore file. When you add in NetBeans options like --enable-preview to actions like debug file or run file, it creates this file. second commit: Removed BufferedInputStream and replaced Measurement with IntSummaryStatistics (thanks davecom): still 23" but cleaner code
* Add entries for pedestrianlove. (#265)pedestrianlove2024-01-101-0/+88
| | | Co-authored-by: jsl <s03851050@go.thu.edu.tw>
* first cut 1brc submission (#216)Pratham2024-01-101-0/+241
| | | | | | | | | | | | | * phd3 initial entry * Optimize parsing doubles * Remove redundant check * Update calculate_average_phd3.sh --------- Co-authored-by: Gunnar Morling <gunnar.morling@googlemail.com>
* Quick and dirty first version (#215)Algirdas Raščius2024-01-101-0/+272
| | | | | * Quick and dirty first version * Update script to new standard
* Initial solution by raipcAnton Rybochkin2024-01-101-0/+470
| | | | | | | | | | | | | | | * Initial solution by raipc * Implemented custom hash map with open addressing * Small optimizations to task splitting and range check disabling * Fixed off-by-one error in merge * Run with EpsilonGC. Borrowed VM params from Shipilev * Make script executable * Add a license
* Initial implementation (#158)Michael Berry2024-01-101-0/+268
| | | | | | | | | | | | | | | | | | | * Initial implementation * Make executable * Deal with collisions (but very slow) * Formatting * Remove superfluous time * Formatting * Fixes * Remove hard coded correction line * Integer parsing speedup
* Some optimizations arjenvaneerdearjenvaneerde2024-01-101-0/+348
| | | | | | | | | | | | | | | | | | | | | | | | | * First working version. * Small adjustments. * Correct number of threads. * Sync * Some fixes. To LF instead of CRLF. * Parallel reading and processing. * Update CreateMeasurements.java * Update CalculateAverage.java * Small fix for bug in switching buffers. * Update calculate_average_arjenvaneerde.sh --------- Co-authored-by: Gunnar Morling <gunnar.morling@googlemail.com>
* Implements CalculateAverage_zerninv.javazerninv2024-01-101-0/+199
| | | | | | | | | | | | | | | | | * initial attempt * adjust to new evaluation env * fix tests * fix typo * some numbers adjustments * Update calculate_average_zerninv.sh --------- Co-authored-by: Gunnar Morling <gunnar.morling@googlemail.com>
* CalculateAverage_AbstractKamenAbstractKamen2024-01-101-0/+220
| | | | | | | | | | | | | | | | | | | * initial commit * first attempt: segment the file and process it in parallel * remove commented stuff * custom parseDouble for this simple case * fixed some issues and improved parsing * format * Update calculate_average_AbstractKamen.sh --------- Co-authored-by: Gunnar Morling <gunnar.morling@googlemail.com>
* Leaderboard, clean-upGunnar Morling2024-01-101-0/+15
|
* gnmathur's solution (#202)Gaurav Mathur2024-01-101-0/+151
| | | Co-authored-by: Gaurav Mathur <gmathur@gurukul>
* Add CalculateAverage_couragelee Java class and shell scriptCourageLee2024-01-101-0/+336
| | | This commit introduces a new java class, CalculateAverage_couragelee, and a shell script for calculating averages. The java class utilizes NIO's memory-mapping and parallel computing techniques to perform calculations. These changes should improve the efficiency and speed of average calculations.
* Implementation by rprabhu Prabhu R2024-01-101-0/+137
| | | Co-authored-by: Prabhu R <prabhu.rengaswamy@outlook.com>
* gabrielreid take 2greid2024-01-101-162/+141
| | | | | Clear up some TODOS, simplify the code a bit, which appears to result in a 25% performance increase.
* Second submission by flippingbits - 50% performance improvementStefan Sprenger2024-01-101-92/+90
| | | | | | | | | | | * feat(flippingbits): Improve parsing of measurement and few cleanups * feat(flippingbits): Reduce chunk size to 10MB * feat(flippingbits): Improve parsing of station names * chore(flippingbits): Remove obsolete import * chore(flippingbits): Few cleanups
* merykitty's second attemptQuan Anh Mai2024-01-101-191/+138
|
* Consume four bytes at a time from buffer using getInt. Store key with unsafe ↵Elliot Barlas2024-01-101-83/+142
| | | | int array rather than byte array. Use custom equals rather than Arrays equals.
* Second tuning for thomaswueThomas Wuerthinger2024-01-101-96/+103
| | | | | | | | | | | | | | | * Optimize checking for collisions by doing this a long at a time always. * Use a long at a time scanning for delimiter. * Minor tuning. Now below 0.80s on Intel i9-13900K. * Add number parsing code from Quan Anh Mai. Fix name length issue. * Include suggestion from Alfonso Peterssen for another 1.5%. * Optimize hash collision check compare for ~4% gain. * Add perf stats based on latest version.
* Revert "Adding Nils Semmelrock's submission"Nils Semmelrock2024-01-101-271/+0
| | | | This reverts commit 12ae36ad
* Add davery22 implDaniel Avery2024-01-071-0/+315
|
* isolgpus: submission 2 - about a 25% improvement on submission 1. (#168)Jamie Stansfield2024-01-071-47/+51
| | | | | | | | | | | | | * isolgpus: fix chunk sizing when not at 8 threads use as many cores as are available don't buffer the station name, only use it when we need it. get rid of the main branch move variables inside the loop * isolgpus: optimistically assume we can read a whole int for the station name, but roll back if we get it wrong. This should be very beneficial on a dataset where station names are mostly over 4 chars --------- Co-authored-by: Jamie Stansfield <jalstansfield@gmail.com>
* Use SIMD for search for delimiter and name compareThomas Wuerthinger2024-01-071-52/+100
|
* Add yehwankim23 (#148)김예환 Ye-Hwan Kim (Sam)2024-01-071-0/+105
|
* My implementation is in dev.morling.onebrc.CalculateAverage_obourgain and ↵Olivier Bourgain2024-01-071-0/+481
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | runnable with provided script calculate_average_obourgain.sh (#75) Runs with standard JDK 21. On my computers (i5 13500, 20 cores, 32GB ram) my best run is (file fully in page cache): 49.78user 0.69system 0:02.81elapsed 1795%CPU A bit older version of the code on Mac pro M1 32 GB: real 0m2.867s user 0m23.956s sys 0m1.329s As I wrote in comments in the code, I have a few different roundings that the reference implementation. I have seend that there is an issue about that, but no specific rule yet. Main points: - use MemorySegment, it's faster than ByteBuffer - split the work in a lot of chunks and distribute to a thread pool - fast measurement parser by using a lot of domain knowledge - very low allocation - visit each byte only once Things I tried that were in fact pessimizations: - use some internal JDK code to vectorize the hashCode computation - use a MemorySegment to represent the keys instead of byte[], to avoid copying Hope I won't have a bad surprise when running on the target server 😱
* Roy: Adding a bit of unsafe...Roy van Rijn2024-01-071-254/+155
| | | Co-authored-by: Gunnar Morling <gunnar.morling@googlemail.com>
* first attemptags2024-01-071-0/+250
|
* Initial Implementation - coolmineman (#196)Cool_Mineman2024-01-071-0/+280
| | | | | | | | | | | | | | | | | | | | | * start * slower * still bad * finally faster than baseline :) * starting to go fast * improve * we ball * fix race condition an newline * change threadpool * ~18sec on my machine
* 1brc submission - Kevin McMurtrie (#195)Kevin McMurtrie2024-01-071-0/+516
| | | | | * v1 * Fix sorting
* An implementation optimised for simplicity/readability.John2024-01-071-0/+320
|
* Leaderboard updateGunnar Morling2024-01-071-2/+2
|
* CalculateAverage semotpan attempt 1Serghei Motpan2024-01-071-0/+199
|
* Implements CalculateAverage_santanu.javaSantanu Barua2024-01-071-0/+62
| | | | | | Execution time: 1 minute and 27.863 seconds System specs: Apple M1, 8 cores, 16GB RAM Co-authored-by: santanu barua <santanu@host109.private.net>
* Optimised Code to use FileSegments with ByteBuffer (#184)Keshavram Kuduwa2024-01-071-48/+156
| | | | | | | | | | | | | | | | | * Keshavram Kuduwa's Submission * Resolves #102 and Code Optimizations * Resolves #102 and Code Optimizations * Optimised Code with Roy's Reference * Fixed Tests * Clean Up Code --------- Co-authored-by: Keshavram Kuduwa <keshavram.kuduwa@apptware.com>
* Continue unrolling and inlining value parser. Make targeted use of ↵Elliot Barlas2024-01-071-43/+78
| | | | ByteBuffer.getInt() instead of ByteBuffer.get(). Switch from GraalVM CE to GraalVM. (#201)
* Add entry by Andrew SunAndrew Sun2024-01-071-0/+329
|
* Experiment from entangled90Carlo2024-01-071-0/+328
| | | | | | | | | | | | | | | | | | | | | * single thread memory mapped file reader, pool of processors * cleanup of inner classes of MetricProcessor * doubles are parsed without external functions, strings are lazily created from byte arrays * remove load() MappedByteBuffer in memory * fixed handling of newline * fix a bug & correct locale used * MappedByteBuffer size set to 1MB * fixed rounding * Do not use ArrayBlockingQueue.offer since it drops elements when queue is full * MappedByteBuffer size = 32 MB
* Adding kgeri's solution (#137)Gergely Kiss2024-01-071-0/+229
| | | | | | | | | | | | | | | | | | | * Adding kgeri's solution * parallelizing CalculatorAverage_kgeri * fixing aggregation bugs, chunk size calc for small files * removed GC logging Co-authored-by: Gunnar Morling <gunnar.morling@googlemail.com> * fix for when there's no newline at end of input * fix for when the final record ends on the chunk boundary --------- Co-authored-by: Gunnar Morling <gunnar.morling@googlemail.com>
generated by cgit v1.2.3 (git 2.25.1) at 2026-05-13 23:08:54 +0000