Generate related posts based on tags. Sort by the number of shared tags.
- Read the posts JSON file.
- Iterate over the posts and populate a map containing:
tag -> List<Post>that has that tag - Iterate over the posts and for each post:
- Create a map:
Post -> intto track the number of shared tags - For each tag, Iterate over the posts that have that tag
- For each post, increment the shared tag count in the map.
- Create a map:
- Sort the related posts by the number of shared tags.
- Write the top 5 related posts for each post to a new JSON file.
./run.sh go | rust | python | allRules
- FFI (including assembly inlining)
- Unsafe code blocks
- Custom benchmarking
- Disabling runtime checks (bounds etc)
- Parse json at runtime
- Not hardcore number of posts
- Support up to 100 tags
Updated Results from github workflow (raw data)
| Language | Processing Time | Total (PT + I/O) |
|---|---|---|
| Go Concurrent | 16.18 ms | 49.7 ms |
| Rust Rayon | 18.34 ms | 36.0 ms |
| Go | 30.75 ms | 60.8 ms |
| Rust | 33.15 ms | 51.1 ms |
| Java (GraalVM) | 47.00 ms | 81.8 ms |
| Crystal | 55.77 ms | 108.6 ms |
| Vlang | 58.42 ms | 377.5 ms |
| Odin | 63.42 ms | 311.5 ms |
| Swift | 64.37 ms | 481.2 ms |
| Zig | 179.15 ms | 220.2 ms |
| JS (Deno) | 182.40 ms | 284.8 ms |
| Dart VM | 185.00 ms | 660.9 ms |
| Nim | 200.83 ms | 303.2 ms |
| JS (Node) | 205.00 ms | 279.7 ms |
| Dart AOT | 226.75 ms | 369.2 ms |
| Java (JIT) | 243.08 ms | 518.5 ms |
| Julia v2 | 616.50 ms | 5.037 s |
| JS (Bun) | 773.80 ms | 853.3 ms |
| Numpy | 0.95 s | 1.335 s |
| Python | 2.72 s | 3.005 s |
Old Results with details (on my machine)
| Language | Processing Time | Total (+ I/O) | Details |
|---|---|---|---|
| Rust | - | 4.5s | Initial |
| Rust v2 | - | 2.60s | Replace std HashMap with fxHashMap by phazer99 |
| Rust v3 | - | 1.28s | Preallocate and reuse map and unstable sort by vdrmn and Darksonn |
| Rust v4 | - | 0.13s | Use Post index as key instead of Pointer and Binary Heap by RB5009 |
| Rust v5 | 38ms | 52ms | Rm hashing from loop and use vec[count] instead of map[index]count by RB5009 |
| Rust v6 | 23ms | 36ms | Optimized Binary Heap Ops by scottlamb |
| Rust Rayon | 9ms | 22ms | Parallelize by masmullin2000 |
| Rust Rayon | 8ms | 22ms | Remove comparison out of hot loop |
| ⠀ | ⠀ | ⠀ | ⠀ |
| Go | - | 1.5s | Initial |
| Go v2 | - | 80ms | Add rust optimizations |
| Go v3 | 56ms | 70ms | Use goccy/go-json |
| Go v3 | 34ms | 55ms | Use generic binaryheap by DrBlury |
| Go v4 | 26ms | 50ms | Replace binary heap with custom priority queue |
| Go v5 | 20ms | 43ms | Remove comparison out of hot loop |
| Go Con | 10ms | 33ms | Go concurrency by tirprox and DrBlury |
| Go Con v2 | 5ms | 29ms | Use arena, use waitgroup, rm binheap by DrBlury |
| ⠀ | ⠀ | ⠀ | ⠀ |
| Python | - | 7.81s | Initial |
| Python v2 | 1.35s | 1.53s | Add rust optimizations by dave-andersen |
| Numpy | 0.57s | 0.85s | Numpy implementation by Copper280z |
| ⠀ | ⠀ | ⠀ | ⠀ |
| Crystal | 50ms | 96ms | Inital w/ previous optimizations |
| Crystal v2 | 33ms | 72ms | Replace binary heap with custom priority queue |
| ⠀ | ⠀ | ⠀ | ⠀ |
| Odin | 110ms | 397ms | Ported from golang code |
| Odin v2 | 104ms | 404ms | Remove comparison out of hot loop |
| ⠀ | ⠀ | ⠀ | ⠀ |
| Dart VM | 125ms | 530ms | Ported frog golang code |
| Dart bin | 274ms | 360ms | Compiled executable |
| ⠀ | ⠀ | ⠀ | ⠀ |
| Vlang | 339ms | 560ms | Ported from golang code |
| ⠀ | ⠀ | ⠀ | ⠀ |
| Zig | 80ms | 110ms | Provided by akhildevelops |