8000 Stream and Pipe benchmarks by johnhungerford · Pull Request #1300 · getkyo/kyo · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Stream and Pipe benchmarks #1300

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 3 commits into
base: main
Choose a base branch
from
Draft

Stream and Pipe benchmarks #1300

wants to merge 3 commits into from
+330 −0

Conversation

johnhungerford
Copy link
Collaborator

Addresses concerns raised in #1166 (see discussion here)

Problem

Pipe currently reimplements a bunch of the same functionality as Stream in a slightly different way. We need to figure out whether to maintain parallel implementations (in which case we would probably try to abstract the tests to a single reusable suite), have Stream use Pipe for its transformation method implementations, find some other way to reuse code between the two types, or rethink Pipe entirely.

What choice we make will depend largely on performance.

Solution

This PR (not meant to be merged in its current form) has some benchmarks for basic stream transformations on both Stream and Pipe (map, mapPure, filter, and filterPure). Results running locally on a macbook pro are posted below.

It also includes a POC attempt to abstract the common logic from Stream and Pipe methods. Both types essentially do the same thing, but one uses ArrowEffect.handleLoop and the other just uses Loop + Poll.andMap. I've added a StreamTransformations object that includes handleMap, handleMapPure, handleFilter, and handleFilterPure which can be used within ArrowEffect.handleLoop or Loop/Poll.andMap to provide a common implementation.

"Abstract" versions of filter and pure methods are included in the benchmark to see how abstracting the logic affects performance.

Notes

@johnhungerford
Copy link
Collaborator Author

Results run a 2021 Macbook Pro, M1, 64gb

[info] StreamPipeBench.filterKyoPipAbstractBench        thrpt   20    151.909 ±    0.976  ops/s
[info] StreamPipeBench.filterKyoPipeBench               thrpt   20    145.942 ±    4.067  ops/s
[info] StreamPipeBench.filterKyoStreamAbstractBench     thrpt   20   1304.232 ±   26.742  ops/s
[info] StreamPipeBench.filterKyoStreamBench             thrpt   20   1298.091 ±   20.668  ops/s
[info] StreamPipeBench.filterMapVarPipeAbstractBench    thrpt   20    397.788 ±    3.492  ops/s
[info] StreamPipeBench.filterMapVarPipeBench            thrpt   20    391.500 ±    4.405  ops/s
[info] StreamPipeBench.filterMapVarStreamAbstractBench  thrpt   20    460.290 ±    5.353  ops/s
[info] StreamPipeBench.filterMapVarStreamBench          thrpt   20    443.107 ±    1.927  ops/s
[info] StreamPipeBench.filterPurePipeAbstractBench      thrpt   20  13536.455 ±  109.865  ops/s
[info] StreamPipeBench.filterPurePipeBench              thrpt   20  13311.110 ±   90.330  ops/s
[info] StreamPipeBench.filterPureStreamAbstractBench    thrpt   20  13435.646 ±  201.809  ops/s
[info] StreamPipeBench.filterPureStreamBench            thrpt   20  14030.982 ±   49.366  ops/s
[info] StreamPipeBench.mapKyoPipeAbstractBench          thrpt   20   1131.097 ±   16.516  ops/s
[info] StreamPipeBench.mapKyoPipeBench                  thrpt   20    959.790 ±    8.838  ops/s
[info] StreamPipeBench.mapKyoStreamAbstractBench        thrpt   20   1198.594 ±   16.927  ops/s
[info] StreamPipeBench.mapKyoStreamBench                thrpt   20   1239.310 ±    4.699  ops/s
[info] StreamPipeBench.mapPurePipeAbstractBench         thrpt   20   7723.916 ±   40.677  ops/s
[info] StreamPipeBench.mapPurePipeBench                 thrpt   20   8047.511 ±   23.184  ops/s
[info] StreamPipeBench.mapPureStreamBench               thrpt   20   6639.025 ± 1269.321  ops/s
  • Benchmarks including -Abstract- use Stream and Pipe versions of filter and map with abstracted transformation logic
  • filterMapVar- benchmarks include filter and map transformations that include Var effects
  • I am really confused by the filterKyoPipe- benchmarks. I can't understand why it's slower than filterMapVar for instance...

@ahoy-jon ahoy-jon mentioned this pull request Jun 30, 2025
Merged
fwbrasil
fwbrasil reviewed Jun 30, 2025
View reviewed changes
kyo-bench/src/main/scala/kyo/bench/arena/StreamPipeBench.scala
import kyo.bench.BaseBench
import org.openjdk.jmh.annotations.*

class StreamPipeBench extends BaseBench:
Copy link
Collaborator
@fwbrasil fwbrasil Jun 30, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the arena package is for comparative benchmarks, this new one should be under the bench package directly

@fwbrasil
Copy link
Collaborator

Does it mean that the abstraction to share code has a minimal overhead? The better performance with Var can be just an artifact of how the code ends up JIT compiled since handles specialize execution

@johnhungerford
Copy link
Collaborator Author

Yeah I think the two main take-aways are: (1) in general Pipe transformations are less performant than their Stream counterparts, and (2) abstracting the transformations to a shared implementation does not meaningfully reduce the performance of either.

I'm going to experiment with some others, but I think the approach in this PR is my preferred way to deduplicate the code in Pipe and Stream. It looks to me like Pipe is performant enough in general that its worth keeping in its current form, but it's slow enough that we do not want to base the Stream transformation methods on it the way ZIO does.

@ahoy-jon ahoy-jon mentioned this pull request Jul 2, 2025
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@fwbrasil fwbrasil fwbrasil left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@johnhungerford @fwbrasil
0