8000 Win Techempower benchmarks · Issue #82 · haywire/haywire · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Win Techempower benchmarks #82

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
kellabyte opened this issue Jan 7, 2016 · 1 comment
Open

Win Techempower benchmarks #82

kellabyte opened this issue Jan 7, 2016 · 1 comment

Comments

@kellabyte
Copy link
Collaborator

Win Techempower benchmarks

Don't stop learning until we win. In Round 10 Haywire placed 9th.

Summary

I had the privilege to gain access to the Techempower environment and benchmark Haywire using the same parameters that Round 11 was run with. ulib is the winner of round 11 so I compared Haywire's experimental picohttpparser branch that includes the new HTTP parser with syscall response batching. This is not polished but it is an experimental branch to see how these changes compete.

The machine benchmarked against has 20 physical cores and 20 hyper-threaded cores. I tested Haywire in 20 and 40 thread configurations to see how this changes performance.

Concurrency: 256

ulib

 wrk -H 'Host: localhost' -H 'Accept: text/plain,text/html;q=0.9,application/xhtml+xml;q=0.9,application/xml;q=0.8,*/*;q=0.7' -H 'Connection: keep-alive' --latency -d 15 -c 256 --timeout 256 -t 40 http://10.0.3.2:8080/plaintext -s ~/pipeline.lua -- 16

Running 15s test @ http://10.0.3.2:8080/plaintext
  40 threads and 256 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   594.00us    3.17ms 405.57ms   99.33%
    Req/Sec   178.28k    53.42k  403.55k    67.22%
  Latency Distribution
     50%  354.00us
     75%  420.00us
     90%  467.00us
     99%  640.00us
  100721735 requests in 14.98s, 13.51GB read
Requests/sec: 6721980.23
Transfer/sec:      0.90GB

Haywire (20 threads)

Running 15s test @ http://10.0.3.2:8000
  40 threads and 256 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   605.36us  260.06us   2.08ms   74.72%
    Req/Sec   157.60k    27.12k  264.89k    66.86%
  Latency Distribution
     50%  536.00us
     75%  766.00us
     90%    0.96ms
     99%    1.57ms
  89564928 requests in 14.99s, 13.18GB read
  Socket errors: connect 0, read 0, write 0, timeout 66
Requests/sec: 5975704.85
Transfer/sec:      0.88GB

Haywire (40 threads)

Running 15s test @ http://10.0.3.2:8000
  40 threads and 256 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   269.42us  164.62us   1.56ms   74.87%
    Req/Sec    97.06k    66.96k  357.33k    65.90%
  Latency Distribution
     50%  251.00us
     75%  370.00us
     90%  449.00us
     99%  767.00us
  56847554 requests in 15.99s, 8.37GB read
  Socket errors: connect 0, read 0, write 0, timeout 1060
Requests/sec: 3554615.69
Transfer/sec:    535.61MB

Concurrency: 1024

ulib

 wrk -H 'Host: localhost' -H 'Accept: text/plain,text/html;q=0.9,application/xhtml+xml;q=0.9,application/xml;q=0.8,*/*;q=0.7' -H 'Connection: keep-alive' --latency -d 15 -c 1024 --timeout 1024 -t 40 http://10.0.3.2:8080/plaintext -s ~/pipeline.lua -- 16

Running 15s test @ http://10.0.3.2:8080/plaintext
  40 threads and 1024 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     5.13ms   44.84ms   1.51s    99.55%
    Req/Sec   156.46k    50.43k  413.33k    70.13%
  Latency Distribution
     50%    0.87ms
     75%    1.15ms
     90%    1.46ms
     99%   39.92ms
  88765885 requests in 14.99s, 11.90GB read
  Socket errors: connect 0, read 1, write 0, timeout 0
Requests/sec: 5922041.65
Transfer/sec:    813.27MB

Haywire (20 threads)

Running 15s test @ http://10.0.3.2:8000
  40 threads and 1024 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     2.64ms  771.11us 213.96ms   93.23%
    Req/Sec   157.78k    11.72k  234.67k    67.85%
  Latency Distribution
     50%    2.49ms
     75%    2.85ms
     90%    3.02ms
     99%    5.41ms
  89539488 requests in 14.99s, 13.18GB read
  Socket errors: connect 0, read 0, write 0, timeout 21
Requests/sec: 5973054.26
Transfer/sec:      0.88GB

Haywire (40 threads)

Running 15s test @ http://10.0.3.2:8000
  40 threads and 1024 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   787.66us    6.03ms   1.46s    99.99%
    Req/Sec   170.01k    47.79k  394.67k    68.75%
  Latency Distribution
     50%  653.00us
     75%    0.88ms
     90%    1.21ms
     99%    1.84ms
  96700770 requests in 14.99s, 14.23GB read
  Socket errors: connect 0, read 0, write 0, timeout 4176
Requests/sec: 6450812.55
Transfer/sec:      0.95GB

Concurrency: 4096

ulib

 wrk -H 'Host: localhost' -H 'Accept: text/plain,text/html;q=0.9,application/xhtml+xml;q=0.9,application/xml;q=0.8,*/*;q=0.7' -H 'Connection: keep-alive' --latency -d 15 -c 4096 --timeout 4096 -t 40 http://10.0.3.2:8080/plaintext -s ~/pipeline.lua -- 16

Running 15s test @ http://10.0.3.2:8080/plaintext
  40 threads and 4096 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     9.29ms   25.53ms   2.47s    90.20%
    Req/Sec   182.33k    29.57k  359.11k    72.88%
  Latency Distribution
     50%    4.89ms
     75%    7.24ms
     90%   28.42ms
     99%   47.94ms
  103143594 requests in 14.98s, 13.83GB read
  Socket errors: connect 0, read 0, write 0, timeout 665
Requests/sec: 6883446.57
Transfer/sec:      0.92GB

Haywire (20 threads)

Running 15s test @ http://10.0.3.2:8000
  40 threads and 4096 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    16.14ms  113.52ms   1.46s    98.71%
    Req/Sec   157.51k    81.82k  400.00k    59.40%
  Latency Distribution
     50%    5.47ms
     75%    7.01ms
     90%    8.91ms
     99%  213.06ms
  89297984 requests in 14.99s, 13.14GB read
  Socket errors: connect 0, read 0, write 0, timeout 12731
Requests/sec: 5958678.53
Transfer/sec:      0.88GB

Haywire (40 threads)

Running 15s test @ http://10.0.3.2:8000
  40 threads and 4096 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    33.71ms  206.60ms   6.46s    98.82%
    Req/Sec   179.26k    91.01k  529.78k    61.22%
  Latency Distribution
     50%    0.94ms
     75%    1.33ms
     90%   10.30ms
     99%    1.45s
  101397506 requests in 14.99s, 14.92GB read
  Socket errors: connect 0, read 0, write 0, timeout 16663
Requests/sec: 6764530.12
Transfer/sec:      1.00GB

Concurrency: 16384

ulib

 wrk -H 'Host: localhost' -H 'Accept: text/plain,text/html;q=0.9,application/xhtml+xml;q=0.9,application/xml;q=0.8,*/*;q=0.7' -H 'Connection: keep-alive' --latency -d 15 -c 16384 --timeout 16384 -t 40 http://10.0.3.2:8080/plaintext -s ~/pipeline.lua -- 16

Running 15s test @ http://10.0.3.2:8080/plaintext
  40 threads and 16384 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   109.86ms  780.16ms  12.40s    98.88%
    Req/Sec   166.08k    39.26k  307.31k    78.61%
  Latency Distribution
     50%   28.80ms
     75%   35.62ms
     90%   49.04ms
     99%    4.45s 
  99147082 requests in 14.96s, 13.30GB read
  Socket errors: connect 0, read 0, write 0, timeout 7707
Requests/sec: 6628593.11
Transfer/sec:      0.89GB

Haywire (20 threads)

Running 15s test @ http://10.0.3.2:8000
  40 threads and 16384 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    10.31ms   38.02ms 646.98ms   99.63%
    Req/Sec   147.54k    77.80k  464.00k    79.79%
  Latency Distribution
     50%    7.19ms
     75%    8.47ms
     90%   12.48ms
     99%   17.85ms
  87268400 requests in 14.97s, 12.84GB read
  Socket errors: connect 0, read 0, write 0, timeout 92384
Requests/sec: 5829863.60
Transfer/sec:      0.86GB

Haywire (40 threads)

Running 15s test @ http://10.0.3.2:8000
  40 threads and 16384 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    53.31ms  218.31ms   3.13s    96.84%
    Req/Sec   172.64k    95.18k  531.55k    68.02%
  Latency Distribution
     50%    1.82ms
     75%   17.80ms
     90%  207.12ms
     99%    1.46s
  98645652 requests in 14.98s, 14.52GB read
  Socket errors: connect 0, read 0, write 0, timeout 93786
Requests/sec: 6584461.64
Transfer/sec:      0.97GB
@shirmanov
Copy link

Why ulib is so performant?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@shirmanov @kellabyte
0