8000 fix(releasing): Update rpm tests by jszwedko · Pull Request #11140 · vectordotdev/vector · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

fix(releasing): Update rpm tests #11140

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Feb 1, 2022
Merged

fix(releasing): Update rpm tests #11140

merged 4 commits into from
Feb 1, 2022

Conversation

jszwedko
Copy link
Member
@jszwedko jszwedko commented Feb 1, 2022

Drop EOL distributions. This was prompted because the mirrors for Centos
8 no longer exist.

https://www.centos.org/centos-linux-eol/

Signed-off-by: Jesse Szwedko jesse@szwedko.me

Drop EOL distributions. This was prompted because the mirrors for Centos
8 no longer exist.

https://www.centos.org/centos-linux-eol/

Signed-off-by: Jesse Szwedko <jesse@szwedko.me>
@netlify
Copy link
netlify bot commented Feb 1, 2022

✔️ Deploy Preview for vector-project canceled.

🔨 Explore the source changes: 2d6f657

🔍 Inspect the deploy log: https://app.netlify.com/sites/vector-project/deploys/61f9ae58bdb8690008b39041

@jszwedko
Copy link
Member Author
jszwedko commented Feb 1, 2022

Kicked off nightly build here: https://github.com/vectordotdev/vector/actions/runs/1780415149

@github-actions
Copy link
github-actions bot commented Feb 1, 2022

Soak Test Results

Baseline: 078c316
Comparison: 78805b4
Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes with confidence ≥ 90.00%:

Fine details of change detection per experiment.
experiment Δ mean Δ mean % baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV p-value erratic declared erratic
http_to_http_acks 602.73KiB 6.61 8.9MiB 4.49MiB 187.55KiB 0 0.503733 9.49MiB 4.71MiB 195.1KiB 0 0.496388 0.0261194 True True
http_datadog_filter_blackhole 115.87KiB 5.56 2.04MiB 819.01KiB 33.27KiB 0 0.39258 2.15MiB 865.92KiB 35.18KiB 0 0.393212 0.0168581 True True
socket_to_socket_blackhole 169.85KiB 1.4 11.86MiB 184.33KiB 7.29KiB 0 0.015165 12.03MiB 195.19KiB 7.93KiB 0 0.015836 3.84333e-51 False False
syslog_humio_logs 124.67KiB 1.38 8.82MiB 98.46KiB 4.0KiB 0 0.0108891 8.94MiB 76.41KiB 3.09KiB 0 0.00833486 4.91651e-108 False False
datadog_agent_remap_datadog_logs_acks 430.47KiB 1.17 35.8MiB 416.17KiB 16.86KiB 0 0.0113438 36.22MiB 467.71KiB 18.58KiB 0 0.0126011 2.40347e-59 False False
syslog_splunk_hec_logs 88.74KiB 0.98 8.82MiB 99.7KiB 4.03KiB 0 0.0110322 8.9MiB 228.74KiB 9.34KiB 0 0.025064 1.48847e-17 False False
syslog_regex_logs2metric_ddmetrics 68.42KiB 0.96 6.99MiB 632.51KiB 28.66KiB 0 0.0883374 7.05MiB 331.92KiB 13.55KiB 0 0.0459268 0.0312642 False False
http_to_http_noack 108.69KiB 0.52 20.6MiB 944.55KiB 38.18KiB 0 0.0447375 20.71MiB 835.46KiB 32.5KiB 0 0.0393701 0.0303592 False False
splunk_hec_to_splunk_hec_logs_acks 52.93KiB 0.24 21.64MiB 1.08MiB 45.33KiB 0 0.0500712 21.69MiB 1.09MiB 45.07KiB 0 0.0500378 0.407815 False False
datadog_agent_remap_blackhole_acks 41.11KiB 0.13 30.57MiB 479.46KiB 19.57KiB 0 0.0153059 30.61MiB 357.04KiB 14.58KiB 0 0.0113829 0.0923524 False False
http_pipelines_blackhole 762.96B 0.1 759.77KiB 400.03KiB 16.43KiB 0 0.526069 760.51KiB 392.94KiB 16.03KiB 0 0.516251 0.974108 True True
fluent_remap_aws_firehose 48.48KiB 0.06 79.48MiB 77.6KiB 3.14KiB 0 0.000952707 79.52MiB 246.93KiB 10.01KiB 0 0.00302979 4.49499e-06 False False
fluent_elasticsearch 40.08KiB 0.05 79.43MiB 719.97KiB 29.1KiB 0 0.00884422 79.47MiB 53.65KiB 2.17KiB 0 0.000658776 0.170097 False False
splunk_hec_indexer_ack_blackhole 4.48KiB 0.02 23.84MiB 396.98KiB 16.21KiB 0 0.016247 23.85MiB 386.71KiB 15.79KiB 0 0.0158236 0.84322 False False
splunk_hec_route_s3 -10.77KiB -0.12 8.64MiB 2.0MiB 82.78KiB 0 0.231313 8.63MiB 2.04MiB 82.66KiB 0 0.2 8000 35955 0.926643 False False
datadog_agent_remap_datadog_logs -48.35KiB -0.13 35.21MiB 439.54KiB 17.94KiB 0 0.0121812 35.16MiB 591.95KiB 24.17KiB 0 0.0164271 0.108486 False False
syslog_loki -37.95KiB -0.54 6.9MiB 637.3KiB 27.81KiB 0 0.0900673 6.87MiB 240.51KiB 10.64KiB 0 0.0341737 0.202961 False False
http_pipelines_no_grok_blackhole -10.5KiB -0.59 1.74MiB 573.8KiB 23.31KiB 0 0.322112 1.73MiB 578.52KiB 23.52KiB 0 0.326689 0.75121 True False
syslog_log2metric_splunk_hec_metrics -55.48KiB -0.61 8.9MiB 59.27KiB 2.42KiB 0 0.00649743 8.85MiB 884.08KiB 39.9KiB 0 0.0974915 0.165792 False False
splunk_hec_to_splunk_hec_logs_noack -143.62KiB -0.62 22.78MiB 863.03KiB 35.12KiB 0 0.0369734 22.64MiB 884.67KiB 35.0KiB 0 0.0381368 0.00383625 False False
syslog_log2metric_humio_metrics -64.78KiB -0.81 7.79MiB 152.98KiB 6.21KiB 0 0.0191667 7.72MiB 604.31KiB 27.33KiB 0 0.0763179 0.0211746 False False
splunk_transforms_splunk3 -85.14KiB -1.62 5.15MiB 1.48MiB 61.41KiB 0 0.287407 5.06MiB 1.47MiB 61.06KiB 0 0.289961 0.325735 False False
datadog_agent_remap_blackhole -549.4KiB -1.77 30.28MiB 632.9KiB 25.71KiB 0 0.0203936 29.75MiB 594.87KiB 24.13KiB 0 0.0195138 5.04485e-50 False False
http_pipelines_blackhole_acks -27.82KiB -3.6 773.5KiB 439.78KiB 18.0KiB 0 0.56808 745.68KiB 435.38KiB 17.76KiB 0 0.583383 0.271445 True True
Fine details of each soak run.
(experiment, variant, run_id) total samples mean std min average p90 p95 p99 max skewness
('fluent_remap_aws_firehose', 'comparison', 'c1ebc29b-e6e0-4b32-9d92-459637bd3d0d') 203 79.62MiB 401.09KiB 79.28MiB 79.49MiB 80.31MiB 80.65MiB 80.97MiB 81.08MiB 2.31839
('fluent_remap_aws_firehose', 'baseline', 'd44f31fe-3636-4b3d-bf8e-aeff19dcb06e') 203 79.48MiB 107.92KiB 79.33MiB 79.48MiB 79.55MiB 79.56MiB 79.59MiB 80.69MiB 7.86081
('fluent_remap_aws_firehose', 'comparison', '071ed4ee-bcc4-4ce3-a5b9-cb142fb8af95') 203 79.47MiB 60.54KiB 79.31MiB 79.47MiB 79.56MiB 79.58MiB 79.62MiB 79.63MiB 0.196382
('fluent_remap_aws_firehose', 'baseline', '8c8f1c9f-a18f-48ae-a0fe-f97c73b1b08c') 203 79.47MiB 53.51KiB 79.31MiB 79.47MiB 79.54MiB 79.57MiB 79.6MiB 79.63MiB 0.215154
('fluent_elasticsearch', 'baseline', '378b7d4c-8d91-4c3a-9f82-f75c192077fb') 204 79.47MiB 56.84KiB 79.35MiB 79.47MiB 79.55MiB 79.57MiB 79.59MiB 79.59MiB 0.116631
('fluent_remap_aws_firehose', 'baseline', 'ff55f507-093b-499f-8a94-b4d8d6dd0fc8') 203 79.47MiB 59.75KiB 79.25MiB 79.47MiB 79.54MiB 79.56MiB 79.6MiB 79.69MiB 0.104167
('fluent_elasticsearch', 'comparison', 'dce386bf-3e0b-42c5-8697-5fd35f132b58') 204 79.47MiB 53.62KiB 79.34MiB 79.47MiB 79.54MiB 79.56MiB 79.58MiB 79.6MiB -0.0796895
('fluent_elasticsearch', 'comparison', '23742ae2-9e36-4eb0-a784-d647941bfc78') 204 79.47MiB 53.36KiB 79.36MiB 79.47MiB 79.54MiB 79.56MiB 79.6MiB 79.61MiB 0.228149
('fluent_remap_aws_firehose', 'comparison', 'e8ce5a07-572d-4acf-be1c-6a6962bc22f5') 203 79.47MiB 53.81KiB 79.31MiB 79.47MiB 79.54MiB 79.56MiB 79.6MiB 79.6MiB -0.0872149
('fluent_elasticsearch', 'baseline', 'dd52c0bc-409a-455e-a9fa-6a20954fae78') 204 79.47MiB 55.13KiB 79.3MiB 79.47MiB 79.54MiB 79.56MiB 79.58MiB 79.63MiB -0.113042
('fluent_elasticsearch', 'comparison', '411b49ae-f12c-45c4-9df3-5832eb98dcac') 204 79.47MiB 54.24KiB 79.3MiB 79.47MiB 79.55MiB 79.56MiB 79.59MiB 79.6MiB -0.0170642
('fluent_elasticsearch', 'baseline', '17ab487a-c1ff-4749-b75c-d1354f727d10') 204 79.35MiB 1.21MiB 76.59MiB 79.18MiB 80.76MiB 82.38MiB 82.96MiB 83.59MiB 1.36165
('datadog_agent_remap_datadog_logs_acks', 'comparison', 'effbf5e1-7fdd-4555-be58-f24544c79c7c') 202 36.59MiB 270.96KiB 35.82MiB 36.6MiB 36.91MiB 37.05MiB 37.12MiB 37.21MiB -0.312075
('datadog_agent_remap_datadog_logs_acks', 'comparison', 'a35f6476-5709-4635-a4f2-d7a92d88cd1d') 230 36.35MiB 318.39KiB 35.44MiB 36.35MiB 36.78MiB 36.89MiB 36.98MiB 37.14MiB -0.0186598
('datadog_agent_remap_datadog_logs_acks', 'baseline', '917d79ae-d379-48ff-8959-70f7bdfebec1') 203 36.04MiB 494.55KiB 35.02MiB 36.06MiB 36.65MiB 36.78MiB 37.02MiB 37.22MiB 0.00951914
('datadog_agent_remap_datadog_logs_acks', 'baseline', 'c2c33270-0972-4f2a-a844-4a099b206e71') 203 35.83MiB 277.19KiB 35.31MiB 35.81MiB 36.15MiB 36.27MiB 36.77MiB 37.09MiB 1.17064
('datadog_agent_remap_datadog_logs_acks', 'comparison', '836b9872-c051-4366-beac-874212a1cac0') 202 35.69MiB 204.8KiB 35.14MiB 35.71MiB 35.96MiB 36.02MiB 36.11MiB 36.31MiB -0.0832975
('datadog_agent_remap_datadog_logs', 'comparison', 'b668f7a8-3850-4b6b-b29f-7c1f00717966') 200 35.68MiB 195.9KiB 35.14MiB 35.68MiB 35.92MiB 35.95MiB 36.08MiB 36.12MiB -0.0705905
('datadog_agent_remap_datadog_logs_acks', 'baseline', 'b3639887-9845-474d-b6cc-4e8f9c0170b4') 203 35.52MiB 234.67KiB 34.92MiB 35.54MiB 35.8MiB 35.88MiB 36.07MiB 36.24MiB 0.0653884
('datadog_agent_remap_datadog_logs', 'baseline', '0bc03de5-4f41-4c5f-94d3-a3bf5ff417af') 200 35.48MiB 185.65KiB 34.94MiB 35.48MiB 35.72MiB 35.77MiB 35.91MiB 36.1MiB 0.147326
('datadog_agent_remap_datadog_logs', 'baseline', 'efff75df-ba05-484e-89d6-b5228224ff5c') 200 35.46MiB 301.01KiB 34.88MiB 35.41MiB 35.86MiB 36.0MiB 36.12MiB 36.33MiB 0.472823
('datadog_agent_remap_datadog_logs', 'comparison', '9c6e3aef-f8dc-407f-842a-56478dc2fc82') 200 35.32MiB 449.48KiB 34.77MiB 35.18MiB 36.12MiB 36.35MiB 36.65MiB 36.72MiB 1.51192
('datadog_agent_remap_datadog_logs', 'baseline', '70f9cd77-21a3-47c3-930d-570adfc89979') 200 34.69MiB 191.26KiB 34.17MiB 34.69MiB 34.93MiB 35.01MiB 35.1MiB 35.23MiB 0.0418103
('datadog_agent_remap_datadog_logs', 'comparison', '94d2b66c-e3c1-4e3d-93c8-690feffdbdcd') 200 34.49MiB 196.86KiB 33.64MiB 34.49MiB 34.73MiB 34.8MiB 34.89MiB 34.91MiB -0.442017
('datadog_agent_remap_blackhole_acks', 'baseline', '9ddab528-11d2-43ff-8c7e-a8d17bd7f663') 200 30.89MiB 210.67KiB 30.34MiB 30.87MiB 31.16MiB 31.26MiB 31.47MiB 31.61MiB 0.459932
('datadog_agent_remap_blackhole_acks', 'comparison', '55383e86-737b-42e3-82f3-ae1445b5ac4c') 200 30.83MiB 323.15KiB 29.65MiB 30.81MiB 31.25MiB 31.37MiB 31.54MiB 31.63MiB 0.0248422
('datadog_agent_remap_blackhole_acks', 'baseline', 'a7eeb772-0044-416a-99b1-5aff7926ab8d') 200 30.82MiB 226.28KiB 30.06MiB 30.82MiB 31.08MiB 31.24MiB 31.32MiB 31.44MiB -0.0802927
('datadog_agent_remap_blackhole', 'baseline', '58114a3b-70cd-46b5-b506-139cdb8473fe') 202 30.7MiB 317.13KiB 29.85MiB 30.72MiB 31.09MiB 31.2MiB 31.32MiB 31.9MiB 0.0951916
('datadog_agent_remap_blackhole_acks', 'comparison', '04dcbec9-808a-4326-bc72-eabe293d8cb8') 200 30.68MiB 262.6KiB 30.03MiB 30.66MiB 31.04MiB 31.12MiB 31.28MiB 31.43MiB 0.284253
('datadog_agent_remap_blackhole', 'baseline', 'da14ee41-6e41-4203-b6b1-36604b205ccf') 202 30.54MiB 504.09KiB 29.72MiB 30.4MiB 31.23MiB 31.32MiB 31.48MiB 31.61MiB 0.23378
('datadog_agent_remap_blackhole_acks', 'comparison', 'e4782d29-bb88-4417-8248-5cf6580cc68c') 200 30.3MiB 236.36KiB 29.64MiB 30.36MiB 30.57MiB 30.64MiB 30.73MiB 30.81MiB -0.455698
('datadog_agent_remap_blackhole', 'comparison', '2c416a29-de05-41ed-b66f-1537732e2b77') 206 30.09MiB 363.98KiB 26.15MiB 30.12MiB 30.39MiB 30.43MiB 30.6MiB 31.28MiB -6.51052
('datadog_agent_remap_blackhole', 'comparison', 'd4a3c5e7-24c6-4273-b422-f35dfa05b1b9') 201 30.02MiB 358.27KiB 29.34MiB 29.97MiB 30.36MiB 30.83MiB 31.19MiB 31.37MiB 1.4784
('datadog_agent_remap_blackhole_acks', 'baseline', '8979b441-ee06-4800-ab6d-ef62d69a5f89') 200 29.98MiB 234.3KiB 29.47MiB 29.98MiB 30.31MiB 30.36MiB 30.4MiB 30.49MiB 0.0350688
('datadog_agent_remap_blackhole', 'baseline', 'd6bb6295-fe2e-4308-85a7-51b35b072221') 202 29.6MiB 334.15KiB 28.73MiB 29.6MiB 30.04MiB 30.18MiB 30.4MiB 30.59MiB 0.322197
('datadog_agent_remap_blackhole', 'comparison', 'e5c59564-7e04-4cfc-85f8-46a7c43fa099') 201 29.11MiB 410.35KiB 28.31MiB 29.07MiB 29.75MiB 29.95MiB 30.15MiB 30.31MiB 0.773108
('splunk_hec_indexer_ack_blackhole', 'comparison', 'bd639d56-2070-43ae-80d1-b4e7fae23a85') 200 23.85MiB 378.35KiB 22.91MiB 23.85MiB 24.32MiB 24.4MiB 24.63MiB 24.73MiB -0.212637
('splunk_hec_indexer_ack_blackhole', 'comparison', 'a102e76e-bd63-4405-a616-b0ca80d5f07c') 200 23.85MiB 383.33KiB 22.9MiB 23.87MiB 24.29MiB 24.4MiB 24.63MiB 24.72MiB -0.25415
('splunk_hec_indexer_ack_blackhole', 'comparison', '0b3efe56-6233-4f2f-b747-0bf483116383') 200 23.84MiB 400.05KiB 22.9MiB 23.87MiB 24.32MiB 24.4MiB 24.6MiB 24.69MiB -0.315597
('splunk_hec_indexer_ack_blackhole', 'baseline', 'b400dbc7-1807-4821-97fe-0f3e589f1cb4') 200 23.84MiB 384.28KiB 22.96MiB 23.84MiB 24.33MiB 24.4MiB 24.55MiB 24.66MiB -0.156056
('splunk_hec_indexer_ack_blackhole', 'baseline', 'bb57b703-0b9a-43ec-9de9-729d5563551c') 200 23.84MiB 415.11KiB 22.91MiB 23.84MiB 24.39MiB 24.51MiB 24.67MiB 24.72MiB -0.0813498
('splunk_hec_indexer_ack_blackhole', 'baseline', 'a13468bc-38ab-4347-97ca-5bdc73bee0f6') 200 23.84MiB 392.9KiB 22.76MiB 23.85MiB 24.31MiB 24.39MiB 24.6MiB 24.63MiB -0.296291
('splunk_hec_to_splunk_hec_logs_noack', 'comparison', '4cde9428-b09a-47a2-ae09-9a1a6a6118cd') 204 23.0MiB 804.98KiB 20.84MiB 22.94MiB 23.99MiB 24.27MiB 24.74MiB 24.81MiB -0.209541
('splunk_hec_to_splunk_hec_logs_noack', 'baseline', 'ebb3b305-064f-4610-a6ce-cbd6185e9a4e') 200 22.93MiB 840.04KiB 20.78MiB 23.04MiB 23.94MiB 24.15MiB 24.51MiB 24.75MiB -0.487307
('splunk_hec_to_splunk_hec_logs_noack', 'baseline', '94f39549-7456-46b8-bbd0-bee0802619eb') 200 22.72MiB 909.28KiB 20.71MiB 22.78MiB 23.87MiB 24.03MiB 24.57MiB 25.13MiB -0.141337
('splunk_hec_to_splunk_hec_logs_noack', 'baseline', '5bf9cdfc-fd92-4c63-b73c-6f6032c30b20') 204 22.68MiB 820.53KiB 20.56MiB 22.76MiB 23.75MiB 23.93MiB 24.07MiB 24.51MiB -0.264673
('splunk_hec_to_splunk_hec_logs_noack', 'comparison', '0c7e379a-89fd-4b98-b900-25a6509b03aa') 231 22.47MiB 846.56KiB 20.57MiB 22.45MiB 23.5MiB 23.75MiB 24.13MiB 24.43MiB -0.0841683
('splunk_hec_to_splunk_hec_logs_noack', 'comparison', 'a3824030-fb08-455e-9a3f-37ded3ff3d7a') 204 22.46MiB 893.27KiB 19.31MiB 22.54MiB 23.52MiB 23.72MiB 24.18MiB 24.48MiB -0.399027
('splunk_hec_to_splunk_hec_logs_acks', 'baseline', 'c70fd99c-7f09-4bd1-b728-66941dc44eb3') 200 21.84MiB 1.05MiB 19.16MiB 21.96MiB 23.05MiB 23.38MiB 23.73MiB 23.85MiB -0.40145
('splunk_hec_to_splunk_hec_logs_acks', 'comparison', '7a44d82b-71c4-4d79-8f5d-2470695f2364') 203 21.75MiB 1.13MiB 18.94MiB 21.83MiB 23.1MiB 23.27MiB 23.75MiB 24.42MiB -0.343172
('splunk_hec_to_splunk_hec_logs_acks', 'comparison', '4963056c-9f76-4005-a2f4-d4944a78e3e8') 203 21.66MiB 1.06MiB 19.0MiB 21.75MiB 22.93MiB 23.11MiB 23.38MiB 23.73MiB -0.531047
('splunk_hec_to_splunk_hec_logs_acks', 'comparison', 'a4b3e916-a0ef-4d8d-97da-cc74f09ef96b') 203 21.66MiB 1.07MiB 18.88MiB 21.72MiB 22.93MiB 23.1MiB 23.64MiB 23.88MiB -0.451215
('splunk_hec_to_splunk_hec_logs_acks', 'baseline', 'a540fd79-cd56-4978-860c-b0652eadc1dd') 200 21.54MiB 1.11MiB 18.53MiB 21.69MiB 22.84MiB 23.02MiB 23.6MiB 23.85MiB -0.454705
('splunk_hec_to_splunk_hec_logs_acks', 'baseline', '3c028860-3651-41d0-937e-ead8b3ad2cc9') 200 21.53MiB 1.07MiB 18.59MiB 21.66MiB 22.92MiB 23.14MiB 23.49MiB 23.61MiB -0.388367
('http_to_http_noack', 'comparison', 'd66dc592-2d27-4d09-ab12-0843e425ebce') 228 20.79MiB 809.23KiB 19.0MiB 20.69MiB 21.9MiB 22.1MiB 22.94MiB 23.01MiB 0.368627
('http_to_http_noack', 'comparison', '8d373a4c-84d4-4d25-891b-adf64ce87dbc') 204 20.71MiB 912.53KiB 18.87MiB 20.63MiB 22.04MiB 22.31MiB 22.9MiB 22.97MiB 0.379965
('http_to_http_noack', 'baseline', '591763aa-493b-41de-9c95-5853ed77ae79') 204 20.64MiB 835.98KiB 18.35MiB 20.63MiB 21.77MiB 21.99MiB 22.25MiB 22.47MiB 0.0124837
('http_to_http_noack', 'comparison', '91006d77-8985-4fa9-b7e5-4ea2b4dc4bb9') 229 20.62MiB 782.17KiB 19.0MiB 20.63MiB 21.64MiB 21.83MiB 22.22MiB 23.35MiB 0.182291
('http_to_http_noack', 'baseline', '70159fed-e346-4a0a-8bb8-44bb01967deb') 204 20.62MiB 1.17MiB 15.95MiB 20.6MiB 22.09MiB 22.51MiB 22.97MiB 23.49MiB -0.617112
('http_to_http_noack', 'baseline', 'bdc16ff4-8606-4851-b592-6d1ac81f0b07') 204 20.55MiB 742.03KiB 19.0MiB 20.49MiB 21.48MiB 21.76MiB 22.2MiB 22.73MiB 0.187294
('socket_to_socket_blackhole', 'comparison', 'e1e8d8fc-bd09-4be4-babb-5ad156b5be17') 202 12.28MiB 31.65KiB 12.2MiB 12.28MiB 12.31MiB 12.33MiB 12.35MiB 12.35MiB -0.320479
('socket_to_socket_blackhole', 'baseline', '4f88bb9b-cd1e-47f5-97bc-502088e70c72') 202 12.1MiB 43.51KiB 12.0MiB 12.08MiB 12.15MiB 12.17MiB 12.22MiB 12.23MiB 0.312025
('socket_to_socket_blackhole', 'comparison', '00d4bc40-386f-4b37-a5a4-f0b7198a7076') 202 11.94MiB 79.85KiB 11.69MiB 11.93MiB 12.03MiB 12.06MiB 12.13MiB 12.17MiB -0.00140082
('socket_to_socket_blackhole', 'comparison', '831c4e68-c5cf-4c94-aea5-d54699f62c19') 202 11.86MiB 31.05KiB 11.76MiB 11.86MiB 11.9MiB 11.91MiB 11.93MiB 11.94MiB -0.0872682
('socket_to_socket_blackhole', 'baseline', '767b1d06-8956-4e98-ae46-7b2b3b346633') 235 11.81MiB 37.25KiB 11.73MiB 11.8MiB 11.85MiB 11.86MiB 11.89MiB 11.91MiB 0.178741
('socket_to_socket_blackhole', 'baseline', '8f2452b4-0c3b-418d-bed0-6bc9e508e174') 202 11.68MiB 81.6KiB 11.45MiB 11.69MiB 11.79MiB 11.81MiB 11.86MiB 11.91MiB -0.0127908
('http_to_http_acks', 'comparison', 'abc058e3-4449-4b91-8c12-f47519bcbbac') 204 9.78MiB 4.9MiB 1.73MiB 9.58MiB 17.55MiB 18.66MiB 19.66MiB 20.14MiB 0.372707
('http_to_http_acks', 'comparison', 'ec8a112f-91ae-428b-b217-2d8b359baf59') 204 9.52MiB 4.52MiB 1.97MiB 9.22MiB 16.69MiB 18.69MiB 19.03MiB 19.62MiB 0.46524
('http_to_http_acks', 'baseline', '8238b228-aab3-4462-b1a2-fc949c088948') 200 9.39MiB 4.74MiB 2.31MiB 9.51MiB 17.68MiB 18.84MiB 20.03MiB 20.27MiB 0.59277
('http_to_http_acks', 'comparison', 'b19709d3-96f3-4428-9abd-7e8e1020243f') 204 9.16MiB 4.72MiB 1006.19KiB 9.25MiB 17.28MiB 17.7MiB 19.18MiB 20.02MiB 0.52795
('syslog_splunk_hec_logs', 'comparison', 'ab2b3d50-65c0-4ceb-b1cb-29aac3380c08') 200 9.13MiB 27.51KiB 9.06MiB 9.13MiB 9.17MiB 9.17MiB 9.18MiB 9.19MiB -0.012766
('http_to_http_acks', 'baseline', '82b5c005-9b85-4f1a-99ee-1bb18ed4ce3c') 200 9.1MiB 4.79MiB 1.55MiB 9.16MiB 17.08MiB 18.44MiB 19.56MiB 20.06MiB 0.598726
('syslog_humio_logs', 'comparison', 'c54b268c-00a9-4635-a78b-0bfeae253cdb') 204 9.02MiB 51.54KiB 8.9MiB 9.01MiB 9.1MiB 9.12MiB 9.14MiB 9.16MiB 0.466006
('syslog_splunk_hec_logs', 'comparison', 'e9014260-cee6-4df5-8126-f94f73904f0f') 200 8.98MiB 34.55KiB 8.87MiB 8.98MiB 9.01MiB 9.03MiB 9.04MiB 9.06MiB -0.389653
('syslog_log2metric_splunk_hec_metrics', 'comparison', '4482b7c9-6ab8-4c64-8683-4101bb621fbe') 200 8.95MiB 31.21KiB 8.87MiB 8.95MiB 8.99MiB 9.0MiB 9.01MiB 9.05MiB 0.0366402
('syslog_log2metric_splunk_hec_metrics', 'baseline', 'd3d7454d-9891-49df-96c9-ce44ed6d6b4c') 200 8.93MiB 41.12KiB 8.84MiB 8.94MiB 8.99MiB 9.0MiB 9.01MiB 9.04MiB 0.101326
('syslog_humio_logs', 'baseline', '46182097-6a11-456c-8b5c-adc9ce88dd77') 202 8.93MiB 55.0KiB 8.82MiB 8.92MiB 9.02MiB 9.05MiB 9.09MiB 9.1MiB 1.2604
('syslog_log2metric_splunk_hec_metrics', 'baseline', 'ea32d7e8-e949-4d36-bd2e-27e5b986fdfc') 200 8.92MiB 55.33KiB 8.75MiB 8.93MiB 8.98MiB 8.99MiB 9.02MiB 9.03MiB -0.945859
('syslog_humio_logs', 'comparison', 'ee80a4a7-2762-45dd-b98c-9c4d7f0b8d5f') 204 8.91MiB 64.78KiB 8.7MiB 8.91MiB 8.98MiB 9.0MiB 9.09MiB 9.14MiB 0.221744
('syslog_humio_logs', 'comparison', 'df17869e-fa4b-448d-b66f-cd92e1d9d74b') 204 8.9MiB 30.21KiB 8.79MiB 8.9MiB 8.94MiB 8.94MiB 8.97MiB 8.98MiB -0.647166
('syslog_splunk_hec_logs', 'baseline', '4e7268fa-2f97-4f1d-8f2c-e9c9fc952eb8') 204 8.9MiB 71.28KiB 8.79MiB 8.88MiB 9.02MiB 9.03MiB 9.05MiB 9.07MiB 0.887415
('syslog_log2metric_splunk_hec_metrics', 'comparison', 'c516afaa-7800-4e40-8129-bd595c99680e') 200 8.88MiB 34.83KiB 8.8MiB 8.87MiB 8.92MiB 8.94MiB 8.95MiB 8.98MiB 0.268018
('syslog_log2metric_splunk_hec_metrics', 'baseline', 'ca249cb6-7f30-4a93-bfc8-4065af3423ab') 200 8.85MiB 32.63KiB 8.77MiB 8.84MiB 8.88MiB 8.9MiB 8.92MiB 8.93MiB 0.00877311
('syslog_humio_logs', 'baseline', 'f0dd833b-49ba-4ba1-af79-74ed965ac25f') 202 8.83MiB 29.1KiB 8.75MiB 8.83MiB 8.87MiB 8.88MiB 8.89MiB 8.9MiB -0.137809
('syslog_splunk_hec_logs', 'baseline', 'e3170b48-0694-4788-a487-ce5d7cff9f8d') 204 8.81MiB 92.35KiB 8.56MiB 8.81MiB 8.91MiB 8.94MiB 8.97MiB 9.05MiB -0.227095
('splunk_hec_route_s3', 'comparison', '97d1c079-32de-468c-9bfb-2dfcedf1b9e2') 204 8.8MiB 1.99MiB 5.15MiB 8.65MiB 11.45MiB 11.78MiB 12.84MiB 13.57MiB 0.154918
('splunk_hec_route_s3', 'baseline', 'e66f04cb-316e-443c-86fa-4b8589af757a') 204 8.75MiB 2.02MiB 5.29MiB 8.54MiB 11.45MiB 12.05MiB 12.88MiB 14.36MiB 0.266995
('syslog_splunk_hec_logs', 'baseline', 'c878b4f4-f169-407d-8a0c-bdf890116776') 204 8.75MiB 62.93KiB 8.55MiB 8.76MiB 8.81MiB 8.83MiB 8.87MiB 8.89MiB -0.61629
('syslog_humio_logs', 'baseline', 'a5dc87e7-523c-46ad-9ae3-05c3e8474cb8') 202 8.71MiB 28.04KiB 8.64MiB 8.71MiB 8.75MiB 8.76MiB 8.77MiB 8.78MiB 0.00231572
('splunk_hec_route_s3', 'comparison', 'feecf9bc-49a9-4e1a-b06b-d3d3693bee58') 204 8.63MiB 2.09MiB 5.21MiB 8.31MiB 11.6MiB 12.01MiB 13.35MiB 14.81MiB 0.375997
('syslog_splunk_hec_logs', 'comparison', 'a3d294d1-0690-4ada-bc00-ddccdcfa3dce') 200 8.61MiB 59.08KiB 8.5MiB 8.6MiB 8.65MiB 8.69MiB 8.83MiB 8.85MiB 2.30306
('splunk_hec_route_s3', 'baseline', '60bc92f3-8155-4d9b-9b3b-82e9b43a8717') 204 8.6MiB 2.06MiB 4.65MiB 8.34MiB 11.63MiB 12.02MiB 12.59MiB 12.98MiB 0.260374
('splunk_hec_route_s3', 'baseline', '50f2baa4-9220-47de-86ef-802a60d23cba') 204 8.57MiB 1.92MiB 5.17MiB 8.39MiB 11.23MiB 11.67MiB 12.66MiB 12.86MiB 0.212814
('syslog_log2metric_splunk_hec_metrics', 'comparison', 'eb9e181e-3d31-4e9c-88db-a89ba8ed70dc') 91 8.56MiB 1.99MiB 12.37KiB 9.05MiB 9.09MiB 9.1MiB 9.11MiB 9.11MiB -3.95649
('splunk_hec_route_s3', 'comparison', 'fe386b24-2692-44f4-a697-e4c36fea28dd') 229 8.47MiB 2.02MiB 4.86MiB 8.39MiB 11.21MiB 11.71MiB 12.39MiB 13.44MiB 0.133135
('http_to_http_acks', 'baseline', '900c786f-21e2-4b52-90a1-d5bf99b2af9a') 200 8.2MiB 3.8MiB 2.25MiB 8.15MiB 13.48MiB 15.13MiB 18.43MiB 19.05MiB 0.542939
('syslog_log2metric_humio_metrics', 'baseline', '5de996cf-0580-4ba4-ba05-2edaad0970ec') 202 7.95MiB 34.29KiB 7.88MiB 7.95MiB 7.99MiB 8.0MiB 8.05MiB 8.11MiB 0.952215
('syslog_log2metric_humio_metrics', 'comparison', 'ee7868f6-c264-4feb-a7f1-aca5505486d1') 200 7.88MiB 29.58KiB 7.82MiB 7.88MiB 7.92MiB 7.93MiB 7.94MiB 7.95MiB -0.0487199
('syslog_log2metric_humio_metrics', 'baseline', 'ea72b9c6-c2cd-41ba-bd29-cb9cb52a0323') 202 7.74MiB 32.55KiB 7.65MiB 7.74MiB 7.78MiB 7.78MiB 7.82MiB 7.84MiB 0.118278
('syslog_log2metric_humio_metrics', 'comparison', 'f3eed845-603b-4d82-a38d-808f151a5fec') 200 7.67MiB 32.09KiB 7.56MiB 7.67MiB 7.71MiB 7.72MiB 7.76MiB 7.77MiB -0.0767107
('syslog_log2metric_humio_metrics', 'baseline', '20cd7b9c-e775-4d21-a350-7806967bf876') 202 7.67MiB 151.15KiB 7.23MiB 7.68MiB 7.85MiB 7.9MiB 7.97MiB 8.0MiB -0.24183
('syslog_log2metric_humio_metrics', 'comparison', '0d0a4409-183e-4cff-91d3-8d7fb4dc6bd8') 89 7.48MiB 1.34MiB 12.73KiB 7.75MiB 7.78MiB 7.8MiB 7.81MiB 7.81MiB -5.08875
('syslog_regex_logs2metric_ddmetrics', 'comparison', '655ee211-61ec-4e49-af2b-8beef15c2e08') 200 7.19MiB 306.62KiB 6.74MiB 7.22MiB 7.54MiB 7.55MiB 7.59MiB 7.67MiB -0.118691
('syslog_regex_logs2metric_ddmetrics', 'comparison', '7aebea49-3a83-4285-acd4-fd0c30b8eed8') 200 7.1MiB 302.7KiB 6.62MiB 7.15MiB 7.43MiB 7.47MiB 7.5MiB 7.53MiB -0.226888
('syslog_loki', 'baseline', 'e7f4497b-b5f8-4482-8b7b-78281726ce2d') 202 7.04MiB 26.57KiB 6.98MiB 7.04MiB 7.08MiB 7.08MiB 7.1MiB 7.1MiB -0.204421
('syslog_regex_logs2metric_ddmetrics', 'baseline', 'afd29015-5fd7-49c7-8bf0-6f0f37aaf95d') 200 7.04MiB 300.84KiB 6.57MiB 7.13MiB 7.35MiB 7.36MiB 7.38MiB 7.44MiB -0.328251
('syslog_regex_logs2metric_ddmetrics', 'baseline', '78c9b1e0-f7e9-4e98-82a5-e4c040a1d03d') 200 6.98MiB 290.96 8000 KiB 6.48MiB 7.08MiB 7.28MiB 7.3MiB 7.33MiB 7.35MiB -0.427664
('syslog_loki', 'comparison', 'a901c543-ed18-4f9f-825c-ec0e6d2ff19a') 200 6.97MiB 27.79KiB 6.89MiB 6.97MiB 7.0MiB 7.01MiB 7.03MiB 7.05MiB -0.135444
('syslog_loki', 'baseline', 'a52e2236-f0dc-41cc-9947-f626d17b4ef5') 235 6.92MiB 32.94KiB 6.82MiB 6.92MiB 6.96MiB 6.97MiB 7.0MiB 7.09MiB 0.548236
('syslog_loki', 'comparison', 'caab2cbe-7eed-4ce9-b141-30b3949b9806') 86 6.9MiB 512.64KiB 2.45MiB 6.95MiB 7.02MiB 7.03MiB 7.05MiB 7.05MiB -8.55947
('syslog_regex_logs2metric_ddmetrics', 'baseline', '7c40e7c4-01f1-4c5f-9d40-58cc3c515df1') 87 6.87MiB 1.32MiB 49.72KiB 7.17MiB 7.48MiB 7.5MiB 7.55MiB 7.55MiB -4.45714
('syslog_regex_logs2metric_ddmetrics', 'comparison', '80d9a313-7332-4218-8448-50b9dcf7807d') 200 6.87MiB 295.64KiB 6.32MiB 7.02MiB 7.15MiB 7.24MiB 7.27MiB 7.27MiB -0.529372
('syslog_loki', 'comparison', '9d6a9c26-75f4-432f-be31-785d94fe92ae') 225 6.76MiB 97.94KiB 6.49MiB 6.77MiB 6.88MiB 6.92MiB 6.96MiB 6.97MiB -0.203646
('syslog_loki', 'baseline', 'e019bc36-c224-421d-83d2-b0df8917c6a0') 88 6.54MiB 1.46MiB 13.09KiB 6.88MiB 6.93MiB 6.95MiB 6.99MiB 6.99MiB -4.14798
('splunk_transforms_splunk3', 'baseline', 'aceb9bdd-63d4-4e22-80b5-c04ff46d54ca') 203 5.21MiB 1.48MiB 2.5MiB 4.92MiB 7.27MiB 7.65MiB 8.28MiB 8.94MiB 0.283402
('splunk_transforms_splunk3', 'baseline', '1ea8f252-fd56-4a90-a553-729f51fe2c12') 203 5.2MiB 1.44MiB 2.53MiB 4.98MiB 7.2MiB 7.63MiB 8.01MiB 8.51MiB 0.253154
('splunk_transforms_splunk3', 'comparison', 'a97679f6-696f-4447-ae59-fc6bdd6c8298') 203 5.13MiB 1.45MiB 2.09MiB 4.83MiB 7.09MiB 7.56MiB 7.92MiB 9.31MiB 0.266251
('splunk_transforms_splunk3', 'comparison', 'b662bb05-66f4-4cae-9197-a362cb69bb4a') 202 5.06MiB 1.49MiB 1.97MiB 4.83MiB 7.08MiB 7.48MiB 8.12MiB 8.6MiB 0.269371
('splunk_transforms_splunk3', 'baseline', '2aef1261-3009-4579-b90a-a4bb19171845') 203 5.03MiB 1.52MiB 2.59MiB 4.69MiB 7.21MiB 7.7MiB 8.69MiB 9.09MiB 0.492695
('splunk_transforms_splunk3', 'comparison', '7385b7df-e2b2-4ad3-b66a-caafe49a31a6') 202 5.0MiB 1.47MiB 2.34MiB 4.86MiB 7.12MiB 7.43MiB 8.49MiB 9.07MiB 0.409655
('http_datadog_filter_blackhole', 'comparison', '6a5c7f07-1e22-4262-9b47-22bfb175d707') 202 2.17MiB 858.05KiB 260.99KiB 2.04MiB 3.33MiB 3.59MiB 4.15MiB 4.49MiB 0.362935
('http_datadog_filter_blackhole', 'baseline', '5a8e89ed-4a17-4139-93e6-d74f731b2100') 202 2.15MiB 838.39KiB 490.01KiB 2.05MiB 3.23MiB 3.63MiB 3.84MiB 4.43MiB 0.278847
('http_datadog_filter_blackhole', 'comparison', '20f18d48-73b3-4df3-ae76-dd63dd6a64dd') 202 2.14MiB 875.91KiB 260.99KiB 2.09MiB 3.32MiB 3.65MiB 4.02MiB 4.18MiB 0.232802
('http_datadog_filter_blackhole', 'comparison', 'b87053f5-4a25-4bfd-83da-942c19cdf821') 202 2.14MiB 867.68KiB 292.97KiB 2.07MiB 3.31MiB 3.56MiB 3.99MiB 4.49MiB 0.196657
('http_datadog_filter_blackhole', 'baseline', '28228f17-7b09-4896-8270-391cd1025882') 202 2.08MiB 794.16KiB 276.98KiB 1.99MiB 3.19MiB 3.49MiB 3.83MiB 3.85MiB 0.220653
('http_datadog_filter_blackhole', 'baseline', '615e2c03-7b80-4835-984a-e48c361b450e') 202 1.88MiB 802.04KiB 276.98KiB 1.76MiB 2.96MiB 3.27MiB 3.57MiB 4.11MiB 0.38504
('http_pipelines_no_grok_blackhole', 'comparison', '6b2fbb49-4b00-48de-9a45-5bd1e60908b4') 200 1.79MiB 564.73KiB 524.14KiB 1.79MiB 2.47MiB 2.78MiB 3.37MiB 3.6MiB 0.385391
('http_pipelines_no_grok_blackhole', 'baseline', '2b50f123-6c92-47d9-b586-dd9ad98d6737') 206 1.77MiB 606.79KiB 323.87KiB 1.8MiB 2.53MiB 2.72MiB 3.15MiB 3.37MiB 0.0773848
('http_pipelines_no_grok_blackhole', 'baseline', '7122dc99-5150-4732-9615-685eca356511') 200 1.73MiB 549.49KiB 232.25KiB 1.71MiB 2.46MiB 2.63MiB 2.91MiB 3.46MiB 0.111491
('http_pipelines_no_grok_blackhole', 'baseline', '497399bf-7299-4d06-ab6b-613acab0bdda') 200 1.71MiB 563.98KiB 322.78KiB 1.72MiB 2.38MiB 2.66MiB 2.79MiB 3.08MiB -0.0612884
('http_pipelines_no_grok_blackhole', 'comparison', '615bf011-bd26-4718-b230-a0e12922f44d') 200 1.7MiB 568.7KiB 201.36KiB 1.67MiB 2.38MiB 2.65MiB 3.12MiB 3.38MiB 0.33233
('http_pipelines_no_grok_blackhole', 'comparison', 'b2565559-118b-494b-b4dc-58c7decec801') 205 1.69MiB 597.94KiB 354.24KiB 1.62MiB 2.48MiB 2.68MiB 2.98MiB 3.19MiB 0.226625
('http_pipelines_blackhole_acks', 'baseline', 'd6fede85-10b8-42af-8b01-2de0f9cb9fc7') 200 828.48KiB 458.63KiB 77.78KiB 764.22KiB 1.41MiB 1.66MiB 2.09MiB 2.24MiB 0.776644
('http_pipelines_blackhole_acks', 'comparison', 'bfbc725c-e2e6-4e94-9d7f-a6cc03d9b8e8') 201 786.65KiB 472.71KiB 15.99KiB 738.25KiB 1.39MiB 1.57MiB 1.97MiB 2.51MiB 0.701049
('http_pipelines_blackhole', 'comparison', '586ee29e-fcca-4ff6-bd6f-a400214eff94') 200 768.29KiB 412.72KiB 30.89KiB 733.33KiB 1.25MiB 1.49MiB 1.88MiB 2.15MiB 0.786213
('http_pipelines_blackhole', 'comparison', 'cb80040b-d54d-4be3-8a9b-a04f98e1dce8') 200 765.72KiB 367.9KiB 15.99KiB 738.22KiB 1.2MiB 1.47MiB 1.73MiB 1.84MiB 0.67239
('http_pipelines_blackhole', 'baseline', 'c8c21094-313c-4345-a8db-3c2fad5b0474') 199 763.68KiB 423.98KiB 15.99KiB 691.38KiB 1.27MiB 1.5MiB 1.86MiB 2.27MiB 0.805823
('http_pipelines_blackhole', 'baseline', '01a658fa-d99d-44ba-8d8c-ae067fbc716e') 198 759.13KiB 389.33KiB 31.98KiB 750.68KiB 1.2MiB 1.44MiB 1.73MiB 2.09MiB 0.585959
('http_pipelines_blackhole', 'baseline', '546d6f02-6907-4c06-a4ac-73f4e85c85bc') 196 756.45KiB 387.47KiB 31.98KiB 737.96KiB 1.27MiB 1.42MiB 1.6MiB 1.63MiB 0.230804
('http_pipelines_blackhole_acks', 'baseline', 'dc8a190b-3712-40d0-8bb8-e59cfaa8bc5a') 199 749.31KiB 454.49KiB 15.99KiB 733.33KiB 1.28MiB 1.53MiB 1.96MiB 2.42MiB 0.670547
('http_pipelines_blackhole', 'comparison', '62ece5a5-10c0-4a4d-9dd0-f42ebd5a43aa') 201 747.6KiB 398.5KiB 62.35KiB 689.68KiB 1.29MiB 1.46MiB 1.69MiB 2.09MiB 0.651097
('http_pipelines_blackhole_acks', 'baseline', 'cfc6c479-7efb-4f32-a710-bb85e0805463') 198 742.26KiB 400.48KiB 15.99KiB 736.86KiB 1.24MiB 1.38MiB 1.53MiB 1.6MiB 0.142303
('http_pipelines_blackhole_acks', 'comparison', '2cf4c61c-6551-4a7b-9f61-06f52bf2bc31') 201 733.77KiB 416.52KiB 15.99KiB 690.81KiB 1.25MiB 1.41MiB 1.79MiB 2.17MiB 0.651542
('http_pipelines_blackhole_acks', 'comparison', 'd3609d10-aa82-4604-859d-1189e177ded1') 199 716.31KiB 413.2KiB 30.89KiB 673.12KiB 1.24MiB 1.42MiB 1.8MiB 2.06MiB 0.722003

Signed-off-by: Jesse Szwedko <jesse@szwedko.me>
Signed-off-by: Jesse Szwedko <jesse@szwedko.me>
@jszwedko
Copy link
Member Author
jszwedko commented Feb 1, 2022

Kicked off second run here: https://github.com/vectordotdev/vector/actions/runs/1780643147

8000
@github-actions
Copy link
github-actions bot commented Feb 1, 2022

Soak Test Results

Baseline: 3f31469
Comparison: 89f4d69
Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes with confidence ≥ 90.00%:

Fine details of change detection per experiment.
experiment Δ mean Δ mean % baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV p-value erratic declared erratic
http_datadog_filter_blackhole 123.31KiB 6.01 2.0MiB 806.66KiB 32.77KiB 0 0.39306 2.12MiB 839.48KiB 34.1KiB 0 0.385849 0.00923835 True True
syslog_humio_logs 364.03KiB 4.12 8.64MiB 1.1MiB 51.0KiB 0 0.127362 8.99MiB 74.98KiB 3.03KiB 0 0.00813479 3.72375e-12 False False
syslog_splunk_hec_logs 176.23KiB 1.96 8.79MiB 140.41KiB 5.68KiB 0 0.0155851 8.96MiB 106.08KiB 4.33KiB 0 0.0115484 4.48529e-108 False False
http_pipelines_no_grok_blackhole 32.8KiB 1.83 1.75MiB 596.1KiB 24.34KiB 0 0.332767 1.78MiB 582.75KiB 23.79KiB 0 0.31946 0.335319 True False
syslog_log2metric_humio_metrics 109.24KiB 1.39 7.68MiB 113.69KiB 4.62KiB 0 0.014451 7.78MiB 190.4KiB 7.77KiB 0 0.0238692 2.01308e-31 False False
datadog_agent_remap_datadog_logs_acks 288.24KiB 0.8 35.38MiB 401.82KiB 16.28KiB 0 0.0110825 35.66MiB 599.42KiB 24.35KiB 0 0.0164018 6.38483e-22 False False
splunk_transforms_splunk3 36.6KiB 0.7 5.08MiB 1.43MiB 59.53KiB 0 0.282061 5.12MiB 1.49MiB 61.79KiB 0 0.290266 0.669779 False False
datadog_agent_remap_blackhole_acks 123.07KiB 0.39 30.46MiB 555.55KiB 22.68KiB 0 0.0177963 30.58MiB 368.93KiB 15.06KiB 0 0.0117718 6.87825e-06 False False
splunk_hec_to_splunk_hec_logs_acks 56.7KiB 0.26 21.5MiB 1.22MiB 49.75KiB 0 0.0565902 21.55MiB 1.14MiB 47.21KiB 0 0.0527365 0.408583 False False
fluent_remap_aws_firehose 8.62KiB 0.01 79.58MiB 523.76KiB 20.77KiB 0 0.00642203 79.59MiB 618.39KiB 25.06KiB 0 0.0075813 0.79123 False False
splunk_hec_indexer_ack_blackhole 1.63KiB 0.01 23.84MiB 384.6KiB 15.7KiB 0 0.0157399 23.84MiB 383.65KiB 15.66KiB 0 0.0157001 0.941524 False False
fluent_elasticsearch -87.82KiB -0.11 79.56MiB 547.15KiB 21.61KiB 0 0.00671093 79.47MiB 53.31KiB 2.15KiB 0 0.000654501 5.89658e-05 False False
socket_to_socket_blackhole -76.05KiB -0.62 11.89MiB 343.33KiB 13.95KiB 0 0.0281715 11.82MiB 792.78KiB 35.74KiB 0 0.0654462 0.0478939 False False
syslog_log2metric_splunk_hec_metrics -71.8KiB -0.79 8.89MiB 954.64KiB 43.62KiB 0 0.104815 8.81MiB 194.85KiB 7.95KiB 0 0.0215685 0.105999 False False
splunk_hec_route_s3 -77.08KiB -0.86 8.79MiB 2.08MiB 85.98KiB 0 0.236349 8.71MiB 2.01MiB 83.26KiB 0 0.23066 0.519679 False False
splunk_hec_to_splunk_hec_logs_noack -246.65KiB -1.07 22.59MiB 885.95KiB 36.17KiB 0 0.0382751 22.34MiB 835.77KiB 33.78KiB 0 0.036497 7.15368e-07 False False
http_to_http_noack -253.39KiB -1.19 20.86MiB 894.68KiB 36.14KiB 0 0.0418412 20.62MiB 854.72KiB 34.52KiB 0 0.0404519 4.58566e-07 False False
datadog_agent_remap_blackhole -376.97KiB -1.22 30.14MiB 410.91KiB 16.69KiB 0 0.0133033 29.77MiB 646.35KiB 26.21KiB 0 0.0211846 9.52863e-32 False False
http_pipelines_blackhole_acks -12.28KiB -1.63 751.54KiB 440.88KiB 17.98KiB 0 0.58614 739.27KiB 444.73KiB 18.14KiB 0 0.601083 0.63091 True True
syslog_loki -128.67KiB -1.8 6.98MiB 51.35KiB 2.09KiB 0 0.00717299 6.86MiB 117.99KiB 4.82KiB 0 0.0167857 7.13083e-100 False False
http_pipelines_blackhole -15.44KiB -1.99 775.92KiB 410.86KiB 16.91KiB 0 0.529066 760.48KiB 437.6KiB 17.84KiB 0 0.574943 0.530142 True True
syslog_regex_logs2metric_ddmetrics -203.35KiB -2.79 7.13MiB 331.61KiB 13.54KiB 0 0.0454038 6.93MiB 327.8KiB 13.38KiB 0 0.0461697 1.6622e-25 False False
http_to_http_acks -267.91KiB -2.83 9.24MiB 4.53MiB 188.48KiB 0 0.490199 8.97MiB 4.2MiB 173.81KiB 0 0.467524 0.296252 True True
datadog_agent_remap_datadog_logs -1.31MiB -3.61 36.4MiB 433.56KiB 17.6KiB 0 0.0116222 35.09MiB 567.31KiB 23.06KiB 0 0.0157775 5.65056e-264 False False
Fine details of each soak run.
(experiment, variant, run_id) total samples mean std min median p90 p95 p99 max skewness
('fluent_elasticsearch', 'baseline', '9fb9d04d-02d7-415f-874a-8df2590e4686') 233 79.71MiB 885.27KiB 79.32MiB 79.48MiB 79.57MiB 82.46MiB 83.42MiB 83.49MiB 3.47237
('fluent_remap_aws_firehose', 'baseline', '0e66b0cb-1c43-4486-9e69-d2a03223dd9d') 231 79.7MiB 784.66KiB 79.34MiB 79.48MiB 79.58MiB 82.07MiB 82.72MiB 82.97MiB 3.13464
('fluent_remap_aws_firehose', 'comparison', 'f788cbb5-f99f-4b4e-8c79-b15741bce44f') 203 79.7MiB 885.84KiB 74.01MiB 79.48MiB 80.9MiB 81.74MiB 82.61MiB 82.97MiB 0.625118
('fluent_remap_aws_firehose', 'comparison', 'c2445fcf-ff58-4081-a877-f4225279424b') 203 79.6MiB 578.14KiB 78.42MiB 79.56MiB 80.35MiB 80.62MiB 80.78MiB 80.96MiB 0.199024
('fluent_remap_aws_firehose', 'baseline', '0951a70a-7b24-4ffd-a2d0-a040c7f63aed') 202 79.56MiB 357.91KiB 79.34MiB 79.47MiB 79.57MiB 80.17MiB 81.29MiB 81.52MiB 4.24506
('fluent_elasticsearch', 'comparison', '6303cc2b-adcb-41f3-b0c6-5d511c7f7ca8') 204 79.47MiB 55.33KiB 79.34MiB 79.47MiB 79.55MiB 79.57MiB 79.59MiB 79.59MiB 0.0156052
('fluent_elasticsearch', 'comparison', 'a14e2355-dd17-45f3-8b3c-1f49ed87b3d6') 204 79.47MiB 51.53KiB 79.33MiB 79.47MiB 79.54MiB 79.56MiB 79.59MiB 79.61MiB 0.0950859
('fluent_elasticsearch', 'baseline', '36fcc76f-a0a4-48b4-a8e7-3f05335dc59d') 204 79.47MiB 51.4KiB 79.33MiB 79.47MiB 79.54MiB 79.56MiB 79.58MiB 79.6MiB 0.00048064
('fluent_remap_aws_firehose', 'baseline', 'a1c90f8f-e4c7-402a-b7b5-984060b1bb5a') 203 79.47MiB 54.23KiB 79.33MiB 79.47MiB 79.54MiB 79.57MiB 79.59MiB 79.6MiB 0.0363315
('fluent_remap_aws_firehose', 'comparison', 'd1e82755-7e65-4529-bb3d-67c2e25a9bfc') 203 79.47MiB 65.64KiB 79.07MiB 79.47MiB 79.54MiB 79.56MiB 79.59MiB 79.88MiB 0.0522861
('fluent_elasticsearch', 'baseline', '4711823f-3174-4165-b9eb-70584e56edd3') 204 79.47MiB 52.44KiB 79.37MiB 79.47MiB 79.54MiB 79.56MiB 79.59MiB 79.6MiB 0.180441
('fluent_elasticsearch', 'comparison', '449bf248-b254-4c10-8de3-678b1c7afc0e') 204 79.47MiB 53.25KiB 79.32MiB 79.47MiB 79.55MiB 79.56MiB 79.58MiB 79.59MiB -0.039843
('datadog_agent_remap_datadog_logs', 'baseline', '5f09ee36-6eb0-4b7d-af6b-a0f3e2e8b457') 200 36.77MiB 207.19KiB 36.19MiB 36.77MiB 37.02MiB 37.09MiB 37.2MiB 37.26MiB -0.0378928
('datadog_agent_remap_datadog_logs', 'baseline', 'e78e6ab8-c360-47cb-9d23-5cd50cafd47c') 201 36.29MiB 226.54KiB 35.75MiB 36.3MiB 36.56MiB 36.64MiB 36.84MiB 37.1MiB 0.241507
('datadog_agent_remap_datadog_logs_acks', 'comparison', 'c2a589a9-cd94-4d95-879b-dd8cb5c4848a') 202 36.25MiB 297.53KiB 35.57MiB 36.25MiB 36.62MiB 36.71MiB 36.89MiB 36.93MiB 0.0383243
('datadog_agent_remap_datadog_logs', 'baseline', '8a514119-8cb1-40c3-a9df-88dd0d0fb4b3') 206 36.15MiB 492.73KiB 31.18MiB 36.14MiB 36.63MiB 36.71MiB 36.89MiB 37.02MiB -5.28189
('datadog_agent_remap_datadog_logs_acks', 'comparison', '8c67eab5-fb0a-4900-a5e0-fa336d6b2d91') 202 35.74MiB 292.23KiB 35.08MiB 35.71MiB 36.03MiB 36.2MiB 36.65MiB 37.03MiB 0.989849
('datadog_agent_remap_datadog_logs', 'comparison', '9701b45b-65f1-4899-bc80-236808fb7a1a') 205 35.68MiB 366.81KiB 31.53MiB 35.69MiB 35.92MiB 36.02MiB 36.17MiB 37.07MiB -7.24496
('datadog_agent_remap_datadog_logs_acks', 'baseline', '35bf8f41-517d-4331-a6c6-68b6f5af2394') 203 35.57MiB 231.36KiB 34.84MiB 35.59MiB 35.83MiB 35.87MiB 36.07MiB 36.32MiB -0.162746
('datadog_agent_remap_datadog_logs_acks', 'baseline', '2081808c-8b98-4189-b12c-0b429158f89f') 203 35.43MiB 274.52KiB 34.67MiB 35.43MiB 35.74MiB 35.81MiB 35.97MiB 36.15MiB -0.100441
('datadog_agent_remap_datadog_logs_acks', 'baseline', '97f44634-95cb-456b-9635-caf32c861907') 203 35.14MiB 508.99KiB 29.54MiB 35.16MiB 35.57MiB 35.74MiB 35.86MiB 35.89MiB -7.06465
('datadog_agent_remap_datadog_logs', 'comparison', '14a136a8-8cf1-40da-9750-119464ae9feb') 200 35.08MiB 204.39KiB 34.58MiB 35.09MiB 35.36MiB 35.43MiB 35.56MiB 35.61MiB 0.151006
('datadog_agent_remap_datadog_logs_acks', 'comparison', '78d5d999-92ff-45bb-bc43-a316fcf532e7') 202 34.99MiB 234.79KiB 34.41MiB 34.98MiB 35.29MiB 35.36MiB 35.51MiB 35.63MiB 0.204149
('datadog_agent_remap_datadog_logs', 'comparison', 'db201b77-9c0d-4d2c-886d-6ae26618c5ba') 200 34.48MiB 184.24KiB 33.93MiB 34.49MiB 34.7MiB 34.76MiB 34.84MiB 34.95MiB -0.301621
('datadog_agent_remap_blackhole_acks', 'baseline', '3fcdc159-09a0-47cb-a9aa-aa36a5e2f616') 200 30.86MiB 284.33KiB 29.98MiB 30.85MiB 31.19MiB 31.35MiB 31.59MiB 31.74MiB 0.340254
('datadog_agent_remap_blackhole_acks', 'comparison', '5248b1d8-b533-46ee-a922-8a9497edaa0e') 200 30.83MiB 284.83KiB 30.22MiB 30.81MiB 31.21MiB 31.27MiB 31.48MiB 31.76MiB 0.237995
('datadog_agent_remap_blackhole_acks', 'baseline', '0562843b-c8bf-44a2-8d27-395e6206cee3') 200 30.74MiB 226.27KiB 29.9MiB 30.76MiB 30.99MiB 31.07MiB 31.27MiB 31.41MiB -0.220789
('datadog_agent_remap_blackhole_acks', 'comparison', '6f514280-c0c1-4e9d-b2b4-c55be6d99bf1') 200 30.7MiB 214.99KiB 29.89MiB 30.72MiB 30.93MiB 31.0MiB 31.14MiB 31.17MiB -0.574357
('datadog_agent_remap_blackhole', 'baseline', '51e39fce-2422-49c5-a7ce-f5edc70f7caa') 202 30.55MiB 192.95KiB 30.04MiB 30.56MiB 30.8MiB 30.86MiB 30.98MiB 31.01MiB -0.150141
('datadog_agent_remap_blackhole_acks', 'comparison', 'aca5d6bc-06dd-43a6-8f16-41a18b8d1b47') 200 30.21MiB 242.93KiB 29.58MiB 30.23MiB 30.5MiB 30.57MiB 30.63MiB 30.67MiB -0.298018
('datadog_agent_remap_blackhole', 'comparison', '8a0e8327-179e-4e64-876c-359bbd618bd8') 201 30.18MiB 205.07KiB 29.57MiB 30.19MiB 30.43MiB 30.52MiB 30.58MiB 30.82MiB -0.0606792
('datadog_agent_remap_blackhole', 'comparison', '1bfe33a6-1331-465f-8f18-1387bd23b61b') 201 30.17MiB 201.2KiB 29.64MiB 30.17MiB 30.41MiB 30.49MiB 30.64MiB 30.67MiB -0.0360431
('datadog_agent_remap_blackhole', 'baseline', 'a26732bf-0e1b-46d6-aa97-817a4fb4b3b8') 202 30.17MiB 204.69KiB 29.66MiB 30.18MiB 30.43MiB 30.47MiB 30.59MiB 30.63MiB -0.207499
('datadog_agent_remap_blackhole_acks', 'baseline', '5ec3e340-9a5b-4627-b7ae-e61b76ea092d') 200 29.78MiB 238.17KiB 29.32MiB 29.75MiB 30.11MiB 30.2MiB 30.28MiB 30.35MiB 0.355285
('datadog_agent_remap_blackhole', 'baseline', 'b0a29d37-7359-4542-8c3c-ca52284d5f9c') 202 29.69MiB 188.21KiB 29.24MiB 29.68MiB 29.93MiB 30.04MiB 30.14MiB 30.2MiB 0.350889
('datadog_agent_remap_blackhole', 'comparison', 'c131b7d8-e09a-45bb-a0cb-d7abda4b7a59') 206 28.98MiB 386.65KiB 25.43MiB 29.0MiB 29.36MiB 29.46MiB 29.7MiB 30.06MiB -3.85937
('splunk_hec_indexer_ack_blackhole', 'baseline', '7ccfb8e2-d11c-48d0-bc0f-9257fb099558') 200 23.85MiB 393.72KiB 22.96MiB 23.85MiB 24.33MiB 24.49MiB 24.63MiB 24.75MiB -0.0678899
('splunk_hec_indexer_ack_blackhole', 'comparison', '65a51a23-5bbb-4523-9dd9-8c502bff81d2') 200 23.85MiB 383.6KiB 22.91MiB 23.84MiB 24.33MiB 24.43MiB 24.63MiB 24.64MiB -0.169837
('splunk_hec_indexer_ack_blackhole', 'comparison', '68e07fc8-2876-4757-bdca-ab641e5cdcd7') 200 23.84MiB 382.75KiB 23.02MiB 23.85MiB 24.32MiB 24.43MiB 24.63MiB 24.72MiB -0.185608
('splunk_hec_indexer_ack_blackhole', 'baseline', '79d853fc-a62c-4340-b7a2-48202ac2c796') 200 23.84MiB 382.72KiB 22.9MiB 23.87MiB 24.29MiB 24.39MiB 24.63MiB 24.73MiB -0.195852
('splunk_hec_indexer_ack_blackhole', 'comparison', '4e5c4d04-cb0e-4c9b-9e1b-d7c2c403f27a') 200 23.84MiB 386.52KiB 22.96MiB 23.84MiB 24.31MiB 24.39MiB 24.52MiB 24.67MiB -0.266951
('splunk_hec_indexer_ack_blackhole', 'baseline', 'adb4c419-1802-4879-b60b-4b8de1b4b930') 200 23.84MiB 379.03KiB 22.96MiB 23.84MiB 24.31MiB 24.43MiB 24.52MiB 24.64MiB -0.157658
('splunk_hec_to_splunk_hec_logs_noack', 'baseline', 'ae8f8e56-87fd-46a2-bf67-b6d55b40e4d0') 200 23.02MiB 849.37KiB 20.95MiB 23.06MiB 24.06MiB 24.21MiB 24.39MiB 24.72MiB -0.353439
('splunk_hec_to_splunk_hec_logs_noack', 'comparison', '07746e14-0051-472c-b87c-cff38d8abc04') 204 22.65MiB 851.67KiB 20.69MiB 22.68MiB 23.63MiB 23.86MiB 24.45MiB 24.6MiB -0.246121
('splunk_hec_to_splunk_hec_logs_noack', 'baseline', '88f2ddd0-c4ec-430f-98f3-5f5f0bde3955') 200 22.39MiB 832.51KiB 20.54MiB 22.36MiB 23.41MiB 23.52MiB 23.82MiB 23.99MiB -0.189598
('splunk_hec_to_splunk_hec_logs_noack', 'baseline', 'ad1cdca8-e413-47c2-b653-6a0de664cbcc') 200 22.34MiB 802.68KiB 20.57MiB 22.3MiB 23.44MiB 23.73MiB 23.85MiB 23.97MiB -0.0784976
('splunk_hec_to_splunk_hec_logs_acks', 'baseline', '3e981dcb-7720-44d3-8f05-fbb2a5ffcd95') 200 22.2MiB 1.15MiB 19.19MiB 22.42MiB 23.5MiB 23.73MiB 24.21MiB 24.48MiB -0.537554
('splunk_hec_to_splunk_hec_logs_noack', 'comparison', '4bccc82b-0b76-4f8d-a957-1d0bb07f114e') 204 22.2MiB 749.84KiB 20.44MiB 22.22MiB 23.03MiB 23.27MiB 23.85MiB 24.12MiB -0.0947678
('splunk_hec_to_splunk_hec_logs_noack', 'comparison', '7bacccad-cecd-46cb-8db5-9ce952c8c775') 204 22.19MiB 818.68KiB 20.49MiB 22.17MiB 23.31MiB 23.48MiB 23.72MiB 23.87MiB -0.0677429
('splunk_hec_to_splunk_hec_logs_acks', 'comparison', '55271bbe-978b-408d-8a8d-4aa9d310ac62') 203 22.17MiB 1.07MiB 19.59MiB 22.32MiB 23.5MiB 23.75MiB 23.97MiB 24.11MiB -0.466191
('splunk_hec_to_splunk_hec_logs_acks', 'comparison', '94d6e57a-d3c2-4944-a4fb-f08d9fa69964') 203 21.37MiB 1.07MiB 17.92MiB 21.53MiB 22.67MiB 22.95MiB 23.35MiB 23.6MiB -0.381861
('splunk_hec_to_splunk_hec_logs_acks', 'baseline', '15fa18da-c4d1-440b-b93c-05f2b4a966ad') 228 21.28MiB 1.11MiB 17.45MiB 21.44MiB 22.67MiB 22.84MiB 23.25MiB 23.6MiB -0.576662
('http_to_http_noack', 'baseline', '98513cec-127b-490a-801a-c1bd00d23ab4') 204 21.17MiB 858.37KiB 19.02MiB 21.19MiB 22.32MiB 22.55MiB 23.11MiB 23.61MiB 0.1398
('splunk_hec_to_splunk_hec_logs_acks', 'comparison', '72a2f760-11ed-4a22-ba69-973de8fe35d1') 203 21.12MiB 1020.24KiB 17.8MiB 21.25MiB 22.36MiB 22.56MiB 22.96MiB 23.14MiB -0.489563
('splunk_hec_to_splunk_hec_logs_acks', 'baseline', '7bab820c-6de1-44d8-bde3-dd2afbca38b2') 200 21.05MiB 1.09MiB 17.79MiB 21.23MiB 22.4MiB 22.66MiB 23.14MiB 23.36MiB -0.490255
('http_to_http_noack', 'comparison', 'dc83cbd9-67a8-4a6d-9641-793ce68a6f07') 204 20.92MiB 865.6KiB 19.0MiB 20.87MiB 22.1MiB 22.32MiB 22.62MiB 22.98MiB 0.0305571
('http_to_http_noack', 'baseline', '26ba2245-f163-471b-aa88-fb728c0389fe') 205 20.74MiB 970.69KiB 17.28MiB 20.67MiB 22.01MiB 22.49MiB 22.88MiB 23.61MiB 0.257155
('http_to_http_noack', 'comparison', 'e66dcd7b-6505-4911-aef3-e53d36265f8d') 205 20.72MiB 848.8KiB 17.93MiB 20.7MiB 21.82MiB 22.18MiB 22.73MiB 23.08MiB 0.158637
('http_to_http_noack', 'baseline', 'b64cb922-d30d-4fa3-86ec-7478bd74d810') 204 20.68MiB 758.92KiB 19.02MiB 20.63MiB 21.66MiB 21.99MiB 22.55MiB 23.23MiB 0.408959
('http_to_http_noack', 'comparison', 'ad0b2298-44a3-45b5-9ab3-22d479f07608') 204 20.21MiB 668.36KiB 17.92MiB 20.2MiB 21.1MiB 21.29MiB 21.72MiB 21.86MiB 0.0725944
('socket_to_socket_blackhole', 'baseline', '7afb8739-04cb-4548-a342-ab2456c3fd8f') 202 12.22MiB 36.72KiB 12.12MiB 12.22MiB 12.26MiB 12.28MiB 12.3MiB 12.3MiB -0.00260576
('socket_to_socket_blackhole', 'baseline', 'f1a73201-f6b2-410d-9610-7a2cf736ac7c') 202 12.02MiB 29.07KiB 11.95MiB 12.02MiB 12.06MiB 12.08MiB 12.08MiB 12.08MiB 0.0667479
('socket_to_socket_blackhole', 'comparison', 'e6e623fa-ccfe-4a35-9e23-bae9b6654205') 202 11.86MiB 32.44KiB 11.76MiB 11.86MiB 11.9MiB 11.91MiB 11.94MiB 11.96MiB 0.0255825
('socket_to_socket_blackhole', 'comparison', '1ec9177e-8ff4-4898-ae29-dbd5ba447fc6') 202 11.79MiB 39.47KiB 11.68MiB 11.8MiB 11.84MiB 11.85MiB 11.87MiB 11.89MiB -0.206494
('socket_to_socket_blackhole', 'comparison', '88dac9e2-2603-4b15-b302-3376f5cc5417') 88 11.77MiB 1.84MiB 584.46KiB 12.11MiB 12.16MiB 12.17MiB 12.19MiB 12.2MiB -5.69545
('socket_to_socket_blackhole', 'baseline', '555b6321-79ad-4eb1-a473-cb0c0f098383') 202 11.43MiB 39.91KiB 11.35MiB 11.43MiB 11.48MiB 11.5MiB 11.58MiB 11.62MiB 1.32116
('http_to_http_acks', 'comparison', '54e424cb-9bcf-4f03-bbb0-c77ff24f97e5') 204 9.71MiB 4.47MiB 1.52MiB 9.37MiB 17.57MiB 18.24MiB 19.14MiB 19.56MiB 0.421487
('http_to_http_acks', 'baseline', '2f451403-7095-468b-a252-ef8f4be3fa1b') 200 9.43MiB 4.71MiB 1.77MiB 9.14MiB 17.58MiB 18.09MiB 19.54MiB 20.05MiB 0.505639
('http_to_http_acks', 'baseline', '3a017af3-270a-4e75-9704-9e09b24e38af') 200 9.34MiB 4.59MiB 2.41MiB 9.4MiB 17.07MiB 18.01MiB 19.17MiB 19.85MiB 0.533103
('syslog_splunk_hec_logs', 'comparison', '49afee75-1f78-43a2-8cd5-b9348231be40') 200 9.08MiB 43.35KiB 9.0MiB 9.07MiB 9.15MiB 9.16MiB 9.18MiB 9.2MiB 0.449855
('syslog_log2metric_splunk_hec_metrics', 'comparison', 'efef67fb-0143-4ead-b292-1b038796b146') 200 9.07MiB 33.42KiB 8.99MiB 9.07MiB 9.11MiB 9.12MiB 9.15MiB 9.16MiB 0.0103437
('syslog_humio_logs', 'comparison', '622769e8-30d7-4c47-95a6-aa73b8d5a965') 204 9.06MiB 28.98KiB 8.96MiB 9.06MiB 9.09MiB 9.11MiB 9.14MiB 9.15MiB 0.0707135
('http_to_http_acks', 'comparison', '387029d6-d2f9-44df-91ee-724f37fc1e9f') 204 9.05MiB 4.34MiB 1.82MiB 9.14MiB 16.06MiB 17.84MiB 19.15MiB 20.26MiB 0.583948
('syslog_log2metric_splunk_hec_metrics', 'baseline', 'b702e986-a984-445a-a027-336155e2882b') 200 9.04MiB 27.83KiB 8.97MiB 9.04MiB 9.07MiB 9.08MiB 9.11MiB 9.12MiB 0.0499697
('syslog_humio_logs', 'comparison', '753b33ef-be7d-41a9-a3fd-221b243fd0b1') 204 9.02MiB 40.37KiB 8.94MiB 9.01MiB 9.06MiB 9.08MiB 9.17MiB 9.2MiB 1.65912
('syslog_splunk_hec_logs', 'comparison', 'bb9e5bf1-ed5b-4adc-b015-5c7bd75c3089') 200 8.96MiB 30.17KiB 8.87MiB 8.96MiB 9.0MiB 9.0MiB 9.02MiB 9.03MiB -0.209672
('http_to_http_acks', 'baseline', '8971b231-285c-42bc-a053-a9b3cbe7aec4') 206 8.95MiB 4.3MiB 1.82MiB 9.14MiB 15.3MiB 18.09MiB 20.12MiB 20.6MiB 0.713565
('syslog_humio_logs', 'comparison', '7bc4d8f3-5241-47a6-88f4-9b7b01286f88') 204 8.91MiB 32.05KiB 8.82MiB 8.9MiB 8.94MiB 8.96MiB 8.99MiB 9.0MiB 0.119705
('syslog_log2metric_splunk_hec_metrics', 'baseline', '5af0bf27-3998-4af6-b27e-eddd328598a5') 200 8.9MiB 46.74KiB 8.78MiB 8.9MiB 8.95MiB 8.97MiB 8.99MiB 9.01MiB -0.162975
('syslog_splunk_hec_logs', 'baseline', 'f7953a1f-4501-4cb1-96f4-d7a9f92eada2') 204 8.9MiB 34.28KiB 8.82MiB 8.89MiB 8.93MiB 8.95MiB 9.0MiB 9.0MiB 0.643896
('syslog_humio_logs', 'baseline', 'a7c82a37-6585-4f57-a5cd-f2a58c897129') 202 8.89MiB 29.66KiB 8.81MiB 8.89MiB 8.93MiB 8.94MiB 8.95MiB 8.97MiB 0.0297736
('syslog_splunk_hec_logs', 'baseline', 'f3c2f956-4760-4539-a235-ba81049d0880') 204 8.85MiB 39.92KiB 8.77MiB 8.84MiB 8.89MiB 8.91MiB 8.99MiB 9.0MiB 1.10914
('syslog_splunk_hec_logs', 'comparison', '10fa9fe7-6342-433b-ac2a-4395f1782f0e') 200 8.85MiB 39.53KiB 8.77MiB 8.84MiB 8.88MiB 8.9MiB 8.98MiB 9.02MiB 1.57889
('splunk_hec_route_s3', 'baseline', '9e1c32c8-de55-476b-a62c-f1ea262a1f97') 205 8.83MiB 2.15MiB 4.93MiB 8.81MiB 11.63MiB 12.16MiB 12.91MiB 13.27MiB 0.0873504
('splunk_hec_route_s3', 'baseline', '2be60b6a-f876-490b-b70e-b1f9892148fd') 204 8.81MiB 2.09MiB 4.84MiB 8.85MiB 11.6MiB 12.09MiB 13.23MiB 13.56MiB 0.171812
('splunk_hec_route_s3', 'comparison', 'eec354cc-123c-4d7c-b0f6-f505f73ddfbd') 204 8.76MiB 2.1MiB 5.15MiB 8.64MiB 11.66MiB 12.11MiB 12.88MiB 13.77MiB 0.171944
('splunk_hec_route_s3', 'baseline', '085fe4fe-2995-45ed-817e-d5e0df581ade') 204 8.72MiB 2.0MiB 5.11MiB 8.35MiB 11.67MiB 11.96MiB 12.68MiB 13.22MiB 0.329381
('syslog_log2metric_splunk_hec_metrics', 'comparison', '77a104f4-2665-4b5b-b90d-ceea3f6c7aac') 200 8.72MiB 63.51KiB 8.47MiB 8.72MiB 8.8MiB 8.82MiB 8.83MiB 8.84MiB -0.483906
('splunk_hec_route_s3', 'comparison', '9595761c-e770-44dc-8e0a-a4f9d534c819') 204 8.72MiB 1.98MiB 4.8MiB 8.65MiB 11.5MiB 12.08MiB 12.82MiB 14.11MiB 0.259181
('splunk_hec_route_s3', 'comparison', '05dd0485-396a-4b6e-8e77-7e61d37d05a4') 204 8.66MiB 1.95MiB 5.01MiB 8.89MiB 11.2MiB 11.63MiB 12.59MiB 12.86MiB 0.0773822
('syslog_log2metric_splunk_hec_metrics', 'comparison', '654b9e84-8589-421d-a069-28c80ad5c2d2') 200 8.65MiB 40.58KiB 8.55MiB 8.65MiB 8.71MiB 8.71MiB 8.73MiB 8.75MiB -0.082178
('syslog_splunk_hec_logs', 'baseline', '651daa8c-a8c2-4508-a005-6e9d98a21f2a') 204 8.63MiB 115.17KiB 8.27MiB 8.63MiB 8.75MiB 8.79MiB 8.87MiB 8.96MiB -0.244209
('syslog_humio_logs', 'baseline', '17e78f35-c236-4b7c-84fa-c698b133bc07') 202 8.61MiB 71.9KiB 8.42MiB 8.62MiB 8.68MiB 8.7MiB 8.87MiB 8.92MiB 0.287009
('syslog_log2metric_splunk_hec_metrics', 'baseline', 'f127e09d-a244-43ca-bede-af4fe58ab12b') 79 8.46MiB 2.25MiB 38.0KiB 9.08MiB 9.12MiB 9.14MiB 9.17MiB 9.19MiB -3.44571
('http_to_http_acks', 'comparison', '388eaf0d-94ff-4a03-a2a5-0fe2e9e6dc79') 204 8.16MiB 3.61MiB 1.49MiB 8.21MiB 11.88MiB 13.67MiB 19.02MiB 20.54MiB 0.681955
('syslog_humio_logs', 'baseline', '018866c5-6a4f-4ec3-a7ac-a03b5fbfcc3d') 85 8.11MiB 2.57MiB 12.6KiB 8.95MiB 9.01MiB 9.02MiB 9.05MiB 9.05MiB -2.79731
('syslog_log2metric_humio_metrics', 'comparison', '2a08debb-8c37-4bd2-8077-3b93434501a1') 200 8.01MiB 33.57KiB 7.89MiB 8.0MiB 8.05MiB 8.06MiB 8.07MiB 8.08MiB -0.247189
('syslog_log2metric_humio_metrics', 'comparison', 'be6b7780-3b37-45b6-bda3-398a0af59f97') 200 7.74MiB 32.04KiB 7.67MiB 7.75MiB 7.78MiB 7.8MiB 7.82MiB 7.83MiB 0.257171
('syslog_log2metric_humio_metrics', 'baseline', '0451efdf-3bb9-4422-9f07-8adec3bbc00c') 202 7.72MiB 131.12KiB 7.31MiB 7.74MiB 7.88MiB 7.9MiB 7.94MiB 7.95MiB -0.536164
('syslog_log2metric_humio_metrics', 'baseline', '2e3e45e5-b45e-4127-86c4-b2efecdf991f') 202 7.68MiB 29.84KiB 7.6MiB 7.68MiB 7.72MiB 7.73MiB 7.75MiB 7.77MiB 0.12618
('syslog_log2metric_humio_metrics', 'baseline', '6c96b7b0-7cec-4cf6-8338-b9fc034bedfc') 202 7.63MiB 128.36KiB 7.28MiB 7.64MiB 7.79MiB 7.82MiB 7.87MiB 7.93MiB -0.150673
('syslog_log2metric_humio_metrics', 'comparison', '78a53407-f182-436e-aff4-ead5ad8e830a') 200 7.6MiB 129.99KiB 7.27MiB 7.61MiB 7.75MiB 7.78MiB 7.8MiB 7.96MiB -0.37605
('syslog_regex_logs2metric_ddmetrics', 'baseline', '906d172f-548e-4767-ba3d-4f0f40deb03b') 200 7.27MiB 295.22KiB 6.82MiB 7.29MiB 7.61MiB 7.62MiB 7.65MiB 7.66MiB -0.0484771
('syslog_regex_logs2metric_ddmetrics', 'baseline', '9c912121-cd26-4b81-9f01-5d279a9e813b') 200 7.07MiB 328.21KiB 6.54MiB 7.16MiB 7.41MiB 7.43MiB 7.49MiB 7.5MiB -0.312854
('syslog_loki', 'baseline', 'fe257c54-71ef-4695-96d3-57bf80dd4bd7') 202 7.04MiB 26.0KiB 6.95MiB 7.04MiB 7.07MiB 7.08MiB 7.09MiB 7.1MiB -0.235713
('syslog_regex_logs2metric_ddmetrics', 'baseline', '899beff6-4852-4cae-ab25-35d6e68c3c55') 200 7.03MiB 319.39KiB 6.55MiB 7.13MiB 7.36MiB 7.37MiB 7.39MiB 7.41MiB -0.33884
('syslog_regex_logs2metric_ddmetrics', 'comparison', '6fe11dd8-fac1-4f52-a0fe-7310494a049f') 200 7.01MiB 333.77KiB 6.46MiB 7.2MiB 7.33MiB 7.35MiB 7.37MiB 7.4MiB -0.394074
('syslog_regex_logs2metric_ddmetrics', 'comparison', '7c69feb2-d29f-47d8-a3d0-59b78ceed8b9') 200 6.99MiB 309.31KiB 6.44MiB 7.09MiB 7.3MiB 7.31MiB 7.36MiB 7.38MiB -0.402307
('syslog_loki', 'comparison', 'bff74eb2-2f36-44b0-a0a2-e0ac913d3bb4') 200 6.98MiB 26.61KiB 6.87MiB 6.98MiB 7.01MiB 7.03MiB 7.04MiB 7.05MiB -0.183268
('syslog_loki', 'baseline', 'bec9aaf6-8677-4ce9-936a-f5eea1573a9d') 202 6.98MiB 33.17KiB 6.91MiB 6.98MiB 7.03MiB 7.04MiB 7.07MiB 7.09MiB 0.691368
('syslog_loki', 'baseline', '71cb5934-2b12-47a1-910c-8ed4c9c132ab') 202 6.94MiB 26.87KiB 6.85MiB 6.94MiB 6.97MiB 6.98MiB 6.99MiB 7.01MiB -0.176536
('syslog_loki', 'comparison', 'f3a38e09-0414-4385-814e-da46db8e6343') 200 6.81MiB 100.9KiB 6.56MiB 6.82MiB 6.93MiB 6.94MiB 6.98MiB 7.02MiB -0.38437
('syslog_regex_logs2metric_ddmetrics', 'comparison', '7c06c1dd-8e53-4a01-8e32-49a95943a222') 200 6.79MiB 291.43KiB 6.21MiB 6.93MiB 7.05MiB 7.08MiB 7.1MiB 7.12MiB -0.703161
('syslog_loki', 'comparison', 'b264cbe5-72f3-4531-87c6-0054763b2926') 200 6.79MiB 84.87KiB 6.55MiB 6.79MiB 6.89MiB 6.93MiB 6.95MiB 7.0MiB -0.195626
('splunk_transforms_splunk3', 'comparison', 'b032c946-4bba-4c04-9889-582d35c78328') 202 5.2MiB 1.5MiB 2.1MiB 5.14MiB 7.31MiB 7.64MiB 8.64MiB 10.67MiB 0.451487
('splunk_transforms_splunk3', 'comparison', 'cd0f4e78-155f-4bb2-9655-90f48c432b28') 203 5.16MiB 1.49MiB 1.55MiB 5.01MiB 7.27MiB 7.74MiB 8.06MiB 8.69MiB 0.180119
('splunk_transforms_splunk3', 'baseline', '3830cbe4-326c-4de4-a929-aaf9bffd1d46') 203 5.13MiB 1.44MiB 2.61MiB 5.1MiB 7.09MiB 7.51MiB 8.33MiB 8.61MiB 0.223944
('splunk_transforms_splunk3', 'baseline', '2b1bf33f-d878-414c-8e79-fbf05003f96f') 203 5.1MiB 1.49MiB 2.2MiB 4.84MiB 7.27MiB 7.56MiB 8.15MiB 9.52MiB 0.397394
('splunk_transforms_splunk3', 'baseline', '473887a9-0c33-44e7-8c19-feb14f24f928') 203 5.01MiB 1.38MiB 2.58MiB 4.83MiB 6.82MiB 7.47MiB 8.04MiB 8.83MiB 0.355134
('splunk_transforms_splunk3', 'comparison', '3c0e419d-dfab-4bcb-bfb9-9c54f2e54fbb') 202 5.0MiB 1.46MiB 2.55MiB 4.63MiB 7.17MiB 7.65MiB 8.05MiB 8.37MiB 0.489809
('http_datadog_filter_blackhole', 'comparison', '2b469149-0cd4-402e-a502-41abf3e46c71') 202 2.17MiB 810.03KiB 400.0KiB 2.1MiB 3.22MiB 3.45MiB 4.19MiB 4.25MiB 0.310076
('http_datadog_filter_blackhole', 'comparison', '01124639-347a-4e6d-a600-6e6c09f7f2aa') 202 2.14MiB 841.43KiB 490.01KiB 2.12MiB 3.3MiB 3.69MiB 4.31MiB 4.48MiB 0.418815
('http_datadog_filter_blackhole', 'baseline', '30e51906-c304-4148-a0a2-99eb1aea7d9c') 202 2.11MiB 822.12KiB 490.01KiB 2.02MiB 3.25MiB 3.51MiB 3.95MiB 4.49MiB 0.272549
('http_datadog_filter_blackhole', 'comparison', 'ddf6d578-679e-4d0a-add1-64bb0b98d1da') 202 2.07MiB 866.77KiB 276.98KiB 2.0MiB 3.22MiB 3.6MiB 3.98MiB 4.43MiB 0.353283
('http_datadog_filter_blackhole', 'baseline', 'b2ac18cd-f746-44c0-b46e-690ea63cbe68') 202 1.97MiB 767.8KiB 323.87KiB 1.92MiB 2.91MiB 3.22MiB 3.79MiB 3.99MiB 0.27131
('http_datadog_filter_blackhole', 'baseline', 'b0043254-81ef-4921-902f-cd6e6ccf65c6') 202 1.93MiB 820.37KiB 245.0KiB 1.83MiB 2.91MiB 3.33MiB 3.76MiB 4.33MiB 0.248027
('http_pipelines_no_grok_blackhole', 'comparison', '6eae8825-ea8e-4dae-b451-3a5155dbcff7') 200 1.8MiB 593.17KiB 369.14KiB 1.69MiB 2.56MiB 2.72MiB 3.17MiB 4.08MiB 0.504095
('http_pipelines_no_grok_blackhole', 'comparison', 'b9759e65-76e7-4364-bf96-1316483bbcff') 200 1.78MiB 569.98KiB 276.98KiB 1.74MiB 2.48MiB 2.84MiB 3.08MiB 3.47MiB 0.259923
('http_pipelines_no_grok_blackhole', 'baseline', '6c7708cc-ecfe-48dc-8c67-45f5d0859503') 200 1.78MiB 614.07KiB 615.69KiB 1.73MiB 2.5MiB 2.9MiB 3.19MiB 4.17MiB 0.677013
('http_pipelines_no_grok_blackhole', 'comparison', 'abe162dc-961e-45df-bbc8-9446b779cc3a') 200 1.76MiB 587.09KiB 521.99KiB 1.74MiB 2.53MiB 2.67MiB 2.96MiB 3.36MiB 0.118594
('http_pipelines_no_grok_blackhole', 'baseline', 'e5ebc50a-1482-4e35-84ec-028473059edd') 200 1.75MiB 573.88KiB 523.02KiB 1.72MiB 2.53MiB 2.69MiB 2.97MiB 3.22MiB 0.124408
('http_pipelines_no_grok_blackhole', 'baseline', '94494490-4b80-47d6-8714-4e732862b15c') 200 1.72MiB 600.88KiB 30.89KiB 1.71MiB 2.47MiB 2.66MiB 2.94MiB 3.21MiB -0.0403447
('http_pipelines_blackhole', 'baseline', '98f2bd68-c324-4711-b196-746f40cbff74') 192 802.68KiB 391.18KiB 93.25KiB 744.6KiB 1.3MiB 1.45MiB 1.68MiB 2.14MiB 0.519182
('http_pipelines_blackhole', 'baseline', 'ac228d4e-8626-4876-afe7-af7b11a83951') 194 776.14KiB 421.81KiB 15.99KiB 720.9KiB 1.29MiB 1.55MiB 1.77MiB 1.91MiB 0.557849
('http_pipelines_blackhole_acks', 'comparison', '4c1b7ea1-a047-4fd6-b062-9772d8e54cc0') 200 772.5KiB 468.26KiB 30.89KiB 697.69KiB 1.36MiB 1.67MiB 1.87MiB 2.65MiB 0.88533
('http_pipelines_blackhole', 'comparison', '9bdaa050-0321-40be-acea-85a8b0c81145') 201 762.04KiB 455.11KiB 15.99KiB 723.29KiB 1.32MiB 1.54MiB 1.92MiB 2.45MiB 0.749632
('http_pipelines_blackhole', 'comparison', 'b1bbe535-1901-4faa-987d-0b1f3ab159f8') 201 761.53KiB 429.77KiB 15.99KiB 676.5KiB 1.34MiB 1.52MiB 1.78MiB 2.21MiB 0.710877
('http_pipelines_blackhole_acks', 'baseline', '84f17bbe-f2c5-4d4a-9173-4d869897b21a') 202 759.12KiB 447.16KiB 15.99KiB 698.54KiB 1.34MiB 1.46MiB 2.04MiB 2.26MiB 0.609235
('http_pipelines_blackhole', 'comparison', 'f2b6ee77-c910-4903-9ce6-0ca059c79bbd') 200 757.88KiB 429.56KiB 15.99KiB 722.52KiB 1.38MiB 1.55MiB 1.73MiB 1.88MiB 0.580919
('http_pipelines_blackhole_acks', 'baseline', '69e7e4cf-9e3b-4257-a32b-ff8a5c3ead3b') 198 751.74KiB 406.33KiB 15.99KiB 737.39KiB 1.26MiB 1.5MiB 1.68MiB 1.94MiB 0.423589
('http_pipelines_blackhole', 'baseline', '33a06962-7cf9-4e69-95b4-b76f014bb55a') 204 750.53KiB 418.72KiB 15.99KiB 733.33KiB 1.25MiB 1.48MiB 1.76MiB 1.93MiB 0.468302
('http_pipelines_blackhole_acks', 'baseline', '828e45b2-5a28-4d63-949c-c2f021273478') 201 743.74KiB 468.43KiB 15.99KiB 689.68KiB 1.42MiB 1.62MiB 1.96MiB 2.3MiB 0.737065
('http_pipelines_blackhole_acks', 'comparison', '09e0a1a2-bce7-4224-b173-4b95aebc0e78') 200 742.94KiB 439.74KiB 15.99KiB 684.15KiB 1.33MiB 1.49MiB 1.82MiB 2.46MiB 0.702718
('http_pipelines_blackhole_acks', 'comparison', '61d7f67a-3110-49bc-a83c-6ddc8f91f2d8') 201 702.55KiB 424.62KiB 15.99KiB 674.82KiB 1.22MiB 1.36MiB 1.81MiB 2.03MiB 0.524616

Signed-off-by: Jesse Szwedko <jesse@szwedko.me>
@github-actions
Copy link
github-actions bot commented Feb 1, 2022

Soak Test Results

Baseline: 3f31469
Comparison: 2d6f657
Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes with confidence ≥ 90.00%:

Fine details of change detection per experiment.
experiment Δ mean Δ mean % baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV p-value erratic declared erratic
http_pipelines_blackhole 33.56KiB 4.43 757.87KiB 379.5KiB 15.61KiB 0 0.50033 791.43KiB 411.37KiB 16.79KiB 0 0.519347 0.143522 True True
http_pipelines_blackhole_acks 30.31KiB 4.06 746.26KiB 451.6KiB 18.48KiB 0 0.604645 776.57KiB 447.88KiB 18.33KiB 0 0.576254 0.244435 True True
datadog_agent_remap_datadog_logs 944.86KiB 2.63 35.09MiB 538.77KiB 22.0KiB 0 0.0149819 36.01MiB 345.86KiB 13.8KiB 0 0.00937159 4.03406e-186 False False
datadog_agent_remap_blackhole 743.49KiB 2.44 29.82MiB 451.49KiB 18.34KiB 0 0.0147756 30.54MiB 400.49KiB 16.24KiB 0 0.0127951 1.99794e-150 False False
syslog_log2metric_splunk_hec_metrics 196.63KiB 2.18 8.8MiB 77.23KiB 3.15KiB 0 0.00856636 8.99MiB 166.91KiB 6.81KiB 0 0.0181175 3.96489e-111 False False
socket_to_socket_blackhole 219.43KiB 1.81 11.82MiB 1.22MiB 56.25KiB 0 0.102944 12.04MiB 127.05KiB 5.16KiB 0 0.0102977 0.000116299 False False
splunk_hec_to_splunk_hec_logs_acks 137.17KiB 0.62 21.61MiB 1.2MiB 49.97KiB 0 0.0552595 21.75MiB 1.14MiB 47.23KiB 0 0.0522965 0.0462597 False False
splunk_hec_route_s3 31.24KiB 0.35 8.66MiB 2.03MiB 84.1KiB 0 0.234293 8.7MiB 2.12MiB 87.77KiB 0 0.243667 0.797248 False False
splunk_transforms_splunk3 3.76KiB 0.07 5.1MiB 1.47MiB 60.86KiB 0 0.287235 5.11MiB 1.46MiB 60.73KiB 0 0.285721 0.965124 False False
splunk_hec_indexer_ack_blackhole 12.82KiB 0.05 23.84MiB 386.93KiB 15.8KiB 0 0.0158349 23.86MiB 442.48KiB 17.99KiB 0 0.018099 0.592449 False False
fluent_elasticsearch -56.61KiB -0.07 79.54MiB 681.48KiB 27.02KiB 0 0.0083607 79.48MiB 101.34KiB 4.1KiB 0 0.0012441 0.0387165 False False
syslog_log2metric_humio_metrics -9.96KiB -0.12 7.87MiB 99.83KiB 4.06KiB 0 0.0123783 7.86MiB 91.27KiB 3.73KiB 0 0.011331 0.0706515 False False
syslog_loki -13.03KiB -0.18 6.9MiB 105.24KiB 4.28KiB 0 0.0148876 6.89MiB 127.25KiB 5.19KiB 0 0.0180336 0.0530226 False False
datadog_agent_remap_datadog_logs_acks -144.27KiB -0.39 36.16MiB 453.99KiB 18.4KiB 0 0.012252 36.02MiB 341.18KiB 13.86KiB 0 0.00924371 5.33884e-10 False False
fluent_remap_aws_firehose -327.52KiB -0.4 79.54MiB 283.61KiB 11.49KiB 0 0.00347912 79.22MiB 624.65KiB 25.31KiB 0 0.00769366 8.7804e-30 False False
http_pipelines_no_grok_blackhole -11.27KiB -0.62 1.78MiB 558.58KiB 22.8KiB 0 0.305395 1.77MiB 546.51KiB 22.31KiB 0 0.300649 0.723964 True False
http_to_http_noack -158.06KiB -0.75 20.53MiB 839.09KiB 33.25KiB 0 0.0398917 20.37MiB 1.03MiB 41.96KiB 0 0.050686 0.00321125 False False
syslog_regex_logs2metric_ddmetrics -99.32KiB -1.38 7.02MiB 306.86KiB 12.53KiB 0 0.0426505 6.92MiB 304.75KiB 12.44KiB 0 0.0429509 2.30448e-08 False False
splunk_hec_to_splunk_hec_logs_noack -329.81KiB -1.42 22.64MiB 895.84KiB 35.63KiB 0 0.0386148 22.32MiB 893.84KiB 36.13KiB 0 0.0390833 1.16765e-10 False False
syslog_humio_logs -141.9KiB -1.55 8.95MiB 99.15KiB 4.03KiB 0 0.0108144 8.81MiB 209.67KiB 8.48KiB 0 0.0232303 4.67037e-46 False False
syslog_splunk_hec_logs -143.08KiB -1.57 8.92MiB 179.63KiB 7.26KiB 0 0.0196533 8.78MiB 133.56KiB 5.45KiB 0 0.0148456 1.03405e-50 False False
datadog_agent_remap_blackhole_acks -621.83KiB -1.99 30.59MiB 416.62KiB 16.87KiB 0 0.0132898 29.98MiB 698.92KiB 27.78KiB 0 0.0227475 3.97312e-70 False False
http_datadog_filter_blackhole -116.82KiB -5.4 2.11MiB 808.62KiB 32.85KiB 0 0.37329 2.0MiB 802.02KiB 32.58KiB 0 0.391364 0.0116925 True True
http_to_http_acks -656.47KiB -6.79 9.45MiB 4.51MiB 188.63KiB 0 0.47719 8.81MiB 4.23MiB 174.96KiB 0 0.479954 0.0108465 True True
Fine details of each soak run.
(experiment, variant, run_id) total samples mean std min median p90 p95 p99 max skewness
('fluent_remap_aws_firehose', 'baseline', 'bc046acc-cbf5-4333-9f31-6d1a9787bb6a') 203 79.68MiB 452.79KiB 78.8MiB 79.51MiB 80.38MiB 80.5MiB 81.23MiB 81.54MiB 1.45563
('fluent_elasticsearch', 'baseline', '6579ca70-aba5-4b9e-ad75-a17a7bcad5c9') 228 79.65MiB 1.1MiB 79.33MiB 79.48MiB 79.55MiB 79.59MiB 86.85MiB 87.25MiB 6.29983
('fluent_remap_aws_firehose', 'comparison', 'bd255e47-3f64-41d6-96f5-4cc82c36c0e2') 203 79.58MiB 404.08KiB 79.32MiB 79.48MiB 79.58MiB 80.62MiB 81.16MiB 81.85MiB 3.62429
('fluent_elasticsearch', 'comparison', '9bc11011-ff4b-4ad6-8ca7-11f3618152c5') 204 79.5MiB 158.71KiB 79.11MiB 79.47MiB 79.57MiB 79.62MiB 80.07MiB 80.87MiB 5.05212
('fluent_elasticsearch', 'comparison', '0aaa16af-91b3-4032-b076-5ac29392a44e') 204 79.47MiB 48.75KiB 79.34MiB 79.47MiB 79.54MiB 79.55MiB 79.57MiB 79.6MiB -0.0428372
('fluent_elasticsearch', 'comparison', 'a65a8eba-5a1a-4471-8fab-4de2a8f2d0fa') 204 79.47MiB 54.09KiB 79.35MiB 79.47MiB 79.54MiB 79.56MiB 79.58MiB 79.59MiB 0.00351677
('fluent_elasticsearch', 'baseline', 'c23810d2-0a2d-45af-b3f5-b89a9bafde39') 204 79.47MiB 59.61KiB 79.32MiB 79.47MiB 79.55MiB 79.57MiB 79.61MiB 79.63MiB 0.158554
('fluent_remap_aws_firehose', 'baseline', '484a72e9-40c2-41a3-a4be-eaa61bba5045') 203 79.47MiB 62.11KiB 79.22MiB 79.47MiB 79.55MiB 79.57MiB 79.62MiB 79.75MiB 0.209678
('fluent_elasticsearch', 'baseline', '939038e0-8acc-4702-9c9a-6c6ef2148f68') 204 79.47MiB 48.81KiB 79.36MiB 79.47MiB 79.53MiB 79.55MiB 79.59MiB 79.6MiB 0.116038
('fluent_remap_aws_firehose', 'comparison', 'af983b7a-3bc7-4a1c-93a0-58cbd42958f7') 203 79.47MiB 54.43KiB 79.33MiB 79.47MiB 79.54MiB 79.56MiB 79.59MiB 79.6MiB 0.0478009
('fluent_remap_aws_firehose', 'baseline', 'f762d082-2467-44cd-8264-d2db2b8a68bf') 203 79.47MiB 53.68KiB 79.29MiB 79.47MiB 79.54MiB 79.55MiB 79.58MiB 79.6MiB -0.240813
('fluent_remap_aws_firehose', 'comparison', 'fd9eba15-2bb3-4987-add2-0bfb87330fc0') 203 78.61MiB 644.33KiB 77.22MiB 78.6MiB 79.4MiB 79.69MiB 79.98MiB 80.26MiB 0.115489
('datadog_agent_remap_datadog_logs_acks', 'baseline', '2be620ae-7af8-484c-abb8-6b7a28375b6c') 203 36.45MiB 259.82KiB 35.6MiB 36.45MiB 36.76MiB 36.84MiB 36.98MiB 37.0MiB -0.360213
('datadog_agent_remap_datadog_logs_acks', 'baseline', '459f1fb4-d01b-47f7-8cfa-acf08cce1f3d') 203 36.38MiB 288.64KiB 35.45MiB 36.39MiB 36.74MiB 36.85MiB 36.96MiB 37.07MiB -0.239869
('datadog_agent_remap_datadog_logs', 'comparison', 'd813dcf6-929a-4db5-82b3-360e4d79587c') 200 36.29MiB 299.64KiB 35.62MiB 36.27MiB 36.68MiB 36.75MiB 36.94MiB 37.23MiB 0.202171
('datadog_agent_remap_datadog_logs_acks', 'comparison', '4bc98d04-7197-41f3-98f1-d31015badb8e') 202 36.23MiB 316.49KiB 35.31MiB 36.21MiB 36.6MiB 36.75MiB 36.93MiB 37.18MiB 0.10274
('datadog_agent_remap_datadog_logs_acks', 'comparison', 'bffa321a-0769-43eb-a148-0c96ef0961ac') 202 36.1MiB 213.57KiB 35.48MiB 36.1MiB 36.34MiB 36.41MiB 36.57MiB 36.97MiB 0.100339
('datadog_agent_remap_datadog_logs', 'comparison', '01f8150a-2998-4266-884b-fbfa063a7f68') 200 36.06MiB 224.88KiB 35.37MiB 36.07MiB 36.33MiB 36.45MiB 36.57MiB 36.64MiB -0.119753
('datadog_agent_remap_datadog_logs', 'comparison', '7d2663b0-3793-49ca-a37a-5a5b88d87080') 228 35.73MiB 221.51KiB 35.23MiB 35.72MiB 36.01MiB 36.08MiB 36.18MiB 36.23MiB -0.0470409
('datadog_agent_remap_datadog_logs_acks', 'comparison', 'b74f5374-5b7d-43c0-a8a0-300b160572de') 202 35.72MiB 244.5KiB 35.01MiB 35.71MiB 36.0MiB 36.09MiB 36.21MiB 36.52MiB 0.00726836
('datadog_agent_remap_datadog_logs', 'baseline', '648c7b22-65c3-4542-b188-d50788619a02') 200 35.67MiB 186.86KiB 35.11MiB 35.66MiB 35.92MiB 35.98MiB 36.05MiB 36.14MiB 0.0330924
('datadog_agent_remap_datadog_logs_acks', 'baseline', '9a66399e-bb29-44cf-9e66-e3bc7796cd95') 203 35.65MiB 234.34KiB 34.98MiB 35.66MiB 35.94MiB 36.01MiB 36.11MiB 36.34MiB -0.0869647
('datadog_agent_remap_datadog_logs', 'baseline', '588c602e-3b65-4f45-88ae-01d613fec259') 200 35.13MiB 196.8KiB 34.67MiB 35.13MiB 35.4MiB 35.48MiB 35.52MiB 35.57MiB 0.129724
('datadog_agent_remap_datadog_logs', 'baseline', 'c1deb62d-e5c8-4601-b4b1-fcfe28e909f6') 200 34.47MiB 210.99KiB 33.89MiB 34.47MiB 34.71MiB 34.8MiB 34.85MiB 35.6MiB 0.605026
('datadog_agent_remap_blackhole_acks', 'comparison', 'b025af22-7dc7-4ada-b0a8-977a57293287') 205 30.81MiB 350.5KiB 26.8MiB 30.81MiB 31.07MiB 31.15MiB 31.39MiB 31.49MiB -7.77121
('datadog_agent_remap_blackhole_acks', 'baseline', '4da7a457-22bc-4c32-919f-9ae7765cf5d5') 205 30.76MiB 382.54KiB 26.96MiB 30.77MiB 31.08MiB 31.26MiB 31.51MiB 31.63MiB -5.04012
('datadog_agent_remap_blackhole', 'comparison', 'fb70f322-29bf-41b5-a968-a1dca0228073') 201 30.74MiB 429.17KiB 29.53MiB 30.81MiB 31.22MiB 31.32MiB 31.46MiB 31.52MiB -0.449005
('datadog_agent_remap_blackhole_acks', 'baseline', 'b4dcb6ad-625a-4849-bb18-086818b3999a') 200 30.74MiB 206.54KiB 30.08MiB 30.73MiB 30.99MiB 31.09MiB 31.22MiB 31.3MiB -0.0048077
('datadog_agent_remap_blackhole', 'comparison', 'c296ccfe-e8fa-447e-95a1-1f96c36fe2c2') 201 30.47MiB 241.4KiB 29.71MiB 30.47MiB 30.69MiB 30.93MiB 31.18MiB 31.26MiB 0.605872
('datadog_agent_remap_blackhole', 'comparison', '8d8551e2-a7d3-4afc-9a21-cb3b7c7200b8') 206 30.42MiB 419.67KiB 26.6MiB 30.39MiB 30.87MiB 31.02MiB 31.21MiB 31.38MiB -3.66836
('datadog_agent_remap_blackhole_acks', 'baseline', '11e76d55-6482-431c-8e44-df2097e48cc9') 205 30.27MiB 411.42KiB 25.55MiB 30.32MiB 30.57MiB 30.61MiB 30.68MiB 30.75MiB -8.08598
('datadog_agent_remap_blackhole', 'baseline', 'e287a553-3a28-480d-8ea9-0e921f50bbac') 202 30.14MiB 189.1KiB 29.7MiB 30.15MiB 30.36MiB 30.44MiB 30.59MiB 30.72MiB 0.0866012
('datadog_agent_remap_blackhole', 'baseline', '8b508025-251e-4f57-8e78-e6422e9e0e47') 202 30.0MiB 199.39KiB 29.57MiB 29.99MiB 30.26MiB 30.31MiB 30.38MiB 30.84MiB 0.254282
('datadog_agent_remap_blackhole_acks', 'comparison', '30014fac-87af-4be9-928c-4e5e9f0c531b') 200 29.86MiB 296.83KiB 29.24MiB 29.84MiB 30.27MiB 30.35MiB 30.52MiB 30.6MiB 0.246173
('datadog_agent_remap_blackhole_acks', 'comparison', 'c45a8c7e-4feb-43f7-8d78-1324c5167b63') 228 29.34MiB 292.4KiB 28.73MiB 29.33MiB 29.61MiB 29.89MiB 30.36MiB 30.42MiB 1.0313
('datadog_agent_remap_blackhole', 'baseline', 'a49b4c21-3c33-45d6-8bce-d4ae1dcd1569') 202 29.3MiB 338.03KiB 28.14MiB 29.28MiB 29.72MiB 29.87MiB 30.16MiB 30.82MiB 0.602343
('splunk_hec_indexer_ack_blackhole', 'comparison', 'ca112343-ffee-48d4-9d78-3e34ed321131') 205 23.88MiB 553.46KiB 23.02MiB 23.87MiB 24.31MiB 24.4MiB 24.68MiB 28.71MiB 4.34422
('splunk_hec_indexer_ack_blackhole', 'comparison', '53c32582-dae7-4465-a97a-5c046365ee26') 200 23.85MiB 370.61KiB 22.9MiB 23.87MiB 24.26MiB 24.36MiB 24.63MiB 24.72MiB -0.308615
('splunk_hec_indexer_ack_blackhole', 'baseline', 'a49a458f-6119-44b1-a52d-f835e4b8eab5') 200 23.85MiB 369.9KiB 23.02MiB 23.87MiB 24.31MiB 24.4MiB 24.6MiB 24.69MiB -0.109809
('splunk_hec_indexer_ack_blackhole', 'baseline', '60323539-e818-44d9-a9f0-1f9420853508') 200 23.84MiB 391.46KiB 22.91MiB 23.85MiB 24.33MiB 24.45MiB 24.55MiB 24.64MiB -0.125718
('splunk_hec_indexer_ack_blackhole', 'comparison', '7e1659f1-eff5-4e9f-9706-af980e5c7595') 200 23.84MiB 377.04KiB 23.02MiB 23.84MiB 24.31MiB 24.4MiB 24.52MiB 24.73MiB -0.176456
('splunk_hec_indexer_ack_blackhole', 'baseline', '3b8b8a6c-93ed-4d39-9795-e79dbb53777f') 200 23.84MiB 400.69KiB 22.76MiB 23.85MiB 24.33MiB 24.39MiB 24.6MiB 24.64MiB -0.320199
('splunk_hec_to_splunk_hec_logs_noack', 'baseline', '7f714cb5-d3d4-4ac5-bf70-cb88c42789a9') 200 22.81MiB 914.85KiB 20.72MiB 22.91MiB 23.88MiB 24.09MiB 24.33MiB 24.66MiB -0.327589
('splunk_hec_to_splunk_hec_logs_noack', 'baseline', '84b194ab-67b4-4dfa-bdf1-ec0f47cc3868') 232 22.64MiB 825.85KiB 20.74MiB 22.68MiB 23.63MiB 23.83MiB 24.22MiB 24.63MiB -0.22738
('splunk_hec_to_splunk_hec_logs_noack', 'comparison', '69422ea5-30b0-4757-88b1-ac4c8e8c03ab') 204 22.55MiB 843.42KiB 20.44MiB 22.56MiB 23.64MiB 23.82MiB 24.41MiB 24.54MiB -0.0525887
('splunk_hec_to_splunk_hec_logs_noack', 'comparison', '92b751ce-cef5-4374-a095-5f4fbabcf419') 204 22.54MiB 866.61KiB 20.56MiB 22.43MiB 23.61MiB 23.82MiB 24.6MiB 24.72MiB 0.0554901
('splunk_hec_to_splunk_hec_logs_noack', 'baseline', 'd44fb773-c90f-4ee5-9c3d-7fed23e72c84') 200 22.46MiB 921.81KiB 20.49MiB 22.45MiB 23.63MiB 23.91MiB 24.21MiB 24.55MiB -0.075379
('splunk_hec_to_splunk_hec_logs_acks', 'baseline', 'f3c9630c-dadb-4e17-a981-c2b5f9343b5f') 200 22.34MiB 1.14MiB 19.47MiB 22.44MiB 23.73MiB 23.97MiB 24.39MiB 24.54MiB -0.40524
('splunk_hec_to_splunk_hec_logs_noack', 'comparison', 'bab4e37f-ada2-4d5a-9ae6-d65b83448f9a') 204 21.86MiB 785.48KiB 20.47MiB 21.86MiB 22.88MiB 23.12MiB 23.48MiB 23.94MiB 0.20042
('splunk_hec_to_splunk_hec_logs_acks', 'comparison', 'a7969a18-3697-45c3-b88f-9d7156f769ea') 203 21.84MiB 1.13MiB 18.77MiB 22.03MiB 23.05MiB 23.27MiB 23.85MiB 23.97MiB -0.667842
('splunk_hec_to_splunk_hec_logs_acks', 'comparison', 'b4b4a873-e0b6-4c1e-8a4d-a2ce02336a57') 203 21.82MiB 1.16MiB 18.7MiB 21.96MiB 23.32MiB 23.51MiB 23.84MiB 23.87MiB -0.431121
('splunk_hec_to_splunk_hec_logs_acks', 'comparison', '691ae430-e74b-479a-b5bf-7525f4bcae7f') 203 21.57MiB 1.11MiB 18.49MiB 21.72MiB 22.99MiB 23.28MiB 23.6MiB 23.61MiB -0.454247
('splunk_hec_to_splunk_hec_logs_acks', 'baseline', 'e0d72dda-27e9-4203-9d7e-5f9b0ab59da0') 200 21.45MiB 1.05MiB 18.07MiB 21.6MiB 22.72MiB 22.98MiB 23.35MiB 23.75MiB -0.426226
('splunk_hec_to_splunk_hec_logs_acks', 'baseline', '5f0e994e-1b99-4bf5-b970-0c25007cabf2') 200 21.05MiB 1.01MiB 17.92MiB 21.17MiB 22.26MiB 22.75MiB 23.02MiB 23.26MiB -0.388419
('http_to_http_noack', 'baseline', '85a14140-2816-437c-ba8d-6d29f15dc5ee') 229 20.62MiB 936.42KiB 18.63MiB 20.54MiB 21.81MiB 22.21MiB 22.77MiB 22.96MiB 0.364793
('http_to_http_noack', 'baseline', '8d48afed-9c20-45d2-b1e8-b2c2ae37358a') 204 20.53MiB 796.07KiB 18.76MiB 20.49MiB 21.6MiB 21.79MiB 22.5MiB 23.35MiB 0.584848
('http_to_http_noack', 'comparison', '328d7885-f5ef-4f04-b541-05c178fba862') 204 20.46MiB 864.73KiB 19.0MiB 20.31MiB 21.6MiB 21.91MiB 22.49MiB 23.23MiB 0.441465
('http_to_http_noack', 'baseline', '4f9fe13f-0cf7-4fb8-aa5f-0d4197ea51eb') 204 20.41MiB 749.87KiB 18.62MiB 20.34MiB 21.27MiB 21.56MiB 21.82MiB 23.0MiB 0.0352143
('http_to_http_noack', 'comparison', '62e5a645-49ac-49ab-ba95-4145e0a221c1') 228 20.41MiB 687.21KiB 18.88MiB 20.33MiB 21.24MiB 21.64MiB 22.21MiB 22.47MiB 0.322202
('http_to_http_noack', 'comparison', '8280f4f8-d730-4d08-9668-6948a075898c') 204 20.24MiB 1.45MiB 15.83MiB 20.37MiB 21.8MiB 22.05MiB 22.66MiB 22.98MiB -1.18054
('socket_to_socket_blackhole', 'comparison', '1ccad984-924f-46f4-a7fc-1fd5f35a86b3') 202 12.17MiB 68.93KiB 12.01MiB 12.19MiB 12.24MiB 12.24MiB 12.26MiB 12.28MiB -0.763307
('socket_to_socket_blackhole', 'baseline', '219016b8-34dc-41ac-b327-d0810693d0d9') 202 12.14MiB 39.02KiB 12.03MiB 12.14MiB 12.19MiB 12.2MiB 12.23MiB 12.24MiB -0.014976
('socket_to_socket_blackhole', 'comparison', 'fb622c0b-00e4-402f-9a64-2407c26bb394') 202 12.06MiB 40.13KiB 11.96MiB 12.07MiB 12.11MiB 12.13MiB 12.13MiB 12.14MiB -0.178986
('socket_to_socket_blackhole', 'comparison', 'a63f134a-097e-4377-98ab-efa09524f5e5') 202 11.89MiB 32.43KiB 11.8MiB 11.89MiB 11.92MiB 11.94MiB 11.96MiB 11.98MiB -0.0597231
('socket_to_socket_blackhole', 'baseline', '087fbda4-d44c-4b23-87e7-f1f1c53f8a55') 202 11.86MiB 30.99KiB 11.77MiB 11.86MiB 11.9MiB 11.91MiB 11.92MiB 11.94MiB -0.0842009
('socket_to_socket_blackhole', 'baseline', 'da5765a7-dfd5-4c2b-b541-dc7170d273a0') 88 11.01MiB 2.73MiB 49.51KiB 11.73MiB 11.77MiB 11.79MiB 11.79MiB 11.8MiB -3.7056
('http_to_http_acks', 'baseline', '8d37b13d-78ff-4566-947f-a54527523aa9') 200 10.42MiB 4.96MiB 2.17MiB 10.08MiB 18.66MiB 19.14MiB 20.05MiB 20.26MiB 0.367199
('http_to_http_acks', 'comparison', '00962da0-6713-4724-bd88-0079bdd3039b') 204 9.54MiB 4.6MiB 2.31MiB 9.26MiB 16.76MiB 17.92MiB 20.03MiB 20.29MiB 0.507451
('syslog_log2metric_splunk_hec_metrics', 'comparison', 'e40254be-4192-4ba0-9fa3-c480eee40fbc') 200 9.11MiB 31.35KiB 9.02MiB 9.11MiB 9.15MiB 9.16MiB 9.2MiB 9.24MiB 0.419504
('syslog_log2metric_splunk_hec_metrics', 'comparison', 'c8c37c20-3e01-4e45-a2cb-8024856b8409') 200 9.08MiB 59.94KiB 8.93MiB 9.08MiB 9.16MiB 9.18MiB 9.21MiB 9.23MiB 0.0110439
('syslog_humio_logs', 'baseline', 'f81576a6-a61e-44b3-a453-50a7feeb541c') 202 9.07MiB 54.43KiB 8.92MiB 9.07MiB 9.14MiB 9.15MiB 9.17MiB 9.18MiB -0.390227
('syslog_splunk_hec_logs', 'baseline', '2c5e79cd-d020-4ab5-8716-546eca75b425') 204 9.05MiB 73.16KiB 8.78MiB 9.07MiB 9.12MiB 9.13MiB 9.16MiB 9.17MiB -0.922267
('syslog_splunk_hec_logs', 'baseline', '4e6eab00-c4ac-4f04-b274-bd323ce58b7d') 204 9.02MiB 29.7KiB 8.94MiB 9.02MiB 9.05MiB 9.06MiB 9.11MiB 9.13MiB 0.490339
('http_to_http_acks', 'baseline', 'ee6b497d-270a-4f6d-8876-3e3dc494a2da') 200 9.01MiB 4.08MiB 2.25MiB 9.16MiB 14.99MiB 17.62MiB 19.15MiB 19.18MiB 0.634634
('syslog_humio_logs', 'comparison', 'ff3dd94d-6a56-4212-864e-467d3b99eefb') 204 9.01MiB 29.33KiB 8.93MiB 9.01MiB 9.05MiB 9.05MiB 9.07MiB 9.09MiB -0.00219456
('http_to_http_acks', 'baseline', '87f515ce-be7b-4934-b0f3-d6150e5a0276') 200 8.91MiB 4.31MiB 1.34MiB 9.08MiB 15.39MiB 17.66MiB 18.96MiB 19.14MiB 0.54568
('syslog_splunk_hec_logs', 'comparison', '815919a9-2bf8-40a9-8270-d2e26b1c6bbc') 200 8.91MiB 69.36KiB 8.66MiB 8.92MiB 8.99MiB 9.0MiB 9.01MiB 9.03MiB -0.856016
('syslog_humio_logs', 'baseline', '8cc20028-5ba1-40d1-baa8-5b551ffbf22b') 202 8.89MiB 27.67KiB 8.83MiB 8.89MiB 8.93MiB 8.94MiB 8.96MiB 8.97MiB 0.28281
('syslog_humio_logs', 'comparison', '26434686-f120-4845-94f8-88f6bed443c1') 204 8.88MiB 45.04KiB 8.81MiB 8.88MiB 8.92MiB 8.94MiB 9.04MiB 9.13MiB 1.92064
('syslog_humio_logs', 'baseline', '78110fd9-832b-47af-a53c-aa0080dfcb4c') 202 8.87MiB 30.4KiB 8.79MiB 8.87MiB 8.91MiB 8.92MiB 8.94MiB 8.95MiB -0.0452142
('syslog_log2metric_splunk_hec_metrics', 'baseline', 'bb49e498-1ba1-4a6d-8276-41da64f10230') 200 8.86MiB 56.14KiB 8.65MiB 8.87MiB 8.91MiB 8.92MiB 8.96MiB 8.97MiB -1.00205
('syslog_splunk_hec_logs', 'comparison', '7ee86c0f-d6c8-4923-9c91-0d16c8e4937b') 200 8.79MiB 32.0KiB 8.71MiB 8.78MiB 8.82MiB 8.84MiB 8.86MiB 8.87MiB -0.0488673
('syslog_log2metric_splunk_hec_metrics', 'baseline', '3aafcd29-b382-4d38-857b-012ab8a65604') 200 8.78MiB 29.26KiB 8.71MiB 8.78MiB 8.82MiB 8.83MiB 8.85MiB 8.88MiB 0.134749
('syslog_log2metric_splunk_hec_metrics', 'comparison', 'f4225c94-7f94-4c64-aa32-4c716b4cd333') 200 8.78MiB 91.6KiB 8.55MiB 8.78MiB 8.9MiB 8.94MiB 8.98MiB 9.05MiB 0.272793
('splun 8000 k_hec_route_s3', 'comparison', 'e5fe60b8-61fa-4135-a456-a5bcdb2e7d5d') 204 8.77MiB 2.06MiB 5.24MiB 8.48MiB 11.53MiB 12.41MiB 13.09MiB 13.56MiB 0.300414
('splunk_hec_route_s3', 'comparison', 'b3912cde-30de-441f-af8b-bf6f8ecf7f11') 204 8.76MiB 2.18MiB 3.86MiB 8.52MiB 11.6MiB 11.98MiB 13.35MiB 13.95MiB 0.152977
('syslog_log2metric_splunk_hec_metrics', 'baseline', '72a31520-4f7a-4f0d-9f73-e4aa0d193c2a') 200 8.75MiB 89.8KiB 8.41MiB 8.77MiB 8.83MiB 8.84MiB 8.89MiB 8.9MiB -1.44246
('splunk_hec_route_s3', 'baseline', 'dff239aa-00c5-4d2b-b2a6-5cf7511be908') 204 8.72MiB 2.03MiB 5.17MiB 8.58MiB 11.27MiB 11.79MiB 13.3MiB 14.24MiB 0.335939
('splunk_hec_route_s3', 'baseline', '7c7a92a1-e645-4aed-b2c8-d7741efbbfa8') 204 8.69MiB 2.04MiB 5.01MiB 8.4MiB 11.6MiB 11.97MiB 12.88MiB 13.05MiB 0.278169
('syslog_splunk_hec_logs', 'baseline', '041f0803-b578-421a-9598-3df82657c5c7') 204 8.68MiB 66.47KiB 8.47MiB 8.69MiB 8.77MiB 8.79MiB 8.84MiB 8.84MiB -0.0548276
('syslog_splunk_hec_logs', 'comparison', 'ad905d23-7fef-4d5b-9070-e4bdc3d64c42') 200 8.64MiB 106.07KiB 8.53MiB 8.61MiB 8.67MiB 8.95MiB 9.01MiB 9.02MiB 2.74585
('splunk_hec_route_s3', 'baseline', '2ddefed2-7191-4ac0-9ddd-9047fb0354e6') 204 8.59MiB 2.03MiB 5.02MiB 8.38MiB 11.48MiB 11.95MiB 12.36MiB 13.51MiB 0.244042
('splunk_hec_route_s3', 'comparison', '04529b3d-6774-475c-a552-d18ae5670d86') 204 8.55MiB 2.12MiB 4.86MiB 8.14MiB 11.6MiB 12.1MiB 12.97MiB 13.35MiB 0.372385
('http_to_http_acks', 'comparison', 'e7c49ca7-18cc-4ff1-8cdb-7f0595e39a8a') 205 8.55MiB 4.08MiB 2.31MiB 8.09MiB 14.3MiB 16.44MiB 19.51MiB 20.12MiB 0.735033
('syslog_humio_logs', 'comparison', 'a4d01a57-71d0-4f1f-aa76-872213d44a0a') 204 8.53MiB 39.58KiB 8.42MiB 8.53MiB 8.58MiB 8.59MiB 8.62MiB 8.75MiB 0.923873
('http_to_http_acks', 'comparison', '7ca16268-b780-4693-857c-0085549fad78') 204 8.34MiB 3.9MiB 2.1MiB 8.4MiB 12.03MiB 17.22MiB 18.96MiB 20.05MiB 0.756818
('syslog_log2metric_humio_metrics', 'baseline', 'b7ac1cdb-7c20-45bc-9fa5-257ca1d27715') 202 7.94MiB 27.59KiB 7.86MiB 7.94MiB 7.97MiB 7.98MiB 7.99MiB 7.99MiB -0.186184
('syslog_log2metric_humio_metrics', 'baseline', '5f5af094-376f-41ae-ab38-2c5f2fc20ad1') 202 7.93MiB 28.47KiB 7.85MiB 7.94MiB 7.97MiB 7.98MiB 7.99MiB 8.0MiB -0.125942
('syslog_log2metric_humio_metrics', 'comparison', '03b62361-0c1b-420f-83e7-23c51ccd34fb') 200 7.93MiB 69.85KiB 7.79MiB 7.91MiB 8.03MiB 8.05MiB 8.07MiB 8.11MiB 0.410377
('syslog_log2metric_humio_metrics', 'comparison', '70c855f3-6448-4eaa-af24-ab8220f318e8') 200 7.89MiB 29.02KiB 7.83MiB 7.89MiB 7.93MiB 7.94MiB 7.95MiB 7.98MiB 0.204143
('syslog_log2metric_humio_metrics', 'comparison', 'eaf312a6-7b98-46fa-82f1-09675a645aa8') 200 7.75MiB 30.37KiB 7.67MiB 7.75MiB 7.79MiB 7.81MiB 7.83MiB 7.86MiB 0.361527
('syslog_log2metric_humio_metrics', 'baseline', '83d31368-c64a-4b3b-8d27-aaa35d069ce5') 202 7.74MiB 28.56KiB 7.67MiB 7.74MiB 7.77MiB 7.78MiB 7.79MiB 7.84MiB 0.1143
('syslog_regex_logs2metric_ddmetrics', 'baseline', '3f7c4b79-9eee-41bf-87a4-4c95ef0f027f') 200 7.06MiB 297.22KiB 6.6MiB 7.17MiB 7.37MiB 7.39MiB 7.47MiB 7.53MiB -0.273164
('syslog_regex_logs2metric_ddmetrics', 'baseline', 'd87f9171-29ce-4fa0-8928-c3022f8b14a8') 200 7.02MiB 328.01KiB 6.52MiB 7.16MiB 7.34MiB 7.37MiB 7.38MiB 7.46MiB -0.374886
('syslog_regex_logs2metric_ddmetrics', 'comparison', '69989d26-edf5-45a7-b873-9b902f03697e') 200 6.99MiB 313.61KiB 6.38MiB 7.12MiB 7.32MiB 7.33MiB 7.36MiB 7.37MiB -0.419622
('syslog_loki', 'comparison', '9201286c-1c90-46c5-b043-1c5261223ed3') 200 6.99MiB 26.38KiB 6.92MiB 6.99MiB 7.02MiB 7.02MiB 7.05MiB 7.06MiB 0.0136827
('syslog_regex_logs2metric_ddmetrics', 'baseline', '1846ef5a-7068-4d18-a0af-97a1e04bcd8f') 200 6.97MiB 288.54KiB 6.47MiB 7.09MiB 7.26MiB 7.28MiB 7.3MiB 7.32MiB -0.433424
('syslog_loki', 'baseline', '36d94748-e0d2-473d-921f-411a237b4c1f') 202 6.96MiB 25.22KiB 6.88MiB 6.95MiB 6.99MiB 6.99MiB 7.01MiB 7.02MiB -0.0761797
('syslog_loki', 'baseline', '3e5855e2-721b-4efc-aa4b-1ad006bac34b') 202 6.95MiB 28.31KiB 6.88MiB 6.95MiB 6.99MiB 7.0MiB 7.01MiB 7.02MiB 0.0989487
('syslog_loki', 'comparison', '255a4273-79aa-4412-8ab0-183826361731') 200 6.94MiB 26.79KiB 6.87MiB 6.94MiB 6.97MiB 6.98MiB 6.99MiB 7.0MiB 0.00503741
('syslog_regex_logs2metric_ddmetrics', 'comparison', '40f866b8-ab70-4b2a-832f-4c79d2c8a38b') 200 6.93MiB 292.16KiB 6.42MiB 7.08MiB 7.22MiB 7.23MiB 7.25MiB 7.28MiB -0.493473
('syslog_regex_logs2metric_ddmetrics', 'comparison', '674accd2-e725-45a8-ae3b-388347d8ecb7') 200 6.85MiB 292.74KiB 6.31MiB 6.97MiB 7.14MiB 7.18MiB 7.23MiB 7.26MiB -0.525401
('syslog_loki', 'baseline', '16a6a120-9979-41cf-bf99-91accec9bacf') 202 6.79MiB 109.93KiB 6.51MiB 6.79MiB 6.92MiB 6.94MiB 6.97MiB 7.03MiB -0.230514
('syslog_loki', 'comparison', 'ca4c11b6-b608-4eb9-8f3f-4b3d4b7176e5') 200 6.73MiB 95.93KiB 6.51MiB 6.74MiB 6.86MiB 6.87MiB 6.9MiB 6.92MiB -0.268556
('splunk_transforms_splunk3', 'comparison', '0497accb-5f9c-45be-a7a3-fe02904600f0') 202 5.22MiB 1.47MiB 2.61MiB 4.98MiB 7.21MiB 7.62MiB 8.24MiB 9.31MiB 0.349412
('splunk_transforms_splunk3', 'baseline', 'aff0e5da-99b3-440a-9a26-33ba1a0e302b') 203 5.17MiB 1.45MiB 2.35MiB 5.04MiB 7.19MiB 7.57MiB 8.1MiB 8.92MiB 0.290578
('splunk_transforms_splunk3', 'baseline', '952295b3-99f1-4684-a412-e207dd654f8c') 203 5.12MiB 1.51MiB 2.32MiB 4.89MiB 7.18MiB 7.54MiB 8.19MiB 8.25MiB 0.24913
('splunk_transforms_splunk3', 'comparison', 'f474bf50-e305-450d-b893-f5aefb47849e') 202 5.08MiB 1.48MiB 2.58MiB 4.75MiB 7.23MiB 7.45MiB 8.12MiB 8.36MiB 0.394495
('splunk_transforms_splunk3', 'baseline', 'f072df01-521c-4d03-96ed-5e30ea21206d') 203 5.02MiB 1.44MiB 2.1MiB 4.81MiB 7.14MiB 7.49MiB 7.84MiB 7.97MiB 0.254771
('splunk_transforms_splunk3', 'comparison', '303ebe57-343d-40a5-83b6-5cf3906501cc') 202 5.02MiB 1.43MiB 2.61MiB 4.81MiB 6.87MiB 7.49MiB 8.52MiB 9.0MiB 0.481274
('http_datadog_filter_blackhole', 'baseline', 'b013f506-4c7e-4b2a-8b92-940a4f67ee88') 202 2.14MiB 837.71KiB 354.76KiB 2.1MiB 3.29MiB 3.57MiB 4.06MiB 4.43MiB 0.293743
('http_datadog_filter_blackhole', 'baseline', 'd774f892-8a15-44e6-bc63-115894d26d54') 202 2.14MiB 823.53KiB 400.0KiB 2.08MiB 3.25MiB 3.68MiB 3.84MiB 4.33MiB 0.220932
('http_datadog_filter_blackhole', 'comparison', '55cf19fe-2b73-474f-8a43-8cbf5d99bf5d') 202 2.08MiB 823.02KiB 260.99KiB 1.89MiB 3.12MiB 3.52MiB 3.8MiB 4.19MiB 0.30564
('http_datadog_filter_blackhole', 'baseline', '89925aaf-617b-416a-8419-9be5ae497bc0') 202 2.06MiB 764.19KiB 307.88KiB 1.96MiB 3.14MiB 3.5MiB 3.84MiB 3.87MiB 0.395076
('http_datadog_filter_blackhole', 'comparison', 'ca3cc19c-3f38-4dc1-9f21-443dfa3ad760') 202 2.05MiB 820.33KiB 400.0KiB 1.96MiB 3.15MiB 3.51MiB 4.01MiB 4.25MiB 0.302704
('http_datadog_filter_blackhole', 'comparison', 'cf68cb53-cc8d-4709-8d96-7c380747976a') 202 1.87MiB 747.96KiB 276.98KiB 1.73MiB 2.86MiB 3.2MiB 3.55MiB 3.78MiB 0.375441
('http_pipelines_no_grok_blackhole', 'comparison', 'bb236460-94fc-4735-8ad4-2ac9d6c4ab69') 200 1.83MiB 539.5KiB 461.78KiB 1.82MiB 2.5MiB 2.75MiB 2.9MiB 3.33MiB 0.113322
('http_pipelines_no_grok_blackhole', 'baseline', '0285dfff-d213-4b84-b7c9-0e7610f2bdd6') 200 1.8MiB 589.14KiB 279.66KiB 1.77MiB 2.62MiB 2.79MiB 3.03MiB 3.33MiB 0.0989989
('http_pipelines_no_grok_blackhole', 'baseline', '9cef3503-4e38-4d09-a269-5a11289b7e15') 200 1.79MiB 517.99KiB 508.72KiB 1.76MiB 2.51MiB 2.68MiB 2.82MiB 3.35MiB 0.217514
('http_pipelines_no_grok_blackhole', 'baseline', '881fb83f-fe57-46d6-8ca5-a23a45408a75') 200 1.76MiB 568.41KiB 508.15KiB 1.76MiB 2.5MiB 2.69MiB 2.96MiB 3.0MiB -1.01439e-05
('http_pipelines_no_grok_blackhole', 'comparison', '0f49fcca-9f71-46cf-ae87-86edc36a0c7d') 200 1.76MiB 514.68KiB 658.22KiB 1.78MiB 2.4MiB 2.58MiB 2.69MiB 3.14MiB -0.0339351
('http_pipelines_no_grok_blackhole', 'comparison', 'fc45af79-8f57-46ae-a940-a35064f6df8d') 200 1.72MiB 579.97KiB 247.72KiB 1.74MiB 2.49MiB 2.67MiB 2.99MiB 3.29MiB 0.0400923
('http_pipelines_blackhole', 'comparison', 'ef1ca661-5359-4534-a027-e929c6ecf946') 201 811.66KiB 428.93KiB 123.02KiB 770.74KiB 1.35MiB 1.55MiB 1.9MiB 2.29MiB 0.646614
('http_pipelines_blackhole_acks', 'comparison', '928bfaf0-37f6-4f8a-8515-984a5feaaae1') 201 806.81KiB 496.69KiB 30.89KiB 765.31KiB 1.45MiB 1.67MiB 2.12MiB 2.64MiB 0.802826
('http_pipelines_blackhole', 'comparison', '37326b0a-7071-4b1a-9790-492d922b5d1a') 200 787.16KiB 411.63KiB 46.88KiB 727.79KiB 1.31MiB 1.51MiB 1.75MiB 1.84MiB 0.486053
('http_pipelines_blackhole_acks', 'comparison', 'ffbe15c2-5700-4aa5-bb33-5d71676922d3') 198 777.62KiB 411.84KiB 62.35KiB 735.24KiB 1.31MiB 1.46MiB 1.94MiB 1.97MiB 0.635287
('http_pipelines_blackhole', 'comparison', '00816458-93e4-4c10-a9f8-b84e1d59a18a') 199 775.28KiB 393.86KiB 47.97KiB 720.58KiB 1.26MiB 1.48MiB 1.77MiB 1.96MiB 0.654549
('http_pipelines_blackhole', 'baseline', 'b7ba8071-854b-44c7-b8e6-99a27752c78a') 194 772.59KiB 378.27KiB 15.99KiB 738.76KiB 1.21MiB 1.38MiB 1.8MiB 2.04MiB 0.501043
('http_pipelines_blackhole_acks', 'baseline', '5495d7e0-7b27-4517-8934-2ab5496623ee') 199 768.95KiB 465.98KiB 46.88KiB 722.26KiB 1.33MiB 1.56MiB 2.01MiB 2.21MiB 0.678284
('http_pipelines_blackhole', 'baseline', '5e58029b-2541-4fbe-926e-4aea7a682c9c') 199 756.21KiB 376.24KiB 61.78KiB 753.72KiB 1.21MiB 1.42MiB 1.7MiB 1.73MiB 0.373289
('http_pipelines_blackhole_acks', 'baseline', '66c554d9-c908-4de4-b718-4c84eb1aba85') 200 745.35KiB 453.98KiB 15.99KiB 673.69KiB 1.33MiB 1.55MiB 2.03MiB 2.55MiB 0.994497
('http_pipelines_blackhole', 'baseline', '428653f8-2450-4237-9b5c-8d409e67ff8c') 198 745.1KiB 385.35KiB 15.99KiB 722.8KiB 1.25MiB 1.39MiB 1.75MiB 2.03MiB 0.575674
('http_pipelines_blackhole_acks', 'comparison', '8bddea03-cad6-4373-a69c-967e7f499185') 198 744.83KiB 429.96KiB 15.99KiB 705.99KiB 1.33MiB 1.51MiB 1.75MiB 1.87MiB 0.468252
('http_pipelines_blackhole_acks', 'baseline', '21ac6941-f276-4359-b9d6-cea764bb5615') 198 724.37KiB 435.39KiB 15.99KiB 674.79KiB 1.24MiB 1.47MiB 2.11MiB 2.24MiB 0.823964

@jszwedko jszwedko merged commit 4272649 into master Feb 1, 2022
@jszwedko jszwedko deleted the update-rpm-tests branch February 1, 2022 23:16
jszwedko added a commit that referenced this pull request Feb 10, 2022
* fix(releasing): Update rpm tests

Drop EOL distributions. This was prompted because the mirrors for Centos
8 no longer exist.

https://www.centos.org/centos-linux-eol/

Signed-off-by: Jesse Szwedko <jesse@szwedko.me>

* Allow replacing existing packages

Signed-off-by: Jesse Szwedko <jesse@szwedko.me>

* Allow replacing installed packages

Signed-off-by: Jesse Szwedko <jesse@szwedko.me>

* conditionally install curl

Signed-off-by: Jesse Szwedko <jesse@szwedko.me>
Sign up for free to join this conve 5FD0 rsation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants
0