8000 feat(ci): enabled automated benchmarking by maateen · Pull Request #3248 · jina-ai/serve · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

feat(ci): enabled automated benchmarking #3248

New issue 8000

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 5 commits into from
Aug 24, 2021
Merged

feat(ci): enabled automated benchmarking #3248

merged 5 commits into from
Aug 24, 2021

Conversation

maateen
Copy link
Contributor
@maateen maateen commented Aug 23, 2021 •
edited
Loading

Changes:

  • feat(ci): enabled automated benchmarking
  • feat(ci): enabled automated benchmarking for pypi releases

This PR will close #3245

feat(ci): enabled automated benchmarking
54857df 
Signed-off-by: Maksudur Rahman Maateen <maateen@jina.ai>
@github-actions github-actions bot added size/S area/cicd This issue/PR affects the cicd pipeline area/housekeeping This issue/PR is housekeeping labels Aug 23, 2021
@codecov
Copy link
codecov bot commented Aug 23, 2021 •
edited
Loading

Codecov Report

Merging #3248 (9fd8112) into master (7dd9db8) will decrease coverage by 0.55%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #3248      +/-   ##
==========================================
- Coverage   88.96%   88.40%   -0.56%     
==========================================
  Files         148      148              
  Lines       10392    10392              
==========================================
- Hits         9245     9187      -58     
- Misses       1147     1205      +58
Flag Coverage Δ
daemon 44.88% <ø> (-0.02%) ⬇️
jina 88.39% <ø> (-0.39%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
jina/parsers/optimizer.py 0.00% <0.00%> (-100.00%) ⬇️
jina/jaml/parsers/default/v1.py 0.00% <0.00%> (-100.00%) ⬇️
jina/optimizers/flow_runner.py 0.00% <0.00%> (-97.68%) ⬇️
jina/optimizers/discovery.py 0.00% <0.00%> (-80.27%) ⬇️
jina/optimizers/__init__.py 47.52% <0.00%> (-49.51%) ⬇️
jina/clients/base/grpc.py 63.82% <0.00%> (-6.39%) ⬇️
jina/optimizers/parameters.py 93.54% <0.00%> (-4.84%) ⬇️
jina/peapods/runtimes/gateway/prefetch.py 93.82% <0.00%> (-1.24%) ⬇️
jina/helper.py 80.82% <0.00%> (-0.36%) ⬇️
jina/jaml/parsers/__init__.py 92.85% <0.00%> (ø)
... and 20 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update db54514...9fd8112. Read the comment docs.

@github-actions
Copy link
github-actions bot commented Aug 23, 2021 •
edited
Loading

Latency summary

Current PR yields:

  • 🐎🐎🐎🐎 index QPS at 1182, delta to last 2 avg.: +13%
  • 🐎🐎🐎🐎 query QPS at 51, delta to last 2 avg.: +18%
  • 🐎🐎🐎🐎 dam extend QPS at 56678, delta to last 2 avg.: +34%
  • 🐎🐎🐎🐎 avg flow time within 1.1483 seconds, delta to last 2 avg.: -7%
  • 🐎🐎 import jina within 0.44 seconds, delta to last 2 avg.: +8%

Breakdown

Version Index QPS Query QPS DAM Extend QPS Avg Flow Time (s) Import Time (s)
current 1182 51 56678 1.1483 0.44
2.0.20 1034 41 41973 1.2249 0.4285
2.0.19 1053 44 42078 1.2693 0.3849

Backed by latency-tracking. Further commits will update this comment.

Maksudur Rahman Maateen added 2 commits August 23, 2021 20:05
@maateen maateen marked this pull request as ready for review August 23, 2021 14:51
@maateen maateen requested a review from hanxiao as a code owner August 23, 2021 14:51
hanxiao
hanxiao reviewed Aug 24, 2021
View reviewed changes
tadejsv
tadejsv suggested changes Aug 24, 2021
View reviewed changes
Maksudur Rahman Maateen added 2 commits August 24, 2021 14:39
feat(ci): fixed benchmark ci env issue
f4820a8 
Signed-off-by: Maksudur Rahman Maateen <maateen@jina.ai>
Merge remote-tracking branch 'origin/master' into feat-enable-automat…
9fd8112 
…ed-benchmarking
@hanxiao hanxiao merged commit 384e91b into master Aug 24, 2021
@hanxiao hanxiao deleted the feat-enable-automated-benchmarking branch August 24, 2021 12:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tadejsv tadejsv tadejsv requested changes

@hanxiao hanxiao hanxiao approved these changes

Assignees
No one assigned
Labels
area/cicd This issue/PR affects the cicd pipeline area/housekeeping This issue/PR is housekeeping size/S
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Re-enable automatic benchmarking
3 participants
@maateen @hanxiao @tadejsv
0