8000 test: replicas with dump-reload by cristianmtr · Pull Request #2331 · jina-ai/serve · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

test: replicas with dump-reload #2331

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 20 commits into from
May 7, 2021
Merged

test: replicas with dump-reload #2331

merged 20 commits into from
May 7, 2021

Conversation

cristianmtr
Copy link
Contributor
@cristianmtr cristianmtr commented Apr 21, 2021 •
edited
Loading

Todo

  • stress test with jinad and threads
  • async reload/query
  • use only REST API
  • remote flow on jinad

added jinad func. for Dump/Rolling Update triggering

Part of https://github.com/jina-ai/internal-tasks/issues/40

Benchmark

### baseline - query time with 3 on 100000 docs ...	         
### baseline - query time with 3 on 100000 docs takes 2 seconds (2.59s)
### baseline - indexing: 100000 docs takes 7 seconds (7.63s)
### indexing 100000 docs takes 12 seconds (12.72s)
### dumping 100000 docs takes 2 seconds (2.73s)
### dump path size: 112.75556 MBs
### rolling update on 100000 takes 2 seconds (2.03s)
### reloading 100000 takes 0 seconds (0.97s)

@jina-bot jina-bot added size/S area/core This issue/PR affects the core codebase area/testing This issue/PR affects testing component/executor executor/indexer labels Apr 21, 2021
@cristianmtr cristianmtr force-pushed the test-dump-reload-replicas branch from 4a70709 to c0ebecb Compare April 21, 2021 16:03
@codecov
Copy link
codecov bot commented Apr 21, 2021 •
edited
Loading

Codecov Report

Merging #2331 (1bbfe75) into master (e2089e5) will decrease coverage by 0.21%.
The diff coverage is 77.20%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #2331      +/-   ##
==========================================
- Coverage   88.58%   88.36%   -0.22%     
==========================================
  Files         230      230              
  Lines       12219    12301      +82     
==========================================
+ Hits        10824    10870      +46     
- Misses       1395     1431      +36
Flag Coverage Δ
daemon 50.26% <33.08%> (-0.08%) ⬇️
jina 88.49% <85.04%> (-0.13%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
daemon/api/endpoints/__init__.py 75.00% <ø> (ø)
daemon/models/custom.py 87.23% <ø> (ø)
daemon/stores/helper.py 95.45% <ø> (ø)
jina/enums.py 95.97% <ø> (-0.03%) ⬇️
jina/peapods/runtimes/zmq/zed.py 91.39% <ø> (ø)
jina/proto/jina_pb2.py 100.00% <ø> (ø)
jina/clients/__init__.py 81.81% <11.11%> (-13.84%) ⬇️
daemon/stores/flow.py 77.14% <27.27%> (-22.86%) ⬇️
daemon/api/endpoints/flow.py 74.46% <46.15%> (-11.25%) ⬇️
jina/flow/base.py 91.48% <75.00%> (-0.79%) ⬇️
... and 22 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e2089e5...1bbfe75. Read the comment docs.

@github-actions
Copy link
github-actions bot commented Apr 21, 2021 •
edited
Loading

Latency summary

Current PR yields:

  • 😶 index QPS at 921, delta to last 3 avg.: +0%
  • 😶 query QPS at 13, delta to last 3 avg.: -3%

Breakdown

Version Index QPS Query QPS
current 921 13
1.2.1 918 13
1.2.0 923 13

Backed by latency-tracking. Further commits will update this comment.

JoanFM
JoanFM reviewed Apr 21, 2021
View reviewed changes
Copy link
Contributor
@JoanFM JoanFM left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The good test would be to have a client deattached from flow and see that the queries are still served while reloading

@jina-bot jina-bot added size/M area/network This issue/PR affects network functionality component/flow component/peapod and removed size/S labels Apr 22, 2021
JoanFM
JoanFM reviewed Apr 22, 2021 8000
View reviewed changes
JoanFM
JoanFM reviewed Apr 22, 2021
View reviewed changes
Copy link
Contributor
@JoanFM JoanFM left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the way replica_args are created need to be refactored

JoanFM
JoanFM reviewed Apr 22, 2021
View reviewed changes
@cristianmtr
Copy link
Contributor Author

I think the way replica_args are created need to be refactored

Not going to happen in this PR. The focus is to just get it to work at this point. But I agree there are a lot of things that need improving here.

@cristianmtr cristianmtr force-pushed the test-dump-reload-replicas branch from e12b066 to ae7dc76 Compare April 22, 2021 12:12
JoanFM
JoanFM reviewed Apr 22, 2021
View reviewed changes
JoanFM
JoanFM reviewed Apr 23, 2021
View reviewed changes
@cristianmtr cristianmtr force-pushed the test-dump-reload-replicas branch from 077f7b3 to 7e3a773 Compare April 23, 2021 11:24
@jina-bot jina-bot added size/M and removed size/XL labels Apr 23, 2021
@cristianmtr cristianmtr requested review from hanxiao and nan-wang May 3, 2021 13:55
JoanFM
JoanFM suggested changes May 3, 2021
View reviewed changes
jina/peapods/pods/__init__.py Outdated
:param timeout: time to wait (seconds)
"""
for pea in self.peas:
if 'head' not in pea.name and 'tail' not in pea.name:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

have a private property for that?

8000
@cristianmtr cristianmtr force-pushed the test-dump-reload-replicas branch from b5495e8 to 3cd9821 Compare May 3, 2021 16:01
@cristianmtr
Copy link
Contributor Author

#2331 (comment)

@JoanFM Added a property

@cristianmtr cristianmtr requested a review from JoanFM May 4, 2021 10:22
JoanFM
JoanFM suggested changes May 4, 2021
View reviewed changes
@cristianmtr cristianmtr force-pushed the test-dump-reload-replicas branch from cd26dc6 to 19e48c4 Compare May 4, 2021 11:02
JoanFM
JoanFM suggested changes May 4, 2021
View reviewed changes
@JoanFM JoanFM mentioned this pull request May 4, 2021
Merged
JoanFM
JoanFM suggested changes May 6, 2021
View reviewed changes
@cristianmtr cristianmtr requested a review from JoanFM May 6, 2021 09:36
JoanFM
JoanFM reviewed May 6, 2021
View reviewed changes
@maximilianwerk maximilianwerk merged commit 5a58e49 into master May 7, 2021
@maximilianwerk maximilianwerk deleted the test-dump-reload-replicas branch May 7, 2021 07:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@JoanFM JoanFM JoanFM approved these changes

@maximilianwerk maximilianwerk Awaiting requested review from maximilianwerk

@florian-hoenicke florian-hoenicke Awaiting requested review from florian-hoenicke

@hanxiao hanxiao Awaiting requested review from hanxiao

@nan-wang nan-wang Awaiting requested review from nan-wang

Assignees
No one assigned
Labels
area/core This issue/PR affects the core codebase area/helper This issue/PR affects the helper functionality area/network This issue/PR affects network functionality area/testing This issue/PR affects testing component/client component/driver component/executor component/flow component/peapod component/proto component/type executor/indexer size/XL
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@cristianmtr @maximilianwerk @JoanFM @jina-bot
0