8000 refactor: remove Peatype metaclass by JoanFM · Pull Request #2688 · jina-ai/serve · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content
8000

refactor: remove Peatype metaclass #2688

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 31 commits into from
Jun 21, 2021
Merged

refactor: remove Peatype metaclass #2688

merged 31 commits into from
Jun 21, 2021

Conversation

JoanFM
Copy link
Contributor
@JoanFM JoanFM commented Jun 17, 2021 •
edited
Loading

Changes introduced
This PR will enable setting spawn as start method for our processes

Based on https://stackoverflow.com/questions/67822543/pickle-picklingerror-cant-pickle-its-not-the-same-object-as
we saw that this PeaType metaclass was the cause of the inhability to spawn new processes.

This will also open the door to Windows support

Benefits

  • Enables spawning methods
  • Runtime lives in a single stack. (Pea is the only object duplicated). Could be easily changed in a next PR
  • RAII for runtime, no setup. Have everything at __init__
  • Easier explanation of the lifetime of runtime

TODO

Decide how to expose the process _set_start_method method.

@jina-bot jina-bot added size/S area/core This issue/PR affects the core codebase area/network This issue/PR affects network functionality component/peapod labels Jun 17, 2021
@codecov
Copy link
codecov bot commented Jun 17, 2021 •
edited
Loading

Codecov Report

Merging #2688 (7b46273) into master (2f2a975) will increase coverage by 1.33%.
The diff coverage is 96.66%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #2688      +/-   ##
==========================================
+ Coverage   88.02%   89.36%   +1.33%     
==========================================
  Files         150      150              
  Lines        9206     9197       -9     
==========================================
+ Hits         8104     8219     +115     
+ Misses       1102      978     -124
Flag Coverage Δ
daemon 47.23% <89.16%> (-0.11%) ⬇️
jina 89.60% <96.66%> (+1.48%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
jina/peapods/peas/__init__.py 93.79% <93.54%> (-3.04%) ⬇️
jina/logging/logger.py 93.81% <100.00%> (+0.06%) ⬆️
jina/peapods/peas/helper.py 97.43% <100.00%> (-0.44%) ⬇️
jina/peapods/pods/__init__.py 84.25% <100.00%> (-0.98%) ⬇️
jina/peapods/runtimes/asyncio/base.py 92.30% <100.00%> (+0.20%) ⬆️
jina/peapods/runtimes/base.py 100.00% <100.00%> (ø)
jina/peapods/runtimes/container/__init__.py 85.32% <100.00%> (-0.14%) ⬇️
jina/peapods/runtimes/jinad/__init__.py 95.55% <100.00%> (+2.69%) ⬆️
jina/peapods/runtimes/zmq/base.py 100.00% <100.00%> (ø)
jina/peapods/runtimes/zmq/zed.py 92.92% <100.00%> (+1.97%) ⬆️
... and 19 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2f2a975...7b46273. Read the comment docs.

@JoanFM JoanFM force-pushed the fix-allow-spawn branch from f27489e to 54e56c0 Compare June 17, 2021 08:03
@jina-bot jina-bot added area/entrypoint This issue/PR affects the entrypoint codebase area/helper This issue/PR affects the helper functionality labels Jun 17, 2021
@github-actions
Copy link
github-actions bot commented Jun 17, 2021 •
edited
Loading

Latency summary

Current PR yields:

  • 😶 index QPS at 1501, delta to last 1 avg.: +2%
  • 🐢🐢 query QPS at 45, delta to last 1 avg.: -11%

Breakdown

Version Index QPS Query QPS
current 1501 45
2.0.0rc6 1470 50

Backed by latency-tracking. Further commits will update this comment.

deepankarm
deepankarm reviewed Jun 17, 2021
View reviewed changes
@JoanFM JoanFM mentioned this pull request Jun 17, 2021
Closed
@JoanFM JoanFM mentioned this pull request Jun 17, 2021
Closed
@jina-bot jina-bot added the area/testing This issue/PR affects testing label Jun 17, 2021
This was linked to issues Jun 17, 2021
PeaType metaclass weird behavior, why does this not work? #2371
Closed
[Advise: The API call failed because the CUDA driver and runtime could not be initialized. ] #2514
Closed
@JoanFM JoanFM force-pushed the fix-allow-spawn branch from 00e952b to 39a42da Compare June 17, 2021 11:26
@JoanFM JoanFM linked an issue Jun 17, 2021 that may be closed by this pull request
Need to enable spawn as potential process starting method. #2691
Closed
@JoanFM JoanFM force-pushed the fix-allow-spawn branch from e90c48d to 7fe8b21 Compare June 17, 2021 14:52
@jina-bot jina-bot added size/L and removed size/M labels Jun 18, 2021
@JoanFM JoanFM force-pushed the fix-allow-spawn branch 2 times, most recently from 0b4c262 to aec11ce Compare June 18, 2021 13:06
@JoanFM JoanFM marked this pull request as ready for review June 18, 2021 13:13
@JoanFM JoanFM force-pushed the fix-allow-spawn branch from aec11ce to 2232dd8 Compare June 18, 2021 13:50
@JoanFM JoanFM force-pushed the fix-allow-spawn branch from 2232dd8 to f04f724 Compare June 18, 2021 14:22
@JoanFM JoanFM force-pushed the fix-allow-spawn branch from c8b5e5b to 0e82727 Compare June 18, 2021 20:55
@JoanFM JoanFM force-pushed the fix-allow-spawn branch from 915b153 to 9ef37f8 Compare June 18, 2021 22:11
@JoanFM JoanFM requested a review from nan-wang June 20, 2021 11:53
@JoanFM JoanFM mentioned this pull request Jun 20, 2021
Merged
nan-wang
nan-wang previously requested changes Jun 21, 2021
View reviewed changes
Copy link
Member
@nan-wang nan-wang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general, LGTM

@JoanFM JoanFM requested a review from nan-wang June 21, 2021 05:22
@JoanFM JoanFM dismissed nan-wang’s stale review June 21, 2021 05:46

Please review again

@JoanFM JoanFM mentioned this pull request Jun 21, 2021
Closed
nan-wang
nan-wang requested changes Jun 21, 2021
View reviewed changes
Copy link
Member
@nan-wang nan-wang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor change request.

@JoanFM JoanFM requested a review from nan-wang June 21, 2021 06:32
nan-wang
nan-wang approved these changes Jun 21, 2021
View reviewed changes
Copy link
Member
@nan-wang nan-wang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM👍

@JoanFM JoanFM merged commit f107759 into master Jun 21, 2021
@JoanFM JoanFM deleted the fix-allow-spawn branch June 21, 2021 06:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@nan-wang nan-wang nan-wang approved these changes

@davidbp davidbp Awaiting requested review from davidbp

@deepankarm deepankarm Awaiting requested review from deepankarm

@bwanglzu bwanglzu Awaiting requested review from bwanglzu

@hanxiao hanxiao Awaiting requested review from hanxiao

@jacobowitz jacobowitz Awaiting requested review from jacobowitz

Assignees
No one assigned
Labels
area/core This issue/PR affects the core codebase area/entrypoint This issue/PR affects the entrypoint codebase area/helper This issue/PR affects the helper functionality area/network This issue/PR affects network functionality area/testing This issue/PR affects testing component/logging component/peapod size/L
Projects
None yet
Milestone
No milestone
4 participants
@JoanFM @nan-wang @deepankarm @jina-bot
0