8000 feat: add binary save load for DocumentArray by hanxiao · Pull Request #2495 · jina-ai/serve · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

feat: add binary save load for DocumentArray #2495

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 3 commits into from
May 28, 2021
Merged

feat: add binary save load for DocumentArray #2495

merged 3 commits into from
May 28, 2021

Conversation

hanxiao
Copy link
Member
@hanxiao hanxiao commented May 28, 2021 •
edited
Loading 
from jina import DocumentArray
from jina.logging.profile import TimeContext
from tests import random_docs

da = DocumentArray(random_docs(10000))
with TimeContext('w/binary'):
    da.save_binary('test.bin')
with TimeContext('w/json'):
    da.save('test.json')
with TimeContext('r/binary'):
    da_r = DocumentArray.load_binary('test.bin')
with TimeContext('r/json'):
    da_r = DocumentArray.load('test.json')
w/binary ...	w/binary takes 0 seconds (0.71s)
w/json ...	w/json takes 3 seconds (3.19s)
r/binary ...	r/binary takes 0 seconds (0.20s)
r/json ...	r/json takes 3 seconds (3.54s)

test.bin  12.6MB
test.json  22.4MB

< 8000 /div>
@hanxiao hanxiao requested a review from a team as a code owner May 28, 2021 03:47
@hanxiao hanxiao requested review from maximilianwerk, JoanFM, numb3r3 and Yongxuanzhang and removed request for maximilianwerk and JoanFM May 28, 2021 03:47
@jina-bot jina-bot added size/M area/core This issue/PR affects the core codebase area/testing This issue/PR affects testing component/type labels May 28, 2021
@codecov
Copy link
codecov bot commented May 28, 2021 •
edited
Loading

Codecov Report

Merging #2495 (6a11fb7) into master (42a296d) will increase coverage by 0.06%.
The diff coverage is 89.74%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #2495      +/-   ##
==========================================
+ Coverage   83.37%   83.44%   +0.06%     
==========================================
  Files         152      152              
  Lines        9414     9445      +31     
==========================================
+ Hits         7849     7881      +32     
+ Misses       1565     1564       -1
Flag Coverage Δ
daemon 46.98% <28.20%> (-0.10%) ⬇️
jina 83.25% <89.74%> (+0.07%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
jina/types/arrays/document.py 82.28% <89.47%> (+1.03%) ⬆️
jina/__init__.py 69.69% <100.00%> (ø)
jina/peapods/pods/compound.py 91.53% <0.00%> (+1.53%) ⬆️
jina/peapods/peas/__init__.py 96.72% <0.00%> (+2.45%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d445457...6a11fb7. Read the comment docs.

@jina-bot jina-bot added the area/housekeeping This issue/PR is housekeeping label May 28, 2021
@hanxiao hanxiao merged commit b857404 into master May 28, 2021
@hanxiao hanxiao deleted the feat-add-binary-save-load branch May 28, 2021 04:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@numb3r3 numb3r3 Awaiting requested review from numb3r3

@Yongxuanzhang Yongxuanzhang Awaiting requested review from Yongxuanzhang

Assignees
No one assigned
Labels
area/core This issue/PR affects the core codebase area/housekeeping This issue/PR is housekeeping area/testing This issue/PR affects testing component/type size/M
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@hanxiao @jina-bot
0