8000 docs: crud by cristianmtr · Pull Request #1461 · jina-ai/serve · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

docs: crud #1461

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jan 30, 2021
Merged

docs: crud #1461

merged 3 commits into from
Jan 30, 2021

Conversation

cristianmtr
Copy link
Contributor
@cristianmtr cristianmtr commented Dec 14, 2020

Please leave feedback and suggestions

Waiting for all PR and tasks related to CRUD to be implemented and finalized before expanding this.

  • implement update/delete in KV Indexers
  • implement update/delete in Vector Indexers
  • implement update/delete requests
  • DocIdCache to handle content collision
  • integration tests

Relevant PRs/issues to be mentioned in here

@jina-bot jina-bot added size/S area/docs This issue/PR affects the docs labels Dec 14, 2020 8000
@github-actions
Copy link
github-actions bot commented Dec 14, 2020

Latency summary

Current PR yields:

  • 🐎🐎 index QPS at 1198, delta to last 3 avg.: +9%
  • 🐢🐢 query QPS at 27, delta to last 3 avg.: -41%

Breakdown

Version Index QPS Query QPS
current 1198 27
0.9.23 1209 27
0.9.22 979 64

Backed by latency-tracking. Further commits will update this comment.

@cristianmtr cristianmtr requested a review from alexcg1 December 14, 2020 16:16
@cristianmtr cristianmtr force-pushed the docs-crud branch 4 times, most recently from 745c8fb to ff9d5ad Compare December 15, 2020 09:44
@codecov
Copy link
codecov bot commented Dec 15, 2020

Codecov Report

Merging #1461 (04792f9) into master (f7fc6d7) will increase coverage by 0.16%.
The diff coverage is 95.41%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1461      +/-   ##
==========================================
+ Coverage   85.62%   85.79%   +0.16%     
==========================================
  Files         136      138       +2     
  Lines        6942     6981      +39     
==========================================
+ Hits         5944     5989      +45     
+ Misses        998      992       -6     
Impacted Files Coverage Δ
jina/optimizers/parameters.py 100.00% <ø> (ø)
jina/parsers/helper.py 50.33% <33.33%> (-0.35%) ⬇️
jina/clients/request/asyncio.py 89.47% <89.47%> (ø)
jina/clients/base.py 81.30% <91.66%> (+1.30%) ⬆️
jina/clients/asyncio.py 92.59% <100.00%> (+14.81%) ⬆️
jina/clients/request/__init__.py 100.00% <100.00%> (ø)
jina/clients/request/helper.py 100.00% <100.00%> (ø)
jina/clients/websocket.py 95.45% <100.00%> (ø)
jina/flow/asyncio.py 76.00% <100.00%> (+7.81%) ⬆️
jina/optimizers/flow_runner.py 95.65% <100.00%> (+8.69%) ⬆️
... and 8 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7b625a1...ad137c0. Read the comment docs.

@alexcg1
Copy link
Member
alexcg1 commented Dec 15, 2020

Want to me to do it as comments on file change tab? Or better I pull, edit and merge my changes into the branch @cristianmtr ?

@cristianmtr
Copy link
Contributor Author

However is easier for you @alexcg1

@alexcg1
Copy link
Member
alexcg1 commented Dec 15, 2020

However is easier for you @alexcg1

Will merge my changes soon

@alexcg1
Copy link
Member
alexcg1 commented Dec 15, 2020

Nice work. I just added some very minor tweakage

@nan-wang
Copy link
Member

Don't forget to add it to docs/index.rst so that it will show up in the sidebar.

@nan-wang nan-wang added this to the v1.0.0 New Features milestone Jan 7, 2021
@cristianmtr cristianmtr marked this pull request as ready for review January 27, 2021 14:43
@cristianmtr cristianmtr requested a review from a team as a code owner January 27, 2021 14:43
@cristianmtr cristianmtr marked this pull request as draft January 27, 2021 14:45
@cristianmtr cristianmtr marked this pull request as ready for review January 28, 2021 16:37

1. **Advanced Indexers**

`Faiss` & `Annoy` indexers don't support updates while querying. You can still use them in query mode, but you will not be able to add, delete, or update data. In order to do this, you will need use a `ref_indexer`-based approach. An example of this can be found [here](https://github.com/jina-ai/examples/master/advanced-vector-search)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not totally true, I think it works since Faiss and Annoy only start working while Querying.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since update and delete are operations on BaseNumpyIndexer these operations are possible on Faiss and Annoy instances.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-wrote this part

@CatStark
Copy link
Contributor

Really well explained!

@hanxiao hanxiao merged commit 415ccb7 into master Jan 30, 2021
@hanxiao hanxiao deleted the docs-crud branch January 30, 2021 08:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/docs This issue/PR affects the docs size/S
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants
0