8000 feat(driver): add collect evaluation driver by hanxiao · Pull Request #1100 · jina-ai/serve · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

feat(driver): add collect evaluation driver #1100

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 7 commits into from
Oct 16, 2020
Merged

feat(driver): add collect evaluation driver #1100

merged 7 commits into from
Oct 16, 2020

Conversation

hanxiao
Copy link
Member
@hanxiao hanxiao commented Oct 16, 2020 •
edited
Loading
  • .gather_inspect now uses _merge_eval executor by default

@hanxiao hanxiao added this to the v0.7.0 New Features milestone Oct 16, 2020
@JoanFM
Copy link
Contributor
JoanFM commented Oct 16, 2020

We should have 2 levels of grouping, one by docID and another by evaluation identifier. We must consider that every document may be evaluated by different evaluations in the same Flow

@hanxiao
Copy link
Member Author
hanxiao commented Oct 16, 2020

do you mean by op_name? I think that should be straightforward by mocking what this driver does, will leave to the others

@codecov
Copy link
codecov bot commented Oct 16, 2020

Codecov Report

Merging #1100 into master will increase coverage by 0.19%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1100      +/-   ##
==========================================
+ Coverage   80.51%   80.71%   +0.19%     
==========================================
  Files          71       71              
  Lines        5317     5377      +60     
==========================================
+ Hits         4281     4340      +59     
- Misses       1036     1037       +1
Impacted Files Coverage Δ
jina/drivers/reduce.py 98.57% <100.00%> (+0.10%) ⬆️
jina/logging/profile.py 55.81% <0.00%> (-0.66%) ⬇️
jina/peapods/pea.py 92.11% <0.00%> (-0.31%) ⬇️
jina/executors/indexers/cache.py 100.00% <0.00%> (ø)
jina/executors/evaluators/decorators.py 100.00% <0.00%> (ø)
jina/executors/evaluators/rank/recall.py 100.00% <0.00%> (ø)
jina/executors/evaluators/rank/precision.py 100.00% <0.00%> (ø)
jina/peapods/pod.py 79.86% <0.00%> (+<0.01%) ⬆️
jina/drivers/helper.py 98.83% <0.00%> (+0.02%) ⬆️
jina/drivers/evaluate.py 96.96% <0.00%> (+0.09%) ⬆️
... and 12 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 48fa1b5...33c6389. Read the comment docs.

@hanxiao hanxiao merged commit 95e8378 into master Oct 16, 2020
@hanxiao hanxiao deleted the feat-collect-eval-driver branch October 16, 2020 15:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
area/core This issue/PR affects the core codebase area/testing This issue/PR affects testing component/driver component/flow component/resource size/M
Projects
None yet
Milestone
v0.7.0 New Features
Development

Successfully merging this pull request may close these issues.
3 participants
@hanxiao @JoanFM @jina-bot
0