8000 fix: no warning for metas in executor by maximilianwerk · Pull Request #3219 · jina-ai/serve · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

fix: no warning for metas in executor #3219

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Aug 20, 2021
Merged

Conversation

maximilianwerk
Copy link
Member
@maximilianwerk maximilianwerk commented Aug 19, 2021

If you use metas inside an Executor definition, there was a warning, that it is an unknown argument beforehands. Even thought it is not really used inside the argparse, this is the correct way in my opinion.

@maximilianwerk maximilianwerk requested a review from a team as a code owner August 19, 2021 06:51
@github-actions github-actions bot added size/S area/core This issue/PR affects the core codebase area/testing This issue/PR affects testing area/cli This issue/PR affects the command line interface labels Aug 19, 2021
@codecov
Copy link
codecov bot commented Aug 19, 2021

Codecov Report

Merging #3219 (35cce1c) into master (4d86d78) will decrease coverage by 0.00%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #3219      +/-   ##
==========================================
- Coverage   90.48%   90.47%   -0.01%     
==========================================
  Files         148      148              
  Lines       10324    10324              
==========================================
- Hits         9342     9341       -1     
- Misses        982      983       +1     
Flag Coverage Δ
daemon 42.85% <ø> (ø)
jina 90.46% <ø> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
jina/peapods/runtimes/jinad/__init__.py 91.42% <0.00%> (-1.91%) ⬇️
jina/peapods/runtimes/container/__init__.py 84.61% <0.00%> (-1.78%) ⬇️
jina/peapods/runtimes/gateway/prefetch.py 95.06% <0.00%> (+1.23%) ⬆️
jina/clients/base/grpc.py 70.21% <0.00%> (+6.38%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0eab182...35cce1c. Read the comment docs.

@github-actions
Copy link
github-actions bot commented Aug 19, 2021

Latency summary

Current PR yields:

  • 🐢🐢 index QPS at 1257, delta to last 2 avg.: -20%
  • 🐢🐢 query QPS at 37, delta to last 2 avg.: -13%
  • 🐢🐢 dam extend QPS at 43791, delta to last 2 avg.: -28%
  • 🐢🐢 avg flow time within 1.9052 seconds, delta to last 2 avg.: +54%
  • 🐎🐎🐎🐎 import jina within 0.3487 seconds, delta to last 2 avg.: +16%

Breakdown

Version Index QPS Query QPS DAM Extend QPS Avg Flow Time (s) Import Time (s)
current 1257 37 43791 1.9052 0.3487
2.0.19 1660 52 65281 1.1928 0.3058
2.0.18 1504 32 57590 1.277 0.2908

Backed by latency-tracking. Further commits will update this comment.

@maximilianwerk
Copy link
Member Author

Can I get some opinions here, if that makes sense. So the point is:

metas is used as an argument for Peas. If we check for known/unknown arguments the parser is the source of truth. Thus, if we are able to specify metas it should consequently also be a parameter, non?

@JoanFM @deepankarm @hanxiao

@JoanFM
Copy link
Contributor
JoanFM commented Aug 19, 2021

I am not sure what's the strategy. I think having the metas defined in the executor config is more handy that having it in the Flow yaml.

Copy link
Member
@hanxiao hanxiao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i need time to figure out why this is necessary, will come back later

@maximilianwerk
Copy link
Member Author

i need time to figure out why this is necessary, will come back later

Following line of thought:

The general target of warn_unknown_args is to give the user a warning they use a not-existing or deprecated argument. This is usually done via checking the unknown_args returning from argparse. If metas is only used somewhere hidden without defined as a proper argument, it will always be treated as an unknown argument. Since we anyhow rely on the power of argparse, all user defined arguments should be represented by one parser or another.

I am not sure, if I 100% agree with this line of thought above, but users complained, that they did not got any warning, when specifying deprecated arguments. So this was my idea of solving it: Get more rigid with arg parsing.

@hanxiao
Copy link
Member
hanxiao commented Aug 19, 2021

the test is incorrect, that YAML in the test is not a valid Flow anyway. so the warning is correct, there is nothing called metas

I'm guessing you wanted to test against the below instead, which gives a valid Flow with one executor and executor has metas name as hello

yaml = '''jtype: Flow
version: 1
executors:
    - uses: 
        jtype: BaseExecutor
        metas:
            name: hello
    '''
with Flow().load_config(yaml):
    pass

which gives no warning on current master

@github-actions github-actions bot removed area/core This issue/PR affects the core codebase area/cli This issue/PR affects the command line interface labels Aug 20, 2021
@maximilianwerk maximilianwerk merged commit 0649cce into master Aug 20, 2021
@maximilianwerk maximilianwerk deleted the fix-metas-in-executor branch August 20, 2021 14:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/testing This issue/PR affects testing size/S
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants
0