8000 ci: classify flow unit test by alanthssss · Pull Request #3346 · jina-ai/serve · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

ci: classify flow unit test #3346

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Sep 6, 2021
Merged

ci: classify flow unit test #3346

merged 1 commit into from
Sep 6, 2021

Conversation

alanthssss
Copy link
Contributor
@alanthssss alanthssss commented Sep 6, 2021 •
edited
Loading

Unit tests parallelism of Jina

Classify flow unit test and break down the unit test folder, to leverage parallel CI implementation.

  • Previous folder: tests/unit/flow
  • New folders:
    • tests/unit/flow-construct
    • tests/unit/flow-orchestrate
    • tests/unit/flow-example

@alanthssss alanthssss requested a review from a team as a code owner September 6, 2021 08:50
@github-actions github-actions bot added size/S area/testing This issue/PR affects testing labels Sep 6, 2021
@alanthssss alanthssss requested review from bwanglzu and hanxiao and removed request for mapleeit and florian-hoenicke September 6, 2021 08:51
@codecov
Copy link
codecov bot commented Sep 6, 2021 •
edited
Loading

Codecov Report

Merging #3346 (53ac739) into master (1feb09f) will increase coverage by 1.02%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #3346      +/-   ##
==========================================
+ Coverage   88.77%   89.79%   +1.02%     
==========================================
  Files         152      152              
  Lines       10820    10820              
==========================================
+ Hits         9605     9716     +111     
+ Misses       1215     1104     -111
Flag Coverage Δ
daemon 45.61% <ø> (-0.02%) ⬇️
jina 89.78% <ø> (+1.46%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
jina/helper.py 82.97% <0.00%> (-0.37%) ⬇️
jina/peapods/zmq/__init__.py 89.04% <0.00%> (+0.69%) ⬆️
jina/peapods/runtimes/jinad/__init__.py 91.42% <0.00%> (+0.95%) ⬆️
jina/peapods/runtimes/gateway/http/app.py 92.40% <0.00%> (+1.26%) ⬆️
jina/peapods/pods/compound.py 91.72% <0.00%> (+1.37%) ⬆️
jina/types/message/__init__.py 87.37% <0.00%> (+1.45%) ⬆️
jina/peapods/runtimes/zmq/zed.py 94.38% <0.00%> (+1.53%) ⬆️
jina/jaml/parsers/flow/v1.py 98.24% <0.00%> (+1.75%) ⬆️
jina/peapods/runtimes/zmq/asyncio.py 92.98% <0.00%> (+1.75%) ⬆️
jina/jaml/helper.py 85.36% <0.00%> (+2.43%) ⬆️
... and 10 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 705dd27...53ac739. Read the comment docs.

@github-actions
Copy link
github-actions bot commented Sep 6, 2021

Latency summary

Current PR yields:

  • 🐎🐎🐎🐎 index QPS at 1224, delta to last 2 avg.: +18%
  • 🐎🐎🐎🐎 query QPS at 51, delta to last 2 avg.: +22%
  • 🐎🐎🐎🐎 dam extend QPS at 51933, delta to last 2 avg.: +33%
  • 🐎🐎🐎🐎 avg flow time within 1.4849 seconds, delta to last 2 avg.: +2%
  • 🐢🐢 import jina within 0.3588 seconds, delta to last 2 avg.: -16%

Breakdown

Version Index QPS Query QPS DAM Extend QPS Avg Flow Time (s) Import Time (s)
current 1224 51 51933 1.4849 0.3588
2.0.23 1060 42 38589 1.2255 0.4266
2.0.22 1006 40 39107 1.6632 0.4366

Backed by latency-tracking. Further commits will update this comment.

@CatStark CatStark linked an issue Sep 6, 2021 that may be closed by this pull request
Break down tests/unit/flow #3345
Closed
@bwanglzu
Copy link
Member
bwanglzu commented Sep 6, 2021

can you elaborate a bit in the thread on how we classify and how could we put future flow test into different folders?

@alanthssss
Copy link
Contributor Author
alanthssss commented Sep 6, 2021 •
edited
Loading

can you elaborate a bit in the thread on how we classify and how could we put future flow test into different folders?

@bwanglzu

Sure!
Here is it:

$:~/code/jina-ai/jina/tests/unit$ tree flow-* -L 1
flow-construct
├── __init__.py
├── mwu-encoder
├── __pycache__
├── test_flow_enter_error.py
├── test_flow_except.py
├── test_flow.py
├── test_flow_skip.py
├── test_flow_start_noblock.py
├── test_flow_visualization.py
├── test_flow_yaml_parser.py
├── test_slow_executor_shutdown.py
└── yaml
flow-orchestrate
├── __init__.py
├── __pycache__
├── test_asyncflow.py
├── test_flow_before_after.py
├── test_flow_change_gateway.py
├── test_flow_dynamic_routing.py
├── test_flow_http_extend.py
├── test_flow_merge.py
├── test_remote_orchestrate.py
└── yaml
flow-example
├── __init__.py
├── __pycache__
├── test_flow_example_face.py
├── test_flow_example_faiss.py
├── test_flow_segmenter.py
└── yaml
  • flow-construct: Basic items in flow
  • flow-orchestrate: Orchestration of flow
  • flow-examples: Whole flow example

@hanxiao
Copy link
Member
hanxiao commented Sep 6, 2021

lgtm, just fyi on the timing
image

@hanxiao hanxiao merged commit 9a8c431 into master Sep 6, 2021
@hanxiao hanxiao deleted the unittest-flow-classify branch September 6, 2021 10:14
deepankarm pushed a commit that referenced this pull request Sep 14, 2021
@deepankarm
ci: classify flow unit test (#3346)
81bf0f3
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@hanxiao hanxiao hanxiao approved these changes

@bwanglzu bwanglzu Awaiting requested review from bwanglzu

Assignees
No one assigned
Labels
area/testing This issue/PR affects testing size/S
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Break down tests/unit/flow
3 participants
@alanthssss @bwanglzu @hanxiao
0