8000 Explicitly set pytest rootdir option for running tests by taldcroft · Pull Request #52 · sot/testr · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Explicitly set pytest rootdir option for running tests #52

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Dec 29, 2024
Merged

Explicitly set pytest rootdir option for running tests #52

merged 2 commits into from
Dec 29, 2024

Conversation

taldcroft
Copy link
Member
@taldcroft taldcroft commented Dec 29, 2024 •
edited by jeanconn
Loading

Description

Fixes sot/skare3#1294

Interface impacts

Tested with pytest == 8.3.4 .

Testing

Unit tests

  • No unit tests

Functional tests

Hoping for help from @jeanconn or @javierggt.

Testing .test()

From this github repo, test the test runner via the >>> package.test(*args, **kwargs) path.

masters) ➜  testr git:(set-rootdir) ipython
Python 3.11.8 | packaged by conda-forge | (main, Feb 16 2024, 20:49:36) [Clang 16.0.6 ]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.29.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import ska_helpers

In [2]: ska_helpers.test(**{"-k": "utils"})
================================================================== test session starts ==================================================================
platform darwin -- Python 3.11.8, pytest-8.3.4, pluggy-1.5.0
rootdir: /Users/aldcroft/miniconda3-arm/envs/masters/lib/python3.11/site-packages
plugins: timeout-2.2.0, anyio-4.3.0
collected 57 items / 27 deselected / 30 selected

ska_helpers/tests/test_utils.py ..............................                                                                                    [100%]

=========================================================== 30 passed, 27 deselected in 0.83s ===========================================================
Out[2]: False

In [3]: ska_helpers.test("-k", "utils")
================================================================== test session starts ==================================================================
platform darwin -- Python 3.11.8, pytest-8.3.4, pluggy-1.5.0
rootdir: /Users/aldcroft/miniconda3-arm/envs/masters/lib/python3.11/site-packages
plugins: timeout-2.2.0, anyio-4.3.0
collected 57 items / 27 deselected / 30 selected

ska_helpers/tests/test_utils.py ..............................                                                                                    [100%]

=========================================================== 30 passed, 27 deselected in 0.43s ===========================================================
Out[3]: False

In [4]: ska_helpers.test("-k=utils")
================================================================== test session starts ==================================================================
platform darwin -- Python 3.11.8, pytest-8.3.4, pluggy-1.5.0
rootdir: /Users/aldcroft/miniconda3-arm/envs/masters/lib/python3.11/site-packages
plugins: timeout-2.2.0, anyio-4.3.0
collected 57 items / 27 deselected / 30 selected

ska_helpers/tests/test_utils.py ..............................                                                                                    [100%]

=========================================================== 30 passed, 27 deselected in 0.41s ===========================================================
Out[4]: False

I also put in a temporary statement to print the args that get passed to the pytest call and confirmed the expected --rootdir.

In [3]: ska_helpers.test("-k", "utils")
('-k', 'utils', '-p', 'no:cacheprovider', '-p', 'no:hypothesispytest', '-p', 'no:hypothesis', '--rootdir=/Users/aldcroft/miniconda3-arm/envs/masters/lib/python3.11/site-packages')

For other functional testing, @jeanconn ran tests for a single package from ska_testr and confirmed success with pytest 8.3.4

(perl-play) jeanconn-fido> cd ~/git/ska_testr
(perl-play) jeanconn-fido> export PYTHONPATH=/home/jeanconn/git/testr
(perl-play) jeanconn-fido> python -c "import testr; print(testr.__version__)"
4.12.1.dev3+g9e6bb45
(perl-play) jeanconn-fido> python -c "import pytest; print(pytest.__version__)"
8.3.4
(perl-play) jeanconn-fido> run_testr --include Quaternion
...
****************************************
*** package Quaternion               ***
****************************************

Copying input tests /proj/sot/ska/jeanproj/git/ska_testr/packages/Quaternion to output dir /proj/sot/ska/jeanproj/git/ska_testr/outputs/logs/Linux_2024-12-29T15-32-22_2af167b_fido.cfa.harvard.edu/Quaternion
Running python test_unit.py script
============================= test session starts ==============================
platform linux -- Python 3.12.8, pytest-8.3.4, pluggy-1.5.0 -- /fido.real/miniforge3/envs/perl-play/bin/python
rootdir: /fido.real/miniforge3/envs/perl-play/lib/python3.12/site-packages
configfile: ../../../../../../../proj/sot/ska/jeanproj/git/ska_testr/pytest.ini
plugins: timeout-2.3.1, anyio-4.7.0, typeguard-4.3.0
collecting ... collected 72 items

Quaternion/tests/test_all.py::test_shape PASSED
Quaternion/tests/test_all.py::test_init_exceptions PASSED
Quaternion/tests/test_all.py::test_from_q PASSED
Quaternion/tests/test_all.py::test_from_eq PASSED
Quaternion/tests/test_all.py::test_from_eq_vectorized PASSED
Quaternion/tests/test_all.py::test_from_eq_shapes PASSED
Quaternion/tests/test_all.py::test_transform_from_eq PASSED
Quaternion/tests/test_all.py::test_from_transform PASSED
Quaternion/tests/test_all.py::test_from_transform_vectorized PASSED
Quaternion/tests/test_all.py::test_eq_from_transform PASSED
Quaternion/tests/test_all.py::test_from_q_vectorized PASSED
Quaternion/tests/test_all.py::test_inv_eq PASSED
Quaternion/tests/test_all.py::test_inv_q PASSED
Quaternion/tests/test_all.py::test_inv_vectorized PASSED
Quaternion/tests/test_all.py::test_dq PASSED
Quaternion/tests/test_all.py::test_dq_vectorized PASSED
Quaternion/tests/test_all.py::test_vector_to_scalar_correspondence PASSED
Quaternion/tests/test_all.py::test_ra0_roll0 PASSED
Quaternion/tests/test_all.py::test_repr PASSED
Quaternion/tests/test_all.py::test_numeric_underflow PASSED
Quaternion/tests/test_all.py::test_div_mult PASSED
Quaternion/tests/test_all.py::test_mult_vectorized PASSED
Quaternion/tests/test_all.py::test_normalize PASSED
Quaternion/tests/test_all.py::test_copy PASSED
Quaternion/tests/test_all.py::test_format ra=10.00000, dec=20.00000, roll=30.00000
PASSED
Quaternion/tests/test_all.py::test_scalar_attribute_types PASSED
Quaternion/tests/test_all.py::test_mult_and_dq_broadcasted PASSED
Quaternion/tests/test_all.py::test_array_attribute_types PASSED
Quaternion/tests/test_all.py::test_pickle PASSED
Quaternion/tests/test_all.py::test_init_quat_from_attitude PASSED
Quaternion/tests/test_all.py::test_rotate_x_to_vec_regress PASSED
Quaternion/tests/test_all.py::test_rotate_x_to_vec_functional[keep_z] PASSED
Quaternion/tests/test_all.py::test_rotate_x_to_vec_functional[shortest] PASSED
Quaternion/tests/test_all.py::test_rotate_x_to_vec_functional[radec] PASSED
Quaternion/tests/test_all.py::test_rotate_x_to_vec_bad_method PASSED
Quaternion/tests/test_all.py::test_rotate_about_vec PASSED
Quaternion/tests/test_all.py::test_rotate_about_vec_exceptions PASSED
Quaternion/tests/test_all.py::test_setting_different_shape[q] PASSED
Quaternion/tests/test_all.py::test_setting_different_shape[equatorial] PASSED
Quaternion/tests/test_all.py::test_setting_different_shape[transform] PASSED
Quaternion/tests/test_all.py::test_quat_to_equatorial PASSED
Quaternion/tests/test_all.py::test_quat_mult PASSED
Quaternion/tests/test_all.py::test_quat_descriptor_not_required_no_default PASSED
Quaternion/tests/test_all.py::test_quat_descriptor_is_required PASSED
Quaternion/tests/test_all.py::test_quat_descriptor_has_default PASSED
Quaternion/tests/test_all.py::test_quat_descriptor_is_required_has_default_exception PASSED
Quaternion/tests/test_shaped.py::test_getitem[q] PASSED
Quaternion/tests/test_shaped.py::test_getitem[equatorial] PASSED
Quaternion/tests/test_shaped.py::test_getitem[transform] PASSED
Quaternion/tests/test_shaped.py::test_shape_changing_methods[copy] PASSED
Quaternion/tests/test_shaped.py::test_shape_changing_methods[reshape] PASSED
Quaternion/tests/test_shaped.py::test_shape_changing_methods[transpose] PASSED
Quaternion/tests/test_shaped.py::test_shape_changing_methods[flatten] PASSED
Quaternion/tests/test_shaped.py::test_shape_changing_methods[ravel] PASSED
Quaternion/tests/test_shaped.py::test_shape_changing_methods[swapaxes] PASSED
Quaternion/tests/test_shaped.py::test_shape_changing_methods[diagonal] PASSED
Quaternion/tests/test_shaped.py::test_shape_changing_methods[squeeze] PASSED
Quaternion/tests/test_shaped.py::test_shape_changing_methods[take] PASSED
Quaternion/tests/test_shaped.py::test_shape_changing_T_property PASSED
Quaternion/tests/test_shaped.py::test_applicable_methods[moveaxis-args0] PASSED
Quaternion/tests/test_shaped.py::test_applicable_methods[rollaxis-args1] PASSED
Quaternion/tests/test_shaped.py::test_applicable_methods[atleast_1d-args2] PASSED
Quaternion/tests/test_shaped.py::test_applicable_methods[atleast_2d-args3] PASSED
Quaternion/tests/test_shaped.py::test_applicable_methods[atleast_3d-args4] PASSED
Quaternion/tests/test_shaped.py::test_applicable_methods[expand_dims-args5] PASSED
Quaternion/tests/test_shaped.py::test_applicable_methods[broadcast_to-args6] PASSED
Quaternion/tests/test_shaped.py::test_applicable_methods[flip-args7] PASSED
Quaternion/tests/test_shaped.py::test_applicable_methods[fliplr-args8] PASSED
Quaternion/tests/test_shaped.py::test_applicable_methods[flipud-args9] PASSED
Quaternion/tests/test_shaped.py::test_applicable_methods[rot90-args10] PASSED
Quaternion/tests/test_shaped.py::test_applicable_methods[roll-args11] PASSED
Quaternion/tests/test_shaped.py::test_applicable_methods[delete-args12] PASSED

- generated xml file: /proj/sot/ska/jeanproj/git/ska_testr/outputs/logs/Linux_2024-12-29T15-32-22_2af167b_fido.cfa.harvard.edu/Quaternion/test_unit.py.xml -
============================== 72 passed in 2.70s ==============================
Running python post_check_logs.py script
****************************************
*** Quaternion Test Summary          ***
*** test_unit.py         pass        ***
*** post_check_logs.py   pass        ***
****************************************

The rootdir looks appropriate to my installation, though it is a bit amusing that the pytest.ini is now relative.

For reference, this was the top matter with testr master

============================= test session starts ==============================
platform linux -- Python 3.12.8, pytest-8.3.4, pluggy-1.5.0 -- /fido.real/miniforge3/envs/perl-play/bin/python
rootdir: /proj/sot/ska/jeanproj/git/ska_testr
configfile: pytest.ini
plugins: timeout-2.3.1, anyio-4.7.0, typeguard-4.3.0
collecting ... collected 72 items

../../../../../../../proj/sot/ska/jeanproj/git/ska_testr/tests/test_all.py::test_shape PASSED

@taldcroft taldcroft mentioned this pull request Dec 29, 2024
Closed
@jeanconn
Copy link
Contributor
jeanconn commented Dec 29, 2024 •
edited
Loading

When trying to run ska_testr / testr from this PR I'm now getting

****************************************
*** package Quaternion               ***
****************************************

Copying input tests /proj/sot/ska/jeanproj/git/ska_testr/packages/Quaternion to output dir /proj/sot/ska/jeanproj/git/ska_testr/outputs/logs/Linux_2024-12-29T13-19-49_2af167b_fido.cfa.harvard.edu/Quaternion
Running python test_unit.py script
Traceback (most recent call last):
  File "/proj/sot/ska/jeanproj/git/ska_testr/outputs/logs/Linux_2024-12-29T13-19-49_2af167b_fido.cfa.harvard.edu/Quaternion/test_unit.py", line 2, in <module>
    testr.testr()
  File "/home/jeanconn/git/testr/testr/runner.py", line 51, in testr
    return test(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/jeanconn/git/testr/testr/runner.py", line 215, in test
    rc = pytest.main([pkg_dir] + list(args), **kwargs)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: main() got an unexpected keyword argument 'rootdir'

@jeanconn
Copy link
Contributor

I had been playing with "args = args + ('--rootdir', pkg_dir)" but hadn't looked at pytest args vs kwargs.

@jeanconn
Copy link
Contributor

I put what I was doing that seemed to work (without as many ../../../../ paths in the test output as testr master) in #53 . I think we need a real 2025.1 candidate environment with a defined pytest version though before we can do appropriate functional testing and pick something reasonable for our use cases.

jeanconn
jeanconn requested changes Dec 29, 2024
View reviewed changes
Copy link
Contributor
@jeanconn jeanconn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did not work for me in a standard ska_testr / run_testr use case.

< 8000 a class="d-inline-block" data-hovercard-type="user" data-hovercard-url="/users/taldcroft/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="/taldcroft">@taldcroft
Copy link
Member Author

@jeanconn - OK, the first draft wasn't entirely baked, but please give the latest a go. I realized that pytest.main() basically does not take keyword arguments, so I removed that from the call in testr.

https://github.com/pytest-dev/pytest/blob/9c02a3f6ed59215a06fb146ccb573182e9481e06/src/_pytest/config/__init__.py#L138

@taldcroft
Copy link
Member Author
taldcroft commented Dec 29, 2024 •
edited
Loading

And I added some functional testing which covers the other half of the main runner functionality import package; package.test().

jeanconn
jeanconn approved these changes Dec 29, 2024
View reviewed changes
Copy link
Contributor
@jeanconn jeanconn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK. This works for me now in the standard use cases.

@jeanconn jeanconn mentioned this pull request Dec 29, 2024
Closed
5 tasks
@taldcroft taldcroft merged commit ad2ef37 into master Dec 29, 2024
@taldcroft taldcroft deleted the set-rootdir branch December 29, 2024 19:54
This was referenced Jan 23, 2025
Merged
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jeanconn jeanconn jeanconn approved these changes

@javierggt javierggt Awaiting requested review from javierggt

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

ska3-core to use pytest < 8.0.0
2 participants
@taldcroft @jeanconn
0