8000 Python 3.8 breaks network drive setups · Issue #365 · gcovr/gcovr · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Python 3.8 breaks network drive setups #365

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
Verequus opened this issue Apr 22, 2020 · 6 comments
Closed

Python 3.8 breaks network drive setups #365

Verequus opened this issue Apr 22, 2020 · 6 comments
Labels
Format: HTML Platform: Windows Type: Bug

Comments

@Verequus
Copy link
Contributor

Note: Due to my issue as described in #364, I could not test it with 4.2, only with 4.1, as the relevant code paths aren't triggered, but AFAICT the issue still remains in the current release.

My project uses a setup where on Windows a directory on the C: drive is mapped to the X: (see https://support.microsoft.com/en-us/help/4026635/windows-map-a-network-drive how to do that). gcovr works with Python 3.6 without problems. But when trying to update to Python 3.8 only (and no other relevant changes), running gcovr results in an exception:

Traceback (most recent call last):
File "C:\Program Files\Python38\lib\runpy.py", line 193, in _run_module_as_main
return run_code(code, main_globals, None,
File "C:\Program Files\Python38\lib\runpy.py", line 86, in run_code
exec(code, run_globals)
File "X:\VuC_Build_PythonVenv\lib\site-packages\gcovr_main.py", line 600, in
main()
File "X:\VuC_Build_PythonVenv\lib\site-packages\gcovr_main.py", line 588, in main
print_html_report(covdata, options)
File "X:\VuC_Build_PythonVenv\lib\site-packages\gcovr\html_generator.py", line 220, in print_html_report
filename=os.path.relpath(
File "C:\Program Files\Python38\lib\ntpath.py", line 703, in relpath
raise ValueError("path is on mount %r, start on mount %r" % (
ValueError: path is on mount 'C:', start on mount 'X:'

This can be fixed by changing

            filename=os.path.relpath(
                os.path.realpath(cdata._filename), data['DIRECTORY']),

to

            filename=os.path.relpath(
                os.path.realpath(cdata._filename), os.path.realpath(data['DIRECTORY'])),

But then I get another exception later:

Traceback (most recent call last):
File "C:\Program Files\Python38\lib\runpy.py", line 193, in _run_module_as_main
return run_code(code, main_globals, None,
File "C:\Program Files\Python38\lib\runpy.py", line 86, in run_code
exec(code, run_globals)
File "X:\VuC_Build_PythonVenv\lib\site-packages\gcovr_main.py", line 600, in
main()
File "X:\VuC_Build_PythonVenv\lib\site-packages\gcovr_main.py", line 588, in main
print_html_report(covdata, options)
File "X:\VuC_Build_PythonVenv\lib\site-packages\gcovr\html_generator.py", line 181, in print_html_report
commondir = commonpath(files)
File "X:\VuC_Build_PythonVenv\lib\site-packages\gcovr\utils.py", line 185, in commonpath
return os.path.join(os.path.relpath(os.path.sep.join(common_dirs)), '')
File "C:\Program Files\Python38\lib\ntpath.py", line 703, in relpath
raise ValueError("path is on mount %r, start on mount %r" % (
ValueError: path is on mount 'C:', start on mount 'X:'

I couldn't figure out how to solve this, so I don't know if there any other issues hidden.
@latk
Copy link
Member
latk commented Apr 22, 2020

Oh, this is a tricky error! Thanks for reporting it.

Since this seems to be a fairly niche problem (and since I don't have a Windows computer) I'm afraid I won't spend time investigating it, but fixes would of course be welcome.

I think it would be necessary to audit the path manipulation code and add more realpath() calls as you had started, although I'm afraid this could break symlink handling on other OSes. Gcovr was developed for Mac/Linux first, Windows was just an afterthought. As a consequence, the code assumes that a relative path can always exist.

@Verequus
Copy link
Contributor Author
Verequus commented Apr 23, 2020 •
edited
Loading

Spontaneously I don't think adding more realpath calls breaks in scenarios with a single tree. The relative path can change but regardless of following a symlink or not, there shouldn't be a deadend. At worst some kind of permission problem (difference between regular access and symlink), but shouldn't in such case the symlink not work in the first case?

In any case, if I need to investigate this myself, I'm not willing to look into fixing this on 4.1, since this is an outdated branch. That means I need #364 to be solved first. I'll provide the requested feedback there.

@Verequus
Copy link
Contributor Author
Verequus commented Jun 2, 2020

After some investigation I did find evidence that symlinks not necessarily have the same permissions as the files they are pointing to. Still, this is fixable on the main side by using a special option to trigger a different behavior. In other words, backwards compatibility can be preserved as default.

@dbywalec
Copy link
Contributor

Hi guys,
I have recently stumbled against the same problem. I am running the gcovr script on Windows 10, with Python 3.8.3.
Empty reports are generated.
gcovr script uses relative path to absolute path resolution in numerous places by using os.path.abspath & os.path.realpath in inconsistent manner. These calls are arbitrarily mixed here and there. In many cases they yield the same results. But there is a significant difference: os.path.realpath dereferences and resolves symlinks, whereas os.path.abspath does not.
This has significance in my case, because I also used Windows drive substitution, which is in Python os.path.realpath is considered as symlink and get resolved into original path on C: drive. This causes mixed use of paths with C: drive and substituted drive and path filters match fail.
I will prepare a pull request with potential solution.

@dbywalec dbywalec mentioned this issue Sep 21, 2020
Closed
@dbywalec
Copy link
Contributor

I have some important finding regarding paths handling, that has not been mentioned yet. In Python 3.8 there is a breaking change in os.path.realpath() behavior under Windows:

https://docs.python.org/3/library/os.path.html#os.path.realpath

Changed in version 3.8: Symbolic links and junctions are now resolved on Windows.

I did check the difference across Python 3.7 & 3.8 on Windows. Python 3.7 does not resolve drive mapping (and symbolic links too), where as Python 3.8 does.
This is why inconsistent use of os.path.abspath() & os.path.realpath() becomes troublesome.

@ManPython ManPython mentioned this issue Nov 10, 2021
Open
Verequus added a commit to Verequus/gcovr that referenced this issue Dec 10, 2021
@Verequus
Python 3.8 breaks network drive setups (gcovr#365)
f65feec
@Verequus Verequus mentioned this issue Dec 10, 2021
Closed
@Spacetown Spacetown mentioned this issue Jan 6, 2022
Closed
Verequus added a commit to Verequus/gcovr that referenced this issue Jan 19, 2022
@Verequus
Python 3.8 breaks network drive setups (gcovr#365)
e8209f8
@Spacetown Spacetown mentioned this issue Jan 31, 2022
Merged
@hydrusnetwork hydrusnetwork mentioned this issue Mar 1, 2022
Closed
@Spacetown
Copy link
Member

This is fixed with #565.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Format: HTML Platform: Windows Type: Bug
Projects
None yet
Milestone
No milestone
4 participants
@Verequus @latk @dbywalec @Spacetown
0