-
Notifications
You must be signed in to change notification settings - Fork 56
Refactor the the ParsedTypeDocstring #874
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Refactor the the ParsedTypeDocstring #874
Conversation
…de> tags are now added by the html translator when the document is a docstring. Otherwise it does not add the enclosing <code> tags because we're already in the middle of a code tag or similar <span class="rst-literal">. Adjust the themes so the <code> tags and <span class="rst-literal"> are really equivalent.
pydoctor/epydoc/markup/_types.py
Outdated
|
||
combined_tokens: list[tuple[Any, TokenType]] = [] | ||
|
||
open_parenthesis = 0 | ||
open_square_braces = 0 | ||
|
||
for _token, _type in tokens: | ||
# The actual type of_token is str | Tag | Node. | ||
|
||
if (_type is TokenType.DELIMITER and _token in ('[', '(', ')', ']')) \ | ||
or _type is TokenType.OBJ: | ||
if _token == "[": open_square_braces += 1 | ||
elif _token == "(": open_parenthesis += 1 | ||
|
||
if _type is TokenType.OBJ: | ||
_token = docstring_linker.link_xref( | ||
_token, _token, self._lineno) | ||
|
||
if open_square_braces + open_parenthesis > 0: | ||
try: last_processed_token = combined_tokens[-1] | ||
except IndexError: | ||
combined_tokens.append((_token, _type)) | ||
else: | ||
if last_processed_token[1] is TokenType.OBJ \ | ||
and isinstance(last_processed_token[0], Tag): | ||
# Merge with last Tag | ||
if _type is TokenType.OBJ: | ||
assert isinstance(_token, Tag) | ||
last_processed_token[0](*_token.children) | ||
else: | ||
last_processed_token[0](_token) | ||
else: | ||
combined_tokens.append((_token, _type)) | ||
else: | ||
combined_tokens.append((_token, _type)) | ||
|
||
if _token == "]": open_square_braces -= 1 | ||
elif _token == ")": open_parenthesis -= 1 | ||
|
||
else: | ||
# the token will be processed in _convert_type_spec_to_stan() method. | ||
combined_tokens.append((_token, _type)) | ||
|
||
return combined_tokens |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This was the unfortunate code
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it does look unfortunate, nice to see it go
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #874 +/- ##
==========================================
+ Coverage 93.11% 93.13% +0.02%
==========================================
Files 47 47
Lines 8699 8702 +3
Branches 1606 1597 -9
==========================================
+ Hits 8100 8105 +5
+ Misses 337 336 -1
+ Partials 262 261 -1 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
1 similar comment
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This reverts commit c6ffcef.
This comment has been minimized.
This comment has been minimized.
The warning diff is OK, I've double checked it and it appears that pydoctor was reporting link not found at several unrelated places, increasing the number of warnings uselessly. |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
1 similar comment
This comment has been minimized.
This comment has been minimized.
test:36: bad rendering of class signature: SAXParseException: <unknown>.+ undefined entity | ||
'''.splitlines() | ||
|
||
# Some how the type processing get rid of the non breaking spaces, but it's more an implementation |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this implementation detail is now gone.
else: | ||
if isinstance(resolved, str): | ||
xref = intersphinx_link(label, url=resolved) | ||
else: | ||
xref = taglink(resolved, self.page_url, label) | ||
|
||
return tags.code(xref) | ||
return xref |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This change is rather important: it makes the linker always produce a <a>
tag only and never wrap it in a <code>
tag. This is good because having the code tag is rather a presentation matter and the linker should,.. well link. This transfers the presentation responsibility to the node2stan.py
module that will include <code>
or not based on the source of the document.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
Diff from pydoctor_primer, showing the effect of this PR on open source code: numpy (https://github.com/numpy/numpy)
- /projects/numpy/numpy/_core/_ufunc_config.py:389: bad docstring: invalid value set (missing closing brace): {divide
+ /projects/numpy/numpy/_core/_ufunc_config.py:388: bad docstring: invalid value set (missing closing brace): {divide
- /projects/numpy/numpy/_core/_ufunc_config.py:389: bad docstring: invalid value set (missing opening brace): invalid}
+ /projects/numpy/numpy/_core/_ufunc_config.py:388: bad docstring: invalid value set (missing opening brace): invalid}
- /projects/numpy/numpy/_core/fromnumeric.py:387: bad docstring: invalid value set (missing closing brace): {'raise'(
+ /projects/numpy/numpy/_core/fromnumeric.py:386: bad docstring: invalid value set (missing closing brace): {'raise'(
- /projects/numpy/numpy/_core/fromnumeric.py:387: bad docstring: invalid value set (missing opening brace): }
+ /projects/numpy/numpy/_core/fromnumeric.py:386: bad docstring: invalid value set (missing opening brace): }
- /projects/numpy/numpy/_core/fromnumeric.py:387: bad docstring: unbalanced parenthesis in type expression
+ /projects/numpy/numpy/_core/fromnumeric.py:386: bad docstring: unbalanced parenthesis in type expression
- /projects/numpy/numpy/_core/fromnumeric.py:3899: bad docstring: invalid value set (missing closing brace): {int
+ /projects/numpy/numpy/_core/fromnumeric.py:3898: bad docstring: invalid value set (missing closing brace): {int
- /projects/numpy/numpy/_core/fromnumeric.py:3899: bad docstring: invalid value set (missing opening brace): float}
+ /projects/numpy/numpy/_core/fromnumeric.py:3898: bad docstring: invalid value set (missing opening brace): float}
- /projects/numpy/numpy/_core/fromnumeric.py:3926: bad docstring: invalid value set (missing closing brace): {int
+ /projects/numpy/numpy/_core/fromnumeric.py:3925: bad docstring: invalid value set (missing closing brace): {int
- /projects/numpy/numpy/_core/fromnumeric.py:3926: bad docstring: invalid value set (missing opening brace): float}
+ /projects/numpy/numpy/_core/fromnumeric.py:3925: bad docstring: invalid value set (missing opening brace): float}
- /projects/numpy/numpy/_core/fromnumeric.py:4103: bad docstring: invalid value set (missing closing brace): {int
+ /projects/numpy/numpy/_core/fromnumeric.py:4102: bad docstring: invalid value set (missing closing brace): {int
- /projects/numpy/numpy/_core/fromnumeric.py:4103: bad docstring: invalid value set (missing opening brace): float}
+ /projects/numpy/numpy/_core/fromnumeric.py:4102: bad docstring: invalid value set (missing opening brace): float}
- /projects/numpy/numpy/_core/fromnumeric.py:4130: bad docstring: invalid value set (missing closing brace): {int
+ /projects/numpy/numpy/_core/fromnumeric.py:4129: bad docstring: invalid value set (missing closing brace): {int
- /projects/numpy/numpy/_core/fromnumeric.py:4130: bad docstring: invalid value set (missing opening brace): float}
+ /projects/numpy/numpy/_core/fromnumeric.py:4129: bad docstring: invalid value set (missing opening brace): float}
- /projects/numpy/numpy/_core/einsumfunc.py:776: bad docstring: invalid value set (missing closing brace): {bool
+ /projects/numpy/numpy/_core/einsumfunc.py:775: bad docstring: invalid value set (missing closing brace): {bool
- /projects/numpy/numpy/_core/einsumfunc.py:776: bad docstring: invalid value set (missing opening brace): }
+ /projects/numpy/numpy/_core/einsumfunc.py:775: bad docstring: invalid value set (missing opening brace): }
- /projects/numpy/numpy/_core/einsumfunc.py:1090: bad docstring: invalid value set (missing closing brace): {data-type
+ /project
1E79
s/numpy/numpy/_core/einsumfunc.py:1089: bad docstring: invalid value set (missing closing brace): {data-type
- /projects/numpy/numpy/_core/einsumfunc.py:1090: bad docstring: invalid value set (missing opening brace): None}
+ /projects/numpy/numpy/_core/einsumfunc.py:1089: bad docstring: invalid value set (missing opening brace): None}
- /projects/numpy/numpy/_core/einsumfunc.py:1114: bad docstring: invalid value set (missing closing brace): {False
+ /projects/numpy/numpy/_core/einsumfunc.py:1113: bad docstring: invalid value set (missing closing brace): {False
- /projects/numpy/numpy/_core/einsumfunc.py:1114: bad docstring: invalid value set (missing opening brace): }
+ /projects/numpy/numpy/_core/einsumfunc.py:1113: bad docstring: invalid value set (missing opening brace): }
- /projects/numpy/numpy/lib/_datasource.py:175: bad docstring: invalid value set (missing closing brace): {None
+ /projects/numpy/numpy/lib/_datasource.py:174: bad docstring: invalid value set (missing closing brace): {None
- /projects/numpy/numpy/lib/_datasource.py:175: bad docstring: invalid value set (missing opening brace): str}
+ /projects/numpy/numpy/lib/_datasource.py:174: bad docstring: invalid value set (missing opening brace): str}
- /projects/numpy/numpy/lib/_datasource.py:177: bad docstring: invalid value set (missing closing brace): {None
+ /projects/numpy/numpy/lib/_datasource.py:176: bad docstring: invalid value set (missing closing brace): {None
- /projects/numpy/numpy/lib/_datasource.py:177: bad docstring: invalid value set (missing opening brace): str}
+ /projects/numpy/numpy/lib/_datasource.py:176: bad docstring: invalid value set (missing opening brace): str}
+ /projects/numpy/numpy/lib/_datasource.py:499: bad docstring: invalid value set (missing closing brace): {None
+ /projects/numpy/numpy/lib/_datasource.py:499: bad docstring: invalid value set (missing opening brace): str}
- /projects/numpy/numpy/lib/_datasource.py:500: bad docstring: invalid value set (missing closing brace): {None
+ /projects/numpy/numpy/lib/_datasource.py:501: bad docstring: invalid value set (missing closing brace): {None
- /projects/numpy/numpy/lib/_datasource.py:500: bad docstring: invalid value set (missing opening brace): str}
+ /projects/numpy/numpy/lib/_datasource.py:501: bad docstring: invalid value set (missing opening brace): str}
- /projects/numpy/numpy/lib/_datasource.py:502: bad docstring: invalid value set (missing closing brace): {None
- /projects/numpy/numpy/lib/_datasource.py:502: bad docstring: invalid value set (missing opening brace): str}
- /projects/numpy/numpy/lib/_datasource.py:669: bad docstring: invalid value set (missing closing brace): {None
+ /projects/numpy/numpy/lib/_datasource.py:668: bad docstring: invalid value set (missing closing brace): {None
- /projects/numpy/numpy/lib/_datasource.py:669: bad docstring: invalid value set (missing opening brace): str}
+ /projects/numpy/numpy/lib/_datasource.py:668: bad docstring: invalid value set (missing opening brace): str}
- /projects/numpy/numpy/lib/_datasource.py:671: bad docstring: invalid value set (missing closing brace): {None
+ /projects/numpy/numpy/lib/_datasource.py:670: bad docstring: invalid value set (missing closing brace): {None
- /projects/numpy/numpy/lib/_datasource.py:671: bad docstring: invalid value set (missing opening brace): str}
+ /projects/numpy/numpy/lib/_datasource.py:670: bad docstring: invalid value set (missing opening brace): str}
- /projects/numpy/numpy/lib/_npyio_impl.py:333: bad docstring: invalid value set (missing closing brace): {None
+ /projects/numpy/numpy/lib/_npyio_impl.py:332: bad docstring: invalid value set (missing closing brace): {None
- /projects/numpy/numpy/lib/_npyio_impl.py:333: bad docstring: invalid value set (missing opening brace): }
+ /projects/numpy/numpy/lib/_npyio_impl.py:332: bad docstring: invalid value set (missing opening brace): }
- /projects/numpy/numpy/lib/_npyio_impl.py:1455: bad docstring: invalid value set (missing closing brace): {None
+ /projects/numpy/numpy/lib/_npyio_impl.py:1454: bad docstring: invalid value set (missing closing brace): {None
- /projects/numpy/numpy/lib/_npyio_impl.py:1455: bad docstring: invalid value set (missing opening brace): str}
+ /projects/numpy/numpy/lib/_npyio_impl.py:1454: bad docstring: invalid value set (missing opening brace): str}
- /projects/numpy/numpy/lib/_npyio_impl.py:1809: bad docstring: invalid value set (missing closing brace): {None
+ /projects/numpy/numpy/lib/_npyio_impl.py:1808: bad docstring: invalid value set (missing closing brace): {None
- /projects/numpy/numpy/lib/_npyio_impl.py:1809: bad docstring: invalid value set (missing opening brace): sequence}
+ /projects/numpy/numpy/lib/_npyio_impl.py:1808: bad docstring: invalid value set (missing opening brace): sequence}
- /projects/numpy/numpy/lib/_npyio_impl.py:1827: bad docstring: invalid value set (missing closing brace): {True
+ /projects/numpy/numpy/lib/_npyio_impl.py:1826: bad docstring: invalid value set (missing closing brace): {True
- /projects/numpy/numpy/lib/_npyio_impl.py:1827: bad docstring: invalid value set (missing opening brace): }
+ /projects/numpy/numpy/lib/_npyio_impl.py:1826: bad docstring: invalid value set (missing opening brace): }
- /projects/numpy/numpy/lib/_function_base_impl.py:1003: bad docstring: invalid value set (missing closing brace): {1
+ /projects/numpy/numpy/lib/_function_base_impl.py:1002: bad docstring: invalid value set (missing closing brace): {1
- /projects/numpy/numpy/lib/_function_base_impl.py:1003: bad docstring: invalid value set (missing opening brace): 2}
+ /projects/numpy/numpy/lib/_function_base_impl.py:1002: bad docstring: invalid value set (missing opening brace): 2}
- /projects/numpy/numpy/lib/_function_base_impl.py:3933: bad docstring: invalid value set (missing closing brace): {int
+ /projects/numpy/numpy/lib/_function_base_impl.py:3932: bad docstring: invalid value set (missing closing brace): {int
- /projects/numpy/numpy/lib/_function_base_impl.py:3933: bad docstring: invalid value set (missing opening brace): None}
+ /projects/numpy/numpy/lib/_function_base_impl.py:3932: bad docstring: invalid value set (missing opening brace): None}
- /projects/numpy/numpy/lib/_function_base_impl.py:4094: bad docstring: invalid value set (missing closing brace): {int
... (truncated 502 lines) ...
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Generally looks good. This is wandering pretty far afield of my usual usage of pydoctor, but just some minor code quality issues inline.
pydoctor/epydoc/markup/_types.py
Outdated
|
||
# TODO: This class should use composition instead of multiple inheritence... |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When you mark for a follow-up like this, can you also add a link in the code, to ensure that when it's addressed, the reader can tell if the comment is stale or not?
pydoctor/epydoc/markup/_types.py
Outdated
def __init__(self, annotation: nodes.document, | ||
warns_on_unknown_tokens: bool = False, | ||
lineno: int = 0) -> None: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The indentation flourish here reminds me that we need to apply black
to this project :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes we need to do it, but I did not had the courage to deal with the mypy issues generated by the formatting.
#859
pydoctor/epydoc/markup/_types.py
Outdated
TypeDocstring.__init__(self, annotation, warns_on_unknown_tokens) | ||
|
||
_tokens = self._tokenize_node_type_spec(annotation) | ||
# yes this overrides the superclass type! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
mypy is telling us that we override the superclass type but it would be more helpful to say why :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since the only answer to that question was “because I’m lazy” (not quite satisfactory), I couldn’t resist fixing the design so it uses composition instead of inheritance.
pydoctor/epydoc/markup/_types.py
Outdated
_tokens = self._tokenize_node_type_spec(annotation) | ||
# yes this overrides the superclass type! | ||
self._tokens: list[tuple[str | nodes.Node, TokenType]] \ | ||
= self._build_tokens(_tokens) # type: ignore |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
type:ignore
s should always be as narrow as possible, to avoid over-broad ignores; so, this should be# type:ignore[assignment]
- there are actually 2 things going on in this statement; changing the type of
self._tokens
, and assigning it to the value of_build_tokens
, which means that thetype:ignore
is going to catch two errors, even if you narrow it to[assignment]
, and thus kinda obviating the point of having a type declaration at all. I would sugggest reverting to what you had before in terms of declaring the field at class scope, leaving thetype:ignore
there, and then removing thecast
here, which should no longer be necessary.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This will be addressed
pydoctor/epydoc/markup/_types.py
Outdated
# we're re-using the variable string css | ||
# class for the whole literal token, it's the | ||
# best approximation we have for now. | ||
TokenType.LITERAL: lambda _token, _: \ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
the backslashes here are unnecessary; you're in an enclosed expression context because of the outer {}
, so you can just remove them.
pydoctor/epydoc/markup/_types.py
Outdated
|
||
combined_tokens: list[tuple[Any, TokenType]] = [] | ||
|
||
open_parenthesis = 0 | ||
open_square_braces = 0 | ||
|
||
for _token, _type in tokens: | ||
# The actual type of_token is str | Tag | Node. | ||
|
||
if (_type is TokenType.DELIMITER and _token in ('[', '(', ')', ']')) \ | ||
or _type is TokenType.OBJ: | ||
if _token == "[": open_square_braces += 1 | ||
elif _token == "(": open_parenthesis += 1 | ||
|
||
if _type is TokenType.OBJ: | ||
_token = docstring_linker.link_xref( | ||
_token, _token, self._lineno) | ||
|
||
if open_square_braces + open_parenthesis > 0: | ||
try: last_processed_token = combined_tokens[-1] | ||
except IndexError: | ||
combined_tokens.append((_token, _type)) | ||
else: | ||
if last_processed_token[1] is TokenType.OBJ \ | ||
and isinstance(last_processed_token[0], Tag): | ||
# Merge with last Tag | ||
if _type is TokenType.OBJ: | ||
assert isinstance(_token, Tag) | ||
last_processed_token[0](*_token.children) | ||
else: | ||
last_processed_token[0](_token) | ||
else: | ||
combined_tokens.append((_token, _type)) | ||
else: | ||
combined_tokens.append((_token, _type)) | ||
|
||
if _token == "]": open_square_braces -= 1 | ||
elif _token == ")": open_parenthesis -= 1 | ||
|
||
else: | ||
# the token will be processed in _convert_type_spec_to_stan() method. | ||
combined_tokens.append((_token, _type)) | ||
|
||
return combined_tokens |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it does look unfortunate, nice to see it go
pydoctor/node2stan.py
Outdated
@@ -144,6 +154,8 @@ def depart_code(self, node: code) -> None: | |||
def should_be_compact_paragraph(self, node: nodes.Element) -> bool: | |||
if self.document.children == [node]: | |||
return True | |||
elif self._document_is_code: | |||
return True |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
huh, why no coverage here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So... A code document should not be composed by paragraphs, but if it is, we should not display <p>
tags.
I'll came up with a test case for that branch.
@@ -277,6 +311,40 @@ def test_convert_numpy_type_spec(self): | |||
for spec, expected in zip(specs, converted): | |||
actual = str(TypeDocstring(spec)) | |||
self.assertEqual(expected, actual) | |||
|
|||
def test_natural_language_delimiters_parsed_tokens(self): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
just a suggestion, whenever possible, allow new tests to be type-checked, to help provide some external validity to the type signatures?
def test_natural_language_delimiters_parsed_tokens(self): | |
def test_natural_language_delimiters_parsed_tokens(self) -> None: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see that test_napoleon_docstring.py
is one of the rare files that have disallow_untyped_defs=False
.
A follow-up issue should be created to enforce strict mode of mypy in these files as well.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the review @glyph, you rock as always
pydoctor/epydoc/markup/_types.py
Outdated
def __init__(self, annotation: nodes.document, | ||
warns_on_unknown_tokens: bool = False, | ||
lineno: int = 0) -> None: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes we need to do it, but I did not had the courage to deal with the mypy issues generated by the formatting.
#859
@@ -277,6 +311,40 @@ def test_convert_numpy_type_spec(self): | |||
for spec, expected in zip(specs, converted): | |||
actual = str(TypeDocstring(spec)) | |||
self.assertEqual(expected, actual) | |||
|
|||
def test_natural_language_delimiters_parsed_tokens(self): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see that test_napoleon_docstring.py
is one of the rare files that have disallow_untyped_defs=False
.
A follow-up issue should be created to enforce strict mode of mypy in these files as well.
pydoctor/node2stan.py
Outdated
@@ -144,6 +154,8 @@ def depart_code(self, node: code) -> None: | |||
def should_be_compact_paragraph(self, node: nodes.Element) -> bool: | |||
if self.document.children == [node]: | |||
return True | |||
elif self._document_is_code: | |||
return True |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So... A code document should not be composed by paragraphs, but if it is, we should not display <p>
tags.
I'll came up with a test case for that branch.
pydoctor/epydoc/markup/_types.py
Outdated
TypeDocstring.__init__(self, annotation, warns_on_unknown_tokens) | ||
|
||
_tokens = self._tokenize_node_type_spec(annotation) | ||
# yes this overrides the superclass type! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since the only answer to that question was “because I’m lazy” (not quite satisfactory), I couldn’t resist fixing the design so it uses composition instead of inheritance.
pydoctor/epydoc/markup/_types.py
Outdated
_tokens = self._tokenize_node_type_spec(annotation) | ||
# yes this overrides the superclass type! | ||
self._tokens: list[tuple[str | nodes.Node, TokenType]] \ | ||
= self._build_tokens(_tokens) # type: ignore |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This will be addressed
README.rst
Outdated
@@ -89,6 +89,8 @@ in development | |||
* Fix a bug in the MRO computing code that would result in an incorrect | |||
``Cannot compute linearization of the class inheritance hierarchy`` message | |||
for valid types extending ``typing.Generic`` as well as other generic classes. | |||
* Fix bug that would result in duplicated "Cannot find link target" warnings when the | |||
types under an docstring *Attributes* section failed to resolved. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
types under an docstring *Attributes* section failed to resolved. | |
types under a docstring *Attributes* section failed to resolved. |
Diff from pydoctor_primer, showing the effect of this PR on open source code: numpy (https://github.com/numpy/numpy)
- /projects/numpy/numpy/_core/_ufunc_config.py:388: bad docstring: invalid value set (missing closing brace): {divide
+ /projects/numpy/numpy/_core/_ufunc_config.py:387: bad docstring: invalid value set (missing closing brace): {divide
- /projects/numpy/numpy/_core/_ufunc_config.py:388: bad docstring: invalid value set (missing opening brace): invalid}
+ /projects/numpy/numpy/_core/_ufunc_config.py:387: bad docstring: invalid value set (missing opening brace): invalid}
- /projects/numpy/numpy/_core/fromnumeric.py:387: bad docstring: invalid value set (missing closing brace): {'raise'(
+ /projects/numpy/numpy/_core/fromnumeric.py:386: bad docstring: invalid value set (missing closing brace): {'raise'(
- /projects/numpy/numpy/_core/fromnumeric.py:387: bad docstring: invalid value set (missing opening brace): }
+ /projects/numpy/numpy/_core/fromnumeric.py:386: bad docstring: invalid value set (missing opening brace): }
- /projects/numpy/numpy/_core/fromnumeric.py:387: bad docstring: unbalanced parenthesis in type expression
+ /projects/numpy/numpy/_core/fromnumeric.py:386: bad docstring: unbalanced parenthesis in type expression
- /projects/numpy/numpy/_core/fromnumeric.py:3898: bad docstring: invalid value set (missing closing brace): {int
+ /projects/numpy/numpy/_core/fromnumeric.py:3897: bad docstring: invalid value set (missing closing brace): {int
- /projects/numpy/numpy/_core/fromnumeric.py:3898: bad docstring: invalid value set (missing opening brace): float}
+ /projects/numpy/numpy/_core/fromnumeric.py:3897: bad docstring: invalid value set (missing opening brace): float}
- /projects/numpy/numpy/_core/fromnumeric.py:3925: bad docstring: invalid value set (missing closing brace): {int
+ /projects/numpy/numpy/_core/fromnumeric.py:3924: bad docstring: invalid value set (missing closing brace): {int
- /projects/numpy/numpy/_core/fromnumeric.py:3925: bad docstring: invalid value set (missing opening brace): float}
+ /projects/numpy/numpy/_core/fromnumeric.py:3924: bad docstring: invalid value set (missing opening brace): float}
- /projects/numpy/numpy/_core/fromnumeric.py:4102: bad docstring: invalid value set (missing closing brace): {int
+ /projects/numpy/numpy/_core/fromnumeric.py:4101: bad docstring: invalid value set (missing closing brace): {int
- /projects/numpy/numpy/_core/fromnumeric.py:4102: bad docstring: invalid value set (missing opening brace): float}
+ /projects/numpy/numpy/_core/fromnumeric.py:4101: bad docstring: invalid value set (missing opening brace): float}
- /projects/numpy/numpy/_core/fromnumeric.py:4129: bad docstring: invalid value set (missing closing brace): {int
+ /projects/numpy/numpy/_core/fromnumeric.py:4128: bad docstring: invalid value set (missing closing brace): {int
- /projects/numpy/numpy/_core/fromnumeric.py:4129: bad docstring: invalid value set (missing opening brace): float}
+ /projects/numpy/numpy/_core/fromnumeric.py:4128: bad docstring: invalid value set (missing opening brace): float}
- /projects/numpy/numpy/_core/einsumfunc.py:775: bad docstring: invalid value set (missing closing brace): {bool
+ /projects/numpy/numpy/_core/einsumfunc.py:774: bad docstring: invalid value set (missing closing brace): {bool
- /projects/numpy/numpy/_core/einsumfunc.py:775: bad docstring: invalid value set (missing opening brace): }
+ /projects/numpy/numpy/_core/einsumfunc.py:774: bad docstring: invalid value set (missing opening brace): }
- /projects/numpy/numpy/_core/einsumfunc.py:1089: bad docstring: invalid value set (missing closing brace): {data-type
+ /projects/numpy/numpy/_core/einsumfunc.py:1088: bad docstring: invalid value set (missing closing brace): {data-type
- /projects/numpy/numpy/_core/einsumfunc.py:1089: bad docstring: invalid value set (missing opening brace): None}
+ /projects/numpy/numpy/_core/einsumfunc.py:1088: bad docstring: invalid value set (missing opening brace): None}
- /projects/numpy/numpy/_core/einsumfunc.py:1113: bad docstring: invalid value set (missing closing brace): {False
+ /projects/numpy/numpy/_core/einsumfunc.py:1112: bad docstring: invalid value set (missing closing brace): {False
- /projects/numpy/numpy/_core/einsumfunc.py:1113: bad docstring: invalid value set (missing opening brace): }
+ /projects/numpy/numpy/_core/einsumfunc.py:1112: bad docstring: invalid value set (missing opening brace): }
- /projects/numpy/numpy/linalg/_linalg.py:399: bad docstring: invalid value set (missing closing brace): {(..., M, ), (..., M, K)} ndarray
+ /projects/numpy/numpy/linalg/_linalg.py:398: bad docstring: invalid value set (missing closing brace): {(..., M, ), (..., M, K)} ndarray
- /projects/numpy/numpy/linalg/_linalg.py:1964: bad docstring: invalid value set (missing closing brace): {None
+ /projects/numpy/numpy/linalg/_linalg.py:1963: bad docstring: invalid value set (missing closing brace): {None
- /projects/numpy/numpy/linalg/_linalg.py:1964: bad docstring: invalid value set (missing opening brace): }
+ /projects/numpy/numpy/linalg/_linalg.py:1963: bad docstring: invalid value set (missing opening brace): }
- /projects/numpy/numpy/linalg/_linalg.py:1967: bad docstring: invalid value set (missing closing brace): {float
+ /projects/numpy/numpy/linalg/_linalg.py:1966: bad docstring: invalid value set (missing closing brace): {float
- /projects/numpy/numpy/linalg/_linalg.py:1967: bad docstring: invalid value set (missing opening brace): inf}
+ /projects/numpy/numpy/linalg/_linalg.py:1966: bad docstring: invalid value set (missing opening brace): inf}
- /projects/numpy/numpy/linalg/_linalg.py:2069: bad docstring: invalid value set (missing closing brace): {(M, ), (..., M, N)} array_like
+ /projects/numpy/numpy/linalg/_linalg.py:2068: bad docstring: invalid value set (missing closing brace): {(M, ), (..., M, N)} array_like
- /projects/numpy/numpy/linalg/_linalg.py:2461: bad docstring: invalid value set (missing closing brace): {(M, ), (M, K)} array_like
+ /projects/numpy/numpy/linalg/_linalg.py:2460: bad docstring: invalid value set (missing closing brace): {(M, ), (M, K)} array_like
- /projects/numpy/numpy/linalg/_linalg.py:2639: bad docstring: invalid value set (missing closing brace): {int
+ /projects/numpy/numpy/linalg/_linalg.py:2638: bad docstring: invalid value set (missing closing brace): {int
- /projects/numpy/numpy/linalg/_linalg.py:2639: bad docstring: invalid value set (missing opening brace): }
+ /projects/numpy/numpy/linalg/_linalg.py:2638: bad docstring: invalid value set (missing opening brace): }
- /projects/numpy/numpy/linalg/_linalg.py:2646: bad docstring: invalid value set (missing closing brace): {None
+ /projects/numpy/numpy/linalg/_linalg.py:2645: bad docstring: invalid value set (missing closing brace): {None
- /projects/numpy/numpy/linalg/_linalg.py:2646: bad docstring: invalid value set (missing opening brace): ints}
+ /projects/numpy/numpy/linalg/_linalg.py:2645: bad docstring: invalid value set (missing opening brace): ints}
- /projects/numpy/numpy/linalg/_linalg.py:3486: bad docstring: invalid value set (missing closing brace): {1
+ /projects/numpy/numpy/linalg/_linalg.py:3485: bad docstring: invalid value set (missing closing brace): {1
- /projects/numpy/numpy/linalg/_linalg.py:3486: bad docstring: invalid value set (missing opening brace): }
+ /projects/numpy/numpy/linalg/_linalg.py:3485: bad docstring: invalid value set (missing opening brace): }
- /projects/numpy/numpy/linalg/_linalg.py:3549: bad docstring: invalid value set (missing closing brace): {None
+ /projects/numpy/numpy/linalg/_linalg.py:3548: bad docstring: invalid value set (missing closing brace): {None
- /projects/numpy/numpy/linalg/_linalg.py:3549: bad docstring: invalid value set (missing opening brace): ints}
+ /projects/numpy/numpy/linalg/_linalg.py:3548: bad docstring: invalid value set (missing opening brace): ints}
- /projects/numpy/numpy/linalg/_linalg.py:3555: bad docstring: invalid value set (missing closing brace): {int
+ /projects/numpy/numpy/linalg/_linalg.py:3554: bad docstring: invalid value set (missing closing brace): {int
- /projects/numpy/numpy/linalg/_linalg.py:3555: bad docstring: invalid value set (missing opening brace): -inf}
+ /projects/numpy/numpy/linalg/_linalg.py:3554: bad docstring: invalid value set (missing opening brace): -inf}
- /projects/numpy/numpy/_core/function_base.py:507: bad docstring: invalid value set (missing closing brace): {str
+ /projects/numpy/numpy/_core/function_base.py:506: bad docstring: invalid value set (missing closing brace): {str
- /projects/numpy/numpy/_core/function_base.py:507: bad docstring: invalid value set (missing opening brace): }
+ /projects/numpy/numpy/_core/function_base.py:506: bad docstring: invalid value set (missing opening brace): }
- /projects/numpy/numpy/lib/_function_base_impl.py:1024: bad docstring: invalid value set (missing closing brace): {1
+ /projects/numpy/numpy/lib/_function_base_impl.py:1023: bad docstring: invalid value set (missing closing brace): {1
- /projects/numpy/numpy/lib/_function_base_impl.py:1024: bad docstring: invalid value set (missing opening brace): 2}
+ /projects/numpy/numpy/lib/_function_base_impl.py:1023: bad docstring: invalid value set (missing opening brace): 2}
- /projects/numpy/numpy/lib/_function_base_impl.py:3949: bad docstring: invalid value set (missing closing brace): {int
+ /projects/numpy/numpy/lib/_function_base_impl.py:3948: bad docstring: invalid value set (missing closing brace): {int
- /projects/numpy/numpy/lib/_function_base_impl.py:3949: bad docstring: invalid value set (missing opening brace): None}
+ /projects/numpy/numpy/lib/_function_base_impl.py:3948: bad docstring: invalid value set (missing opening brace): None}
- /projects/numpy/numpy/lib/_function_base_impl.py:4110: bad docstring: invalid value set (missing closing brace): {int
+ /projects/numpy/numpy/lib/_function_base_impl.py:4109: bad docstring: invalid value set (missing closing brace): {int
... (truncated 503 lines) ...
|
Diff from pydoctor_primer, showing the effect of this PR on open source code: numpy (https://github.com/numpy/numpy)
- /projects/numpy/numpy/_core/_ufunc_config.py:388: bad docstring: invalid value set (missing closing brace): {divide
+ /projects/numpy/numpy/_core/_ufunc_config.py:387: bad docstring: invalid value set (missing closing brace): {divide
- /projects/numpy/numpy/_core/_ufunc_config.py:388: bad docstring: invalid value set (missing opening brace): invalid}
+ /projects/numpy/numpy/_core/_ufunc_config.py:387: bad docstring: invalid value set (missing opening brace): invalid}
- /projects/numpy/numpy/_core/fromnumeric.py:387: bad docstring: invalid value set (missing closing brace): {'raise'(
+ /projects/numpy/numpy/_core/fromnumeric.py:386: bad docstring: invalid value set (missing closing brace): {'raise'(
- /projects/numpy/numpy/_core/fromnumeric.py:387: bad docstring: invalid value set (missing opening brace): }
+ /projects/numpy/numpy/_core/fromnumeric.py:386: bad docstring: invalid value set (missing opening brace): }
- /projects/numpy/numpy/_core/fromnumeric.py:387: bad docstring: unbalanced parenthesis in type expression
+ /projects/numpy/numpy/_core/fromnumeric.py:386: bad docstring: unbalanced parenthesis in type expression
- /projects/numpy/numpy/_core/fromnumeric.py:3898: bad docstring: invalid value set (missing closing brace): {int
+ /projects/numpy/numpy/_core/fromnumeric.py:3897: bad docstring: invalid value set (missing closing brace): {int
- /projects/numpy/numpy/_core/fromnumeric.py:3898: bad docstring: invalid value set (missing opening brace): float}
+ /projects/numpy/numpy/_core/fromnumeric.py:3897: bad docstring: invalid value set (missing opening brace): float}
- /projects/numpy/numpy/_core/fromnumeric.py:3925: bad docstring: invalid value set (missing closing brace): {int
+ /projects/numpy/numpy/_core/fromnumeric.py:3924: bad docstring: invalid value set (missing closing brace): {int
- /projects/numpy/numpy/_core/fromnumeric.py:3925: bad docstring: invalid value set (missing opening brace): float}
+ /projects/numpy/numpy/_core/fromnumeric.py:3924: bad docstring: invalid value set (missing opening brace): float}
- /projects/numpy/numpy/_core/fromnumeric.py:4102: bad docstring: invalid value set (missing closing brace): {int
+ /projects/numpy/numpy/_core/fromnumeric.py:4101: bad docstring: invalid value set (missing closing brace): {int
- /projects/numpy/numpy/_core/fromnumeric.py:4102: bad docstring: invalid value set (missing opening brace): float}
+ /projects/numpy/numpy/_core/fromnumeric.py:4101: bad docstring: invalid value set (missing opening brace): float}
- /projects/numpy/numpy/_core/fromnumeric.py:4129: bad docstring: invalid value set (missing closing brace): {int
+ /projects/numpy/numpy/_core/fromnumeric.py:4128: bad docstring: invalid value set (missing closing brace): {int
- /projects/numpy/numpy/_core/fromnumeric.py:4129: bad docstring: invalid value set (missing opening brace): float}
+ /projects/numpy/numpy/_core/fromnumeric.py:4128: bad docstring: invalid value set (missing opening brace): float}
- /projects/numpy/numpy/_core/einsumfunc.py:775: bad docstring: invalid value set (missing closing brace): {bool
+ /projects/numpy/numpy/_core/einsumfunc.py:774: bad docstring: invalid value set (missing closing brace): {bool
- /projects/numpy/numpy/_core/einsumfunc.py:775: bad docstring: invalid value set (missing opening brace): }
+ /projects/numpy/numpy/_core/einsumfunc.py:774: bad docstring: invalid value set (missing opening brace): }
- /projects/numpy/numpy/_core/einsumfunc.py:1089: bad docstring: invalid value set (missing closing brace): {data-type
+ /projects/numpy/numpy/_core/einsumfunc.py:1088: bad docstring: invalid value set (missing closing brace): {data-type
- /projects/numpy/numpy/_core/einsumfunc.py:1089: bad docstring: invalid value set (missing opening brace): None}
+ /projects/numpy/numpy/_core/einsumfunc.py:1088: bad docstring: invalid value set (missing opening brace): None}
- /projects/numpy/numpy/_core/einsumfunc.py:1113: bad docstring: invalid value set (missing closing brace): {False
+ /projects/numpy/numpy/_core/einsumfunc.py:1112: bad docstring: invalid value set (missing closing brace): {False
- /projects/numpy/numpy/_core/einsumfunc.py:1113: bad docstring: invalid value set (missing opening brace): }
+ /projects/numpy/numpy/_core/einsumfunc.py:1112: bad docstring: invalid value set (missing opening brace): }
- /projects/numpy/numpy/linalg/_linalg.py:399: bad docstring: invalid value set (missing closing brace): {(..., M, ), (..., M, K)} ndarray
+ /projects/numpy/numpy/linalg/_linalg.py:398: bad docstring: invalid value set (missing closing brace): {(..., M, ), (..., M, K)} ndarray
- /projects/numpy/numpy/linalg/_linalg.py:1964: bad docstring: invalid value set (missing closing brace): {None
+ /projects/numpy/numpy/linalg/_linalg.py:1963: bad docstring: invalid value set (missing closing brace): {None
- /projects/numpy/numpy/linalg/_linalg.py:1964: bad docstring: invalid value set (missing opening brace): }
+ /projects/numpy/numpy/linalg/_linalg.py:1963: bad docstring: invalid value set (missing opening brace): }
- /projects/numpy/numpy/linalg/_linalg.py:1967: bad docstring: invalid value set (missing closing brace): {float
+ /projects/numpy/numpy/linalg/_linalg.py:1966: bad docstring: invalid value set (missing closing brace): {float
- /projects/numpy/numpy/linalg/_linalg.py:1967: bad docstring: invalid value set (missing opening brace): inf}
+ /projects/numpy/numpy/linalg/_linalg.py:1966: bad docstring: invalid value set (missing opening brace): inf}
- /projects/numpy/numpy/linalg/_linalg.py:2069: bad docstring: invalid value set (missing closing brace): {(M, ), (..., M, N)} array_like
+ /projects/numpy/numpy/linalg/_linalg.py:2068: bad docstring: invalid value set (missing closing brace): {(M, ), (..., M, N)} array_like
- /projects/numpy/numpy/linalg/_linalg.py:2461: bad docstring: invalid value set (missing closing brace): {(M, ), (M, K)} array_like
+ /projects/numpy/numpy/linalg/_linalg.py:2460: bad docstring: invalid value set (missing closing brace): {(M, ), (M, K)} array_like
- /projects/numpy/numpy/linalg/_linalg.py:2639: bad docstring: invalid value set (missing closing brace): {int
+ /projects/numpy/numpy/linalg/_linalg.py:2638: bad docstring: invalid value set (missing closing brace): {int
- /projects/numpy/numpy/linalg/_linalg.py:2639: bad docstring: invalid value set (missing opening brace): }
+ /projects/numpy/numpy/linalg/_linalg.py:2638: bad docstring: invalid value set (missing opening brace): }
- /projects/numpy/numpy/linalg/_linalg.py:2646: bad docstring: invalid value set (missing closing brace): {None
+ /projects/numpy/numpy/linalg/_linalg.py:2645: bad docstring: invalid value set (missing closing brace): {None
- /projects/numpy/numpy/linalg/_linalg.py:2646: bad docstring: invalid value set (missing opening brace): ints}
+ /projects/numpy/numpy/linalg/_linalg.py:2645: bad docstring: invalid value set (missing opening brace): ints}
- /projects/numpy/numpy/linalg/_linalg.py:3486: bad docstring: invalid value set (missing closing brace): {1
+ /projects/numpy/numpy/linalg/_linalg.py:3485: bad docstring: invalid value set (missing closing brace): {1
- /projects/numpy/numpy/linalg/_linalg.py:3486: bad docstring: invalid value set (missing opening brace): }
+ /projects/numpy/numpy/linalg/_linalg.py:3485: bad docstring: invalid value set (missing opening brace): }
- /projects/numpy/numpy/linalg/_linalg.py:3549: bad docstring: invalid value set (missing closing brace): {None
+ /projects/numpy/numpy/linalg/_linalg.py:3548: bad docstring: invalid value set (missing closing brace): {None
- /projects/numpy/numpy/linalg/_linalg.py:3549: bad docstring: invalid value set (missing opening brace): ints}
+ /projects/numpy/numpy/linalg/_linalg.py:3548: bad docstring: invalid value set (missing opening brace): ints}
- /projects/numpy/numpy/linalg/_linalg.py:3555: bad docstring: invalid value set (missing closing brace): {int
+ /projects/numpy/numpy/linalg/_linalg.py:3554: bad docstring: invalid value set (missing closing brace): {int
- /projects/numpy/numpy/linalg/_linalg.py:3555: bad docstring: invalid value set (missing opening brace): -inf}
+ /projects/numpy/numpy/linalg/_linalg.py:3554: bad docstring: invalid value set (missing opening brace): -inf}
- /projects/numpy/numpy/_core/function_base.py:507: bad docstring: invalid value set (missing closing brace): {str
+ /projects/numpy/numpy/_core/function_base.py:506: bad docstring: invalid value set (missing closing brace): {str
- /projects/numpy/numpy/_core/function_base.py:507: bad docstring: invalid value set (missing opening brace): }
+ /projects/numpy/numpy/_core/function_base.py:506: bad docstring: invalid value set (missing opening brace): }
- /projects/numpy/numpy/lib/_function_base_impl.py:1024: bad docstring: invalid value set (missing closing brace): {1
+ /projects/numpy/numpy/lib/_function_base_impl.py:1023: bad docstring: invalid value set (missing closing brace): {1
- /projects/numpy/numpy/lib/_function_base_impl.py:1024: bad docstring: invalid value set (missing opening brace): 2}
+ /projects/numpy/numpy/lib/_function_base_impl.py:1023: bad docstring: invalid value set (missing opening brace): 2}
- /projects/numpy/numpy/lib/_function_base_impl.py:3949: bad docstring: invalid value set (missing closing brace): {int
+ /projects/numpy/numpy/lib/_function_base_impl.py:3948: bad docstring: invalid value set (missing closing brace): {int
- /projects/numpy/numpy/lib/_function_base_impl.py:3949: bad docstring: invalid value set (missing opening brace): None}
+ /projects/numpy/numpy/lib/_function_base_impl.py:3948: bad docstring: invalid value set (missing opening brace): None}
- /projects/numpy/numpy/lib/_function_base_impl.py:4110: bad docstring: invalid value set (missing closing brace): {int
+ /projects/numpy/numpy/lib/_function_base_impl.py:4109: bad docstring: invalid value set (missing closing brace): {int
... (truncated 503 lines) ...
|
Refactor the the
ParsedTypeDocstring
- used in numpy and google docformat as well as when --process-types option is passed.This gets rid of some unfortunate code that was not doing it's thing right. We replace that dirty processing by legitimately creating a docutils tree for these types are rendering correctly. In order to do that, the linker had to be adjusted not to wrap links inside
<code>
tags anymore (see the preliminary refactor commit ec6c907).Make possible for PR #723 to be complete
Fixes #581
Fixes #873
Before:

After:

The warning diff was unexpected... it appears that pydoctor was reporting link not found at several unrelated places, increasing the number of warnings uselessly.