8000 add a non-mutating incremental resolver interface by froydnj · Pull Request #5456 · sorbet/sorbet · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

add a non-mutating incremental resolver interface #5456

8000 New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 14 commits into from
Mar 16, 2022

Conversation

froydnj
Copy link
Contributor
@froydnj froydnj commented Mar 10, 2022

Motivation

When we're running queries on stale GlobalState (#5407), we may need to run resolver in a way that modifies the affected trees we're looking at, but deliberately does not modify the GlobalState we're using. (We expect that either a) the new GlobalState resulting from a successful ongoing typecheck will replace it or b) mutating it for temporary changes, but then having to put it back without those changes if the ongoing typecheck is cancelled for some reason, is fraught with problems.)

This PR attempts to provide such an interface with a minimum of code duplication, leaning on templates and if constexpr to magically wall off mutating sections of the resolver. This way the compiler will automatically flag misuses of const GlobalState in the templated function(s) and it should be relatively obvious how to proceed to satisfy the compiler.

What's not clear to me is how effective this version of the resolver is going to be, since the WIP being filed here shuts off quite a bit of the resolver, including things like resolving superclasses, etc. But maybe the expectation is that the mutating incremental resolver wouldn't have found that many changes to make anyway...?

Test plan

Not sure how to test this. Honestly this PR was just going to put the infrastructure in place; future PRs that actually call the non-mutating incremental resolver would need to figure out some sort of testing strategy.

@froydnj froydnj force-pushed the froydnj-non-mutating-resolver branch from 8803804 to ba27690 Compare March 15, 2022 15:20
@froydnj froydnj marked this pull request as ready for review March 15, 2022 15:44
@froydnj froydnj requested a review from a team as a code owner March 15, 2022 15:44
@froydnj froydnj requested review from elliottt and removed request for a team March 15, 2022 15:44
@@ -3608,14 +3576,71 @@ class ResolveSignaturesWalk {
}
fillInInfoFromSig(ctx, overloadSym, sig.loc, sig.sig, isOverloaded, mdef);
}
handleAbstractMethod(ctx, mdef);
// handleAbstractMethod called elsewhere
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not 100% happy about this part, but handleAbstractMethod seemed important for making sure the bodies of abstract methods were empty, so calling it from resolveSigs seemed like a good option.

I initially wanted to go a little farther here, but calling recordMethodInfoInSig from a const context (hence the movement of the function in one of the commits here). But actually doing so seemed complicated in the face of multiple signatures...and it turns out that the only thing that recordMethodInfoInSig does is really only important for compiled code, which we don't so much care about in LSP mode. So I left that part as-is.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like moving handleAbstractMethod to resolveSigs 👍

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ooc, why leave these comments behind? (we don't usually leave behind breadcrumbs documenting how sorbet has evolved over time)

Copy link
Collaborator
@elliottt elliottt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great! I really like the use of the template parameter to test const-ness of the state parameter 👍

Comment on lines +2906 to +2907
template <typename StateType>
static vector<ast::ParsedFile> run(StateType &gs, vector<ast::ParsedFile> trees, WorkerPool &workers) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a great idea!

@@ -3608,14 +3576,71 @@ class ResolveSignaturesWalk {
}
fillInInfoFromSig(ctx, overloadSym, sig.loc, sig.sig, isOverloaded, mdef);
}
handleAbstractMethod(ctx, mdef);
// handleAbstractMethod called elsewhere
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like moving handleAbstractMethod to resolveSigs 👍

@froydnj froydnj requested a review from elliottt March 15, 2022 20:40
@froydnj
Copy link
Contributor Author
froydnj commented Mar 15, 2022

I added a call to this in pipeline tests before we invoke the incremental resolver, and amazingly everything passes locally.

ptal @elliottt for the new test.

Copy link
Collaborator
@elliottt elliottt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the changes to the test runner make sense, though it looks like there's exactly one test failure for test/testdata/resolver/infinite.rb.

Comment on lines +737 to +739
move(resolver::Resolver::runIncrementalWithoutStateMutation(*gs, move(treesCopy)).result());
errorQueue->flushAllErrors(*gs);
errorCollector->drainErrors();
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We used to have ENFORCE(!gs->hadCriticalError()); here in an earlier commit.

However, this ENFORCE would fail on test/testdata/resolver/infinite.rb. This test is testing recursive dealiasing of constants and deliberately sets out to make Sorbet hit its internal limit of recursive dealiasing. As a result, we hit:

sorbet/core/Symbols.cc

Lines 1935 to 1939 in ebe537d

if (auto e = gs.beginError(symbol.loc(gs), errors::Internal::CyclicReferenceError)) {
e.setHeader("Too many alias expansions for symbol {}, the alias is either too long or infinite. Next "
"expansion would have been to {}",
symbol.showFullName(gs), alias.symbol.showFullName(gs));
}

and this error is defined as StrictLevel::Internal:

constexpr ErrorClass CyclicReferenceError{1003, StrictLevel::Internal};

such errors are considered "critical" by Error and wind up setting a flag on the error queue.

I cribbed this ENFORCE from jez's analogous testing for non-mutating namer, and naively considered that it would enforce that we hadn't ENFORCE'd or somesuch. In retrospect, this was a silly thing for me to assume: ENFORCEs should just fire without respect to the errors raised on GlobalState (they do not interact with Sorbet's error reporting mechanism), and a generic test runner shouldn't assume anything about whether a particular testcase does or does not trigger critical errors. (The one test we have checking hadCriticalError uses a testcase specifically designed to trigger such an error.)

I think "critical" is the wrong terminology to use for this, but that is a separate issue.

Long story short, I have deleted the ENFORCE(!gs->hadCriticalError());, partly in an effort to make tests pass, but partly because I think it really is the right thing to do. Ideally folks will agree with the rationale above. If not, we can discuss it later.

@froydnj froydnj enabled auto-merge (squash) March 16, 2022 14:04
@froydnj froydnj merged commit 390a4c4 into master Mar 16, 2022
@froydnj froydnj deleted the froydnj-non-mutating-resolver branch March 16, 2022 14:11
jez added a commit that referenced this pull request Mar 30, 2022
Mostly uses the technique that froydnj established in #5456.
jez added a commit that referenced this pull request Mar 31, 2022
* Move computeLinearization to GlobalState

* pre-work: Use lambda for better clang-format

* computeLinearization in GlobalState::initEmpty

This should be safe--if we ever call addMixin after this, we will reset
the linearization status on that class.

This allows us to call `verifyLinearizationComputed` in
`runIncrementalBestEffort` (because our test suite tests an edge case
where we run `runIncrementalBestEffort` with `--no-stdlib` so Resolver
truly has never been run before).

* Non-mutating version of `resolveConstants`

Mostly uses the technique that froydnj established in #5456.

* Rest of the failures

* Also run infer
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants
0