This repository was archived by the owner on Mar 3, 2023. It is now read-only.
Cache node_modules on Azure Pipelines #19437
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
775f07e to
0baaa4f
Compare
Co-Authored-By: Nathan Sobo <nathan@github.com> Co-Authored-By: Rafael Oleza <rafeca@github.com>
This was referenced Jul 9, 2020
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Refs: #19366
This pull request's aim is to speed up our CI build times by taking advantage of https://github.com/microsoft/azure-pipelines-artifact-caching-tasks on this repository.
Caches are looked up by a key based on the contents of all
package-lock.jsonfiles in the repository. When (at least) one of them changes, the cache won't be used and dependencies will be re-bootstrapped from scratch. We could probably find ways of starting with a base set of dependencies (e.g., the ones found onmaster) and only perform a partial bootstrap, but @rafeca pointed out that this could lead to some non-determinism in the package installed bynpm. Maybe this is one argument in favor of switching toyarnat some point.For now, this pull request picks the low-hanging fruit of skipping
script/bootstrapentirely if the lockfile perfectly matches the set of dependencies that we have in the cache. When that happens, it saves ~8-9 minutes of build time.Next Steps
In the future, we may want to investigate a couple of different options:
yarn. Doing so should speedscript/bootstrapup and, most importantly, it could remove the aforementioned non-determinism, which would allow for more aggressive caching.script/buildtoo. That may still require using a fork of https://github.com/microsoft/azure-pipelines-artifact-caching-tasks, but we're spending a lot of time doing redundant work, such as generating the snapshot or transpiling files.