fix: verified svm builds #997
Conversation
Signed-off-by: Reinis Martinsons <reinis@umaproject.org>
Signed-off-by: Reinis Martinsons <reinis@umaproject.org>
Signed-off-by: Reinis Martinsons <reinis@umaproject.org>
Signed-off-by: Reinis Martinsons <reinis@umaproject.org>
Signed-off-by: Reinis Martinsons <reinis@umaproject.org>
Signed-off-by: Reinis Martinsons <reinis@umaproject.org>
Signed-off-by: Reinis Martinsons <reinis@umaproject.org>
Signed-off-by: Reinis Martinsons <reinis@umaproject.org>
Signed-off-by: Reinis Martinsons <reinis@umaproject.org>
Signed-off-by: Reinis Martinsons <reinis@umaproject.org>
Signed-off-by: Reinis Martinsons <reinis@umaproject.org>
Signed-off-by: Reinis Martinsons <reinis@umaproject.org>
Signed-off-by: Reinis Martinsons <reinis@umaproject.org>
Signed-off-by: Reinis Martinsons <reinis@umaproject.org>
Signed-off-by: Reinis Martinsons <reinis@umaproject.org>
Signed-off-by: Reinis Martinsons <reinis@umaproject.org>
Signed-off-by: Reinis Martinsons <reinis@umaproject.org>
Signed-off-by: Reinis Martinsons <reinis@umaproject.org>
Signed-off-by: Reinis Martinsons <reinis@umaproject.org>
| cache: yarn | ||
| - name: Install packages | ||
| shell: bash | ||
| run: yarn install --frozen-lockfile |
There was a problem hiding this comment.
This still have below errors here when building optional node-hid and cpu-features. We cannot do --ignore-optional here like in other places as hardhat compile depends on other optional dependencies being installed.
warning Error running install script for optional dependency: "/home/runner/work/contracts/contracts/node_modules/node-hid: Command failed.
info This module is OPTIONAL, you can safely ignore this error
warning Error running install script for optional dependency: "/home/runner/work/contracts/contracts/node_modules/cpu-features: Command failed.
info This module is OPTIONAL, you can safely ignore this error
| env: | ||
| NODE_VERSION: 20 |
There was a problem hiding this comment.
Before we had matrix strategy, but that got overly repetitive when splitting to multiple jobs. And we were not using the matrix functionality anyways as all jobs were run against a single node version.
| - name: Extract Solana versions | ||
| uses: solana-developers/github-actions/extract-versions@v0.2.5 | ||
| id: versions | ||
| - name: Setup Anchor & Solana | ||
| uses: solana-developers/github-actions/setup-all@v0.2.5 | ||
| with: | ||
| anchor_version: ${{ steps.versions.outputs.anchor_version }} | ||
| solana_version: ${{ steps.versions.outputs.solana_version }} | ||
| node_version: 20 |
There was a problem hiding this comment.
solana and anchor binaries were not needed for lint
| # Perform installs, run tests, run a build step, etc. here, as needed. | ||
| - run: yarn | ||
| # Perform installs and run a build step. | ||
| - run: yarn install --frozen-lockfile && yarn build |
There was a problem hiding this comment.
With removed prepublish script we have to build explicitly. For now this includes regular SVM build without caching, but would need to create a follow-up PR that covers verified builds.
| "build-ts": "rm -rf ./dist && tsc && rsync -a --include '*/' --include '*.d.ts' --exclude '*' ./typechain ./dist/", | ||
| "copy-idl": "mkdir -p dist/src/svm/assets/idl && cp src/svm/assets/idl/*.json dist/src/svm/assets/idl/", | ||
| "build": "yarn build-evm && yarn build-svm && yarn generate-svm-assets && yarn build-ts && yarn copy-idl", | ||
| "build-svm": "bash ./scripts/svm/buildHelpers/buildSvmLocalToolchain.sh", |
There was a problem hiding this comment.
this was unnecessarily duplicating anchor build.
| "build-ts": "rm -rf ./dist && tsc && rsync -a --include '*/' --include '*.d.ts' --exclude '*' ./typechain ./dist/ && yarn export-idl", | ||
| "export-idl": "mkdir -p dist/src/svm/assets/idl && cp src/svm/assets/idl/*.json dist/src/svm/assets/idl/", |
There was a problem hiding this comment.
moved export-idl (renamed from copy-idl before) from build to build-ts as it is more relevant for dist files.
| "test:report-gas": "IS_TEST=true REPORT_GAS=true hardhat test", | ||
| "generate-evm-assets": "rm -rf typechain && TYPECHAIN=ethers yarn hardhat typechain", | ||
| "prepublish": "yarn build && hardhat export --export-all ./cache/massExport.json && ts-node ./scripts/processHardhatExport.ts && prettier --write ./deployments/deployments.json && yarn generate-evm-assets", |
There was a problem hiding this comment.
Had to delete the prepublish script as it got triggered on all yarn install commands making it impossible to split SVM/EVM builds in parallel CI jobs.
| "generate-evm-assets": "rm -rf typechain && TYPECHAIN=ethers yarn hardhat typechain", | ||
| "prepublish": "yarn build && hardhat export --export-all ./cache/massExport.json && ts-node ./scripts/processHardhatExport.ts && prettier --write ./deployments/deployments.json && yarn generate-evm-assets", | ||
| "generate-evm-artifacts": "rm -rf typechain && TYPECHAIN=ethers yarn hardhat typechain", | ||
| "process-hardhat-export": "hardhat export --export-all ./cache/massExport.json && ts-node ./scripts/processHardhatExport.ts && prettier --write ./deployments/deployments.json", |
There was a problem hiding this comment.
We had deployments generation as part of prepublish script, but these commands were ran after yarn build which already had populated the dist files, so this could have potentially left local tree inconsistent with dist. Hence, these steps are moved to a dedicated script if anyone still needs them. Though I understand in practice we update deployments.json by hand.
Also removed typechain generation completely as they are already generated in hardhat compile with @typechain/hardhat plugin.
| runs: | ||
| using: "composite" | ||
| steps: | ||
| - name: Extract Solana versions |
There was a problem hiding this comment.
Nit: think we could reuse this in the publish workflow too?
There was a problem hiding this comment.
yes, indeed - I planned handling publish workflow in a follow-up
Signed-off-by: Reinis Martinsons <reinis@umaproject.org>
| "test-evm": "IS_TEST=true hardhat test", | ||
| "test-svm": "anchor test -- --features test", | ||
| "test-svm": "IS_TEST=true yarn build-svm && yarn generate-svm-artifacts && anchor test --skip-build", | ||
| "test-svm-solana-verify": "IS_TEST=true yarn build-svm-solana-verify && yarn generate-svm-artifacts && anchor test --skip-build", |
There was a problem hiding this comment.
In CI we invoke anchor test --skip-build directly in order to reuse build that was potentially grabbed from cache, but test-svm and test-svm-solana-verify scripts are only meant for running locally
Signed-off-by: Reinis Martinsons <reinis@umaproject.org>
Signed-off-by: Reinis Martinsons <reinis@umaproject.org>
Signed-off-by: Reinis Martinsons <reinis@umaproject.org>
Signed-off-by: Reinis Martinsons <reinis@umaproject.org>
This introduces testing against verifyable SVM builds in CI. Since this is rather slow, this breaks build in parallel EVM/SVM jobs and uses caching where possible so that not to delay development unnecessarily for changes that don't affect program code.