8000 [manta] CI runtime upgrade test and github templates improvements by ghzlatarev · Pull Request #393 · Manta-Network/Manta · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

[manta] CI runtime upgrade test and github templates improvements #393

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 14 commits into from
Mar 9, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 10 additions & 5 deletions .github/ISSUE_TEMPLATE/release.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,7 @@ of the Calamari/Manta runtime or client. Some need to be completed after the new

### Runtime Releases

These checks should be performed on the codebase prior to forking to a release-
candidate branch.
These checks should be performed on the codebase prior to freezing our release candidate:

- [ ] Verify [`spec_version`](#spec-version) has been incremented since the
last release for any native runtimes from any existing use on public
Expand All @@ -24,18 +23,24 @@ candidate branch.
runtime logic.
- [ ] Check for any upstream storage migrations and perform tests with `try-runtime`, if any.

The following checks can be performed after we have forked off to the release branch.
The following checks can be performed after we have frozen our release candidate:

- [ ] Code freeze should typically happen one week prior to release, to ensure we have enough time for related testing.
- [ ] Notify everyone, especially people with merge rights to `manta` (stechu, Dengjianping) that a release is ongoing and no more merges to `manta` should happen until told otherwise
- [ ] Complete the following [manual QA workflow](https://hackmd.io/TbFmorG2RnOPmLuFcg9JOQ?view).
- [ ] Verify [Polkadot JS API](#polkadot-js) are up to date with the latest
runtime changes.
- [ ] Execute runtime upgrade to Como and verify network stability.
- [ ] Execute runtime upgrade to Baikal and verify network stability.
- [ ] Prepare a governance post and submit to our forum with description and motivation for changes.

### Client Releases

- [ ] Verify that each crate's `version` has been bumped from previous release.
- [ ] Check that the new client versions have [run on the network](#burn-in) without issue for 12 hours.

### All Releases

- [ ] Check that the new client versions have [run on the network](#burn-in)
without issue for 12 hours.
- [ ] Check that a draft release has been created at
https://github.com/Manta-Network/Manta/releases with relevant [release
notes](#release-notes)
Expand Down
6 changes: 1 addition & 5 deletions .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,6 @@ Before we can merge this PR, please make sure that all the following items have
checked off. If any of the checklist items are not applicable, please leave them but
write a little note why.

- [ ] Targeted PR against correct branch (`manta` or `dolphin`) with right title (start with [Manta] or [Dolphin]),
- [ ] Linked to Github issue with discussion and accepted design OR have an explanation in the PR that describes this work.
- [ ] Wrote unit tests.
- [ ] Updated relevant documentation in the code.
Expand All @@ -29,10 +28,7 @@ write a little note why.
* `spec_version`: The version of the runtime specification. A full node will not attempt to use its native runtime in substitute for the on-chain Wasm runtime unless all of spec_name, spec_version, and authoring_version are the same between Wasm and native.
* `impl_version`: The version of the implementation of the specification. Nodes are free to ignore this; it serves only as an indication that the code is different; as long as the other two versions are the same then while the actual code may be different, it is nonetheless required to do the same thing. Non-consensus-breaking optimizations are about the only changes that could be made which would result in only the impl_version changing.
* `transaction_version`: The version of the extrinsics interface. This number must be updated in the following circumstances: extrinsic parameters (number, order, or types) have been changed; extrinsics or pallets have been removed; or the pallet order in the construct_runtime! macro or extrinsic order in a pallet has been changed. If this number is updated, then the spec_version must also be updated
- [ ] If needed, notify the committer this is a draft-release and a tag is needed after merging the PR.
- [ ] Verify benchmarks & weights have been updated for any modified runtime logics
- [ ] If needed, bump `version` for every crate.
- [ ] If import a new pallet, choose a proper module index for it, and allow it in `BaseFilter`. Ensure **every** extrinsic works from front-end. If there's corresponding tool, ensure both work for each other.
- [ ] If needed, update our Javascript/Typescript APIs. These APIs are offcially used by exchanges or community developers.
- [ ] If we're going to issue a new release, freeze the code one week early(it depends, but usually it's one week), ensure we have enough time for related testing.
- [ ] Check if inheriting any upstream runtime storage migrations. If any, perform tests with `try-runtime`.
- [ ] If modifying existing runtime storage items, make sure to implement storage migrations for the runtime and test them with `try-runtime`. This includes migrations inhreited from upstream changes, and you can search the diffs for modifications of `#[pallet::storage]` items to check for any.
136 changes: 0 additions & 136 deletions .github/workflows/publish-draft-releases.yml
Original file line number Diff line number Diff line change
Expand Up @@ -175,98 +175,6 @@ jobs:
with:
name: config-for-integration-test
path: .github/resources/config-for-integration-test.json

build-node-base:
needs:
- start-node-builder-base
- check-for-runtime-upgrade
runs-on: ${{ needs.start-node-builder-base.outputs.runner-label }}
env:
CARGO_TERM_COLOR: always
steps:
-
if: ${{ needs.check-for-runtime-upgrade.outputs.do-versions-match == 'false' }}
uses: actions/checkout@v2
with:
ref: 'manta'
-
name: install sccache
if: ${{ needs.check-for-runtime-upgrade.outputs.do-versions-match == 'false' }}
env:
SCCACHE_RELEASE_URL: https://github.com/mozilla/sccache/releases/download
SCCACHE_VERSION: v0.2.15
run: |
SCCACHE_FILE=sccache-$SCCACHE_VERSION-x86_64-unknown-linux-musl
mkdir -p $HOME/.local/bin
curl -L "$SCCACHE_RELEASE_URL/$SCCACHE_VERSION/$SCCACHE_FILE.tar.gz" | tar xz
mv -f $SCCACHE_FILE/sccache $HOME/.local/bin/sccache
chmod +x $HOME/.local/bin/sccache
echo "$HOME/.local/bin" >> $GITHUB_PATH
echo "$HOME/.cargo/bin" >> $GITHUB_PATH
-
if: ${{ needs.check-for-runtime-upgrade.outputs.do-versions-match == 'false' }}
name: cache cargo registry
uses: actions/cache@v2
with:
path: |
~/.cargo/registry
~/.cargo/git
key: cargo-${{ hashFiles('**/Cargo.lock') }}
restore-keys: |
cargo-
-
if: ${{ needs.check-for-runtime-upgrade.outputs.do-versions-match == 'false' }}
name: cache sccache
uses: actions/cache@v2
continue-on-error: false
with:
path: /home/runner/.cache/sccache
key: sccache-${{ hashFiles('**/Cargo.lock') }}
restore-keys: |
sccache-
-
if: ${{ needs.check-for-runtime-upgrade.outputs.do-versions-match == 'false' }}
name: start sccache server
run: sccache --start-server
-
if: ${{ needs.check-for-runtime-upgrade.outputs.do-versions-match == 'false' }}
name: init
run: |
curl -s https://sh.rustup.rs -sSf | sh -s -- -y
source ${HOME}/.cargo/env
rustup toolchain install stable
rustup toolchain install nightly
rustup default stable
rustup target add wasm32-unknown-unknown --toolchain nightly
cargo +nightly install --git https://github.com/alexcrichton/wasm-gc --force
rustup update
-
if: ${{ needs.check-for-runtime-upgrade.outputs.do-versions-match == 'false' }}
name: build
env:
RUST_BACKTRACE: full
RUSTC_WRAPPER: sccache
SCCACHE_CACHE_SIZE: 2G
SCCACHE_DIR: /home/runner/.cache/sccache
run: |
source ${HOME}/.cargo/env
cargo build --profile production --verbose
-
if: ${{ needs.check-for-runtime-upgrade.outputs.do-versions-match == 'false' }}
name: stop sccache server
run: sccache --stop-server || true
-
if: ${{ needs.check-for-runtime-upgrade.outputs.do-versions-match == 'false' }}
name: strip
run: |
strip target/production/manta
-
if: ${{ needs.check-for-runtime-upgrade.outputs.do-versions-match == 'false' }}
name: upload
uses: actions/upload-artifact@v2
with:
name: manta-base
path: target/production/manta
-
name: upload
uses: actions/upload-artifact@v2
Expand All @@ -276,7 +184,6 @@ jobs:

runtime-upgrade-test:
needs:
- build-node-base
- build-node-current
- build-runtimes
- start-runtime-upgrade-tester
Expand Down Expand Up @@ -308,7 +215,6 @@ jobs:
name: fetch manta-base
uses: actions/download-artifact@v2
with:
# Workaround for 0.9.16 breaking change, using the latest binary instead for base.
name: manta
-
if: ${{ needs.check-for-runtime-upgrade.outputs.do-versions-match == 'false' }}
Expand Down Expand Up @@ -988,48 +894,6 @@ jobs:
runner-label: ${{ needs.start-node-builder-current.outputs.runner-label }}
aws-instance-id: ${{ needs.start-node-builder-current.outputs.aws-instance-id }}

start-node-builder-base:
runs-on: ubuntu-20.04
outputs:
runner-label: ${{ steps.start-self-hosted-runner.outputs.runner-label }}
aws-region: ${{ steps.start-self-hosted-runner.outputs.aws-region }}
aws-instance-id: ${{ steps.start-self-hosted-runner.outputs.aws-instance-id }}
steps:
-
id: start-self-hosted-runner
uses: audacious-network/aws-github-runner@v1.0.33
with:
mode: start
github-token: ${{ secrets.GH_SHR_TOKEN }}
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-instance-ssh-public-key: ${{ env.AWS_INSTANCE_SSH_PUBLIC_KEY }}
aws-region: ${{ env.AWS_REGION }}
aws-subnet-id: ${{ env.AWS_SUBNET_ID }}
aws-security-group-id: ${{ env.AWS_SECURITY_GROUP_ID } 6D4E }
aws-instance-type: ${{ env.AWS_INSTANCE_TYPE }} # 32 vcpu, 64gb ram, $1.392 hourly
aws-instance-root-volume-size: 32
aws-image-search-pattern: ${{ env.AWS_IMAGE_SEARCH_PATTERN }}
aws-image-search-owners: ${{ env.AWS_IMAGE_SEARCH_OWNERS }} # canonical

stop-node-builder-base:
needs:
- start-node-builder-base
- build-node-base
runs-on: ubuntu-20.04
if: ${{ always() }}
steps:
-
uses: audacious-network/aws-github-runner@v1.0.33
with:
mode: stop
github-token: ${{ secrets.GH_SHR_TOKEN }}
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: ${{ needs.start-node-builder-base.outputs.aws-region }}
runner-label: ${{ needs.start-node-builder-base.outputs.runner-label }}
aws-instance-id: ${{ needs.start-node-builder-base.outputs.aws-instance-id }}

start-integration-tester:
runs-on: ubuntu-20.04
outputs:
Expand Down
0