10000 ci: build microservices reproducibly using ko by leongross · Pull Request #1108 · edgelesssys/constellation · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

ci: build microservices reproducibly using ko #1108

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jan 30, 2023
Merged

ci: build microservices reproducibly using ko #1108

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
f945128
remove -ko suffix from workflows
leongross Jan 23, 2023
40d8173
update build_ko action, update build_micro_service_ko action, update …
leongross Jan 23, 2023
cd0eb60
fix: replace key-service/cmd with correct keyservice path
leongross Jan 23, 2023
29a3629
adjust helm charts to use hard coded kop binary path
leongross Jan 30, 2023
eeb7785
fix: typo and checkout ref
leongross Jan 30, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 10 additions & 3 deletions .github/actions/build_ko/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ inputs:
required: true
pushTag:
description: "Use this image tag"
required: false
required: true
githubToken:
description: "GitHub authorization token"
required: true
Expand Down Expand Up @@ -50,6 +50,12 @@ runs:
id: pseudo-version
uses: ./.github/actions/pseudo_version

- name: Setup Go environment
uses: actions/setup-go@6edd4406fa81c3da01a34fa6f6343087c207a568 # v3.5.0
with:
go-version: "1.19.5"
cache: true

- name: Set up ko
uses: imjasonh/setup-ko@ace48d793556083a76f1e3e6068850c1f4a369aa # v0.6

Expand All @@ -60,7 +66,7 @@ runs:
KO_USER: ${{ github.actor }}
KO_CONFIG_PATH: ${{ inputs.koConfig }}
KO_PASSWORD: ${{ inputs.githubToken }}
KO_DOCKER_REPO: ${{ inputs.registry }}/edgelesssys/${{ inputs.name }}-ko
KO_DOCKER_REPO: ${{ inputs.registry }}/edgelesssys/constellation/${{ inputs.name }}
run: |
tags=""
sbom=""
Expand Down Expand Up @@ -91,7 +97,8 @@ runs:
sbom="--sbom=none"
fi

container_full=$(ko build ${{ inputs.koTarget }} --bare --tags ${tags} ${sbom})
echo "Building container image with tags: ${tags}"
container_full=$(ko build ${{ inputs.koTarget }} --bare --tags "${tags}" ${sbom})
container_image=$(echo $container_full | cut -d@ -f1)
container_sha256=$(echo $container_full | cut -d: -f2)

Expand Down
8 changes: 4 additions & 4 deletions .github/actions/build_micro_service_ko/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,10 +27,10 @@ inputs:
default: "false"
cosignPublicKey:
description: "Cosign public key"
required: false
required: true
cosignPrivateKey:
description: "Cosign private key"
required: false
required: true
cosignPassword:
description: "Password for Cosign private key"
required: false
Expand All @@ -50,11 +50,11 @@ runs:
uses: ./.github/actions/build_ko
with:
name: ${{ inputs.name }}
koConfig: ${{ inputs.koConfig }}
pseudoVersion: ${{ inputs.pseudoVersion }}
koConfig: ${{ inputs.koConfig }}
koTarget: ${{ inputs.koTarget }}
pushTag: ${{ inputs.pushTag }}
githubToken: ${{ inputs.GITHUB_TOKEN }}
pushTag: ci-test

- name: Download ko Container Data
id: download_container_data
Expand Down
31 changes: 0 additions & 31 deletions .github/workflows/build-constellation-node-operator-ko.yml
View file
Open in desktop

This file was deleted.

4 changes: 2 additions & 2 deletions .github/workflows/build-constellation-node-operator.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,10 +30,10 @@ jobs:
go-version: "1.19.5"

- name: Build and upload constellation-node-operator container image
uses: ./.github/actions/build_operator
uses: ./.github/actions/build_micro_service_ko
with:
name: node-operator
sourceDir: operators/constellation-node-operator
koTarget: ./operators/constellation-node-operator
githubToken: ${{ secrets.GITHUB_TOKEN }}
cosignPublicKey: ${{ startsWith(github.ref, 'refs/heads/release/v') && secrets.COSIGN_PUBLIC_KEY || secrets.COSIGN_DEV_PUBLIC_KEY }}
cosignPrivateKey: ${{ startsWith(github.ref, 'refs/heads/release/v') && secrets.COSIGN_PRIVATE_KEY || secrets.COSIGN_DEV_PRIVATE_KEY }}
Expand Down
View file
Open in desktop

This file was deleted.

13 changes: 7 additions & 6 deletions .github/workflows/build-joinservice-image.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
name: Build and upload join-service image
name: Build and upload join service image
env:
REGISTRY: ghcr.io

on:
workflow_dispatch:
Expand Down Expand Up @@ -29,13 +31,12 @@ jobs:
with:
go-version: "1.19.5"

- name: Build and upload join-service container image
- name: Build and upload join service container image
id: build-and-upload
uses: ./.github/actions/build_micro_service
uses: ./.github/actions/build_micro_service_ko
with:
name: join-service
projectVersion: "0.0.0"
dockerfile: joinservice/Dockerfile
name: joinservice
koTarget: ./joinservice/cmd
githubToken: ${{ secrets.GITHUB_TOKEN }}
cosignPublicKey: ${{ startsWith(github.ref, 'refs/heads/release/v') && secrets.COSIGN_PUBLIC_KEY || secrets.COSIGN_DEV_PUBLIC_KEY }}
cosignPrivateKey: ${{ startsWith(github.ref, 'refs/heads/release/v') && secrets.COSIGN_PRIVATE_KEY || secrets.COSIGN_DEV_PRIVATE_KEY }}
Expand Down
43 changes: 0 additions & 43 deletions .github/workflows/build-keyservice-image-ko.yml
View file
Open in desktop

This file was deleted.

11 changes: 8 additions & 3 deletions .github/workflows/build-keyservice-image.yml
View file
Open in desktop
F438
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
name: Build and upload KeyService image
env:
REGISTRY: ghcr.io

on:
workflow_dispatch:
Expand Down Expand Up @@ -29,13 +31,16 @@ jobs:
with:
go-version: "1.19.5"

- name: Set up ko
uses: imjasonh/setup-ko@9a31684920a610d5dbe8012888714d64706f9787 # tag=v0.6

- name: Build and upload KeyService container image
id: build-and-upload
uses: ./.github/actions/build_micro_service
uses: ./.github/actions/build_micro_service_ko
with:
name: key-service
projectVersion: "0.0.0"
dockerfile: keyservice/Dockerfile
koConfig: .ko.yaml
koTarget: ./kms/cmd
githubToken: ${{ secrets.GITHUB_TOKEN }}
cosignPublicKey: ${{ startsWith(github.ref, 'refs/heads/release/v') && secrets.COSIGN_PUBLIC_KEY || secrets.COSIGN_DEV_PUBLIC_KEY }}
cosignPrivateKey: ${{ startsWith(github.ref, 'refs/heads/release/v') && secrets.COSIGN_PRIVATE_KEY || secrets.COSIGN_DEV_PRIVATE_KEY }}
Expand Down
37 changes: 0 additions & 37 deletions .github/workflows/build-verification-service-ko.yml
View file
Open in desktop

This file was deleted.

5 changes: 2 additions & 3 deletions .github/workflows/build-verification-service.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,11 +31,10 @@ jobs:

- name: Build and upload verification-service container image
id: build-and-upload
uses: ./.github/actions/build_micro_service
uses: ./.github/actions/build_micro_service_ko
with:
name: verification-service
projectVersion: "0.0.0"
dockerfile: verify/Dockerfile
koTarget: ./verify/cmd
githubToken: ${{ secrets.GITHUB_TOKEN }}
cosignPublicKey: ${{ startsWith(github.ref, 'refs/heads/release/v') && secrets.COSIGN_PUBLIC_KEY || secrets.COSIGN_DEV_PUBLIC_KEY }}
cosignPrivateKey: ${{ startsWith(github.ref, 'refs/heads/release/v') && secrets.COSIGN_PRIVATE_KEY || secrets.COSIGN_DEV_PRIVATE_KEY }}
Expand Down
44 changes: 38 additions & 6 deletions .github/workflows/release.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -115,19 +115,49 @@ jobs:
git push origin "${BRANCH}"

micro-services:
name: Build micro services
runs-on: ubuntu-22.04
needs: [verify-inputs, prepare-release-branch]
strategy:
matrix:
koTarget:
[./joinservice/cmd, ./keyservice/cmd, ./verify/cmd, ./operators/constellation-node-operator]
include:
- koTarget: ./joinservice/cmd
name: join-service
- koTarget: ./keyservice/cmd
name: key-service
- koTarget: ./verify/cmd
name: verification-service
- koTarget: ./operators/constellation-node-operator
name: node-operator
steps:
- name: Checkout
uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # v3.3.0
with:
ref: ${{ needs.verify-inputs.outputs.RELEASE_BRANCH }}

- name: Build ${{ matrix.name }} micro service
uses: ./.github/actions/build_micro_service_ko
with:
koTarget: ${{ matrix.koTarget }}
name: ${{ matrix.name }}
pushTag: ${{ inputs.version }}
githubToken: ${{ secrets.GITHUB_TOKEN }}
cosignPublicKey: ${{ startsWith(github.ref, 'refs/heads/release/v') && secrets.COSIGN_PUBLIC_KEY || secrets.COSIGN_DEV_PUBLIC_KEY }}
cosignPrivateKey: ${{ startsWith(github.ref, 'refs/heads/release/v') && secrets.COSIGN_PRIVATE_KEY || secrets.COSIGN_DEV_PRIVATE_KEY }}
cosignPassword: ${{ startsWith(github.ref, 'refs/heads/release/v') && secrets.COSIGN_PASSWORD || secrets.COSIGN_DEV_PASSWORD }}

micro-services-metadata:
name: Build micro services
needs: [verify-inputs, prepare-release-branch]
uses: ./.github/workflows/build-micro-service-manual.yml
permissions:
contents: read
packages: write
secrets: inherit
strategy:
matrix:
service:
[join-service, key-service, verification-service, qemu-metadata-api]
with:
microService: ${{ matrix.service }}
microService: qemu-metadata-api
imageTag: ${{ inputs.version }}
version: ${{ needs.verify-inputs.outputs.WITHOUT_V }}
ref: ${{ needs.verify-inputs.outputs.RELEASE_BRANCH }}
Expand All @@ -148,7 +178,7 @@ jobs:

update-versions:
name: Update container image versions
needs: [verify-inputs, micro-services, constellation-node-operator]
needs: [verify-inputs, micro-services]
runs-on: ubuntu-22.04
permissions:
contents: write
Expand Down Expand Up @@ -182,7 +212,9 @@ jobs:
yq eval -i ".version = \"${WITHOUT_V}\"" "cli/internal/helm/charts/edgeless/constellation-services/charts/${service}/Chart.yaml"
git add "cli/internal/helm/charts/edgeless/constellation-services/charts/${service}/Chart.yaml"
done

git add cli/internal/helm/charts/edgeless/constellation-services/Chart.yaml

yq eval -i ".version = \"${WITHOUT_V}\"" cli/internal/helm/charts/edgeless/operators/Chart.yaml
for service in node-maintenance-operator constellation-operator; do
yq eval -i "(.dependencies[] | select(.name == \"${service}\")).version = \"${WITHOUT_V}\"" cli/internal/helm/charts/edgeless/operators/Chart.yaml
Expand Down
2 changes: 1 addition & 1 deletion ...al/helm/charts/edgeless/operators/charts/constellation-operator/templates/deployment.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ spec:
- --metrics-bind-address=127.0.0.1:8080
- --leader-elect
command:
- /manager
- /ko-app/v2
env:
- name: KUBERNETES_CLUSTER_DOMAIN
value: {{ .Values.kubernetesClusterDomain }}
Expand Down
2 changes: 1 addition & 1 deletion ...helm/charts/edgeless/operators/charts/node-maintenance-operator/templates/deployment.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ spec:
- --metrics-bind-address=:8080
- --leader-elect
command:
- /manager
- /ko-app/v2
env:
- name: OPERATOR_NAMESPACE
valueFrom:
Expand Down
0