Open
Description
Hello,
I'm trying to centralise this action; we have close to 800 repos in our org, and I don't want to commit (or use .git) this action to each of them.
So, using the APP authentication, I gather a list of repos that have changed since it last ran:
echo "SARIFLASTRUNTIMESTAMP: $TIMESTAMP"
response=$(curl -s -H "Authorization: token ${{ steps.access_token.outputs.token }}" \
"https://api.github.com/search/repositories?q=org:ourOrgName+archived:false+pushed:>=$TIMESTAMP&per_page=$per_page&page=$page")
Then I update the timestamp with the now date.
so I have a JSON list of repos that the sarif uploader need to run on, so I create a matrix:
run-scorecard:
needs: fetch-repos
runs-on: ubuntu-latest
if: ${{ needs.fetch-repos.outputs.repo-list != '[]' }}
strategy:
matrix:
repo: ${{ fromJson(needs.fetch-repos.outputs.repo-list) }}
with the matrix item I re-authenticate with the APP, and run the scorecard (sarif)
The trouble is, the scorecard gathers the repo name from the repo info in file: /github/workflow/event.json - main workflow json;
Is there any way, as I have checked out the 'child repo' in the matrix, I can pass a input to the:
- name: "Run analysis"
uses: ossf/scorecard-action@v2.3.1
with:
results_file: results.sarif
results_format: sarif
repo_token: ${{ secrets.GITHUB_TOKEN }}
publish_results: false
so it runs against the repo I have checked out in the matrix?
By using this technique I stay in control of the updates of the security alerts (aligning with my other tools), and I can for example, nightly update the alerts for only repos that have had commits and produce scorecards for the org at the same time which get published...
I'll include my code (it's full of debug information steps atm, dirty wip) for completeness so you know what I am talking about:
name: New Central Sarif Alert Uploader
on:
workflow_dispatch:
permissions:
contents: read
issues: read
pull-requests: read
security-events: write
id-token: write
actions: read
checks: read
jobs:
fetch-repos:
runs-on: ubuntu-latest
outputs:
repo-list: ${{ steps.set-repo-list.outputs.repo-list }}
steps:
- name: Generate JWT
id: jwt
run: |
python - <<EOF
import os
import time
import jwt
private_key = os.getenv('SECDEVOPS_APP_PRIVATE_KEY', '').replace('\\n', '\n')
payload = {
'iat': int(time.time()),
'exp': int(time.time()) + (10 * 60),
'iss': os.getenv('SECDEVOPS_APP_ID')
}
token = jwt.encode(payload, private_key, algorithm='RS256')
print(f"::set-output name=token::{token}")
EOF
env:
SECDEVOPS_APP_PRIVATE_KEY: ${{ secrets.SECDEVOPS_APP_PRIVATE_KEY }}
SECDEVOPS_APP_ID: ${{ secrets.SECDEVOPS_APP_ID }}
- name: Get Installation Access Token
id: access_token
run: |
curl -X POST \
-H "Authorization: Bearer ${{ steps.jwt.outputs.token }}" \
-H "Accept: application/vnd.github.v3+json" \
https://api.github.com/app/installations/${{ secrets.SECDEVOPS_APP_INSTALLATION_ID }}/access_tokens \
> response.json
token=$(jq -r .token < response.json)
echo "::set-output name=token::$token"
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Fetch repository list based on timestamp
id: fetch
run: |
# Define test mode flag
TEST_MODE=${TEST_MODE:-false} # Set TEST_MODE to true to enable test mode true/false
# Set timestamp to a very early date if in test mode
if [ "$TEST_MODE" = true ]; then
echo "Test mode is enabled. Overriding TIMESTAMP to a very early date."
TIMESTAMP="1970-01-01T00:00:00Z"
else
TIMESTAMP=${{ vars.SARIFLASTRUNTIMESTAMP }}
fi
echo "SARIFLASTRUNTIMESTAMP: $TIMESTAMP"
# Validate timestamp format (ISO 8601 format check)
if [[ ! "$TIMESTAMP" =~ ^[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}Z$ ]]; then
echo "Invalid or missing SARIFLASTRUNTIMESTAMP: $TIMESTAMP"
exit 1
fi
page=1
per_page=100
> raw_repos.json
while true; do
response=$(curl -s -H "Authorization: token ${{ steps.access_token.outputs.token }}" \
"https://api.github.com/search/repositories?q=org:<redactedorg>+archived:false+pushed:>=$TIMESTAMP&per_page=$per_page&page=$page")
echo "API response for page $page:"
echo "$response"
repo_count=$(echo "$response" | jq -r '.items | length')
if [ "$repo_count" -eq 0 ]; then
break
fi
echo "$response" | jq -r '.items[] | "\(.full_name) - created_at: \(.created_at), pushed_at: \(.pushed_at)"'
echo "$response" | jq -r '.items[] | select(.size > 0) | .full_name' >> raw_repos.json # added check to not include empty repos
echo "Fetched page $page with $repo_count repositories"
page=$((page + 1))
done
echo "Repositories found (raw):"
cat raw_repos.json || echo "No repositories found"
# Filter to include only test repositories
filtered_repos=()
while IFS= read -r repo; do
repo=$(echo "$repo" | tr -d '\n' | tr -d '\r')
if [[ "$repo" =~ SecDevOps---(Test) ]]; then
filtered_repos+=("$repo")
fi
done < raw_repos.json
echo "Filtered repositories:"
printf "%s\n" "${filtered_repos[@]}"
if [ ${#filtered_repos[@]} -eq 0 ]; then
echo "No repositories to process."
echo "::set-output name=repo-list::[]"
exit 0
fi
printf "%s\n" "${filtered_repos[@]}" | jq -R -s -c 'split("\n")[:-1]' > repo_list.json
cat repo_list.json
# Toggle between filtered and raw repositories
USE_FILTERED=true # Default to false
if [ "$TEST_MODE" = true ]; then
USE_FILTERED=true # Override to true in test mode
fi
if [ "$USE_FILTERED" = true ]; then
echo "Using filtered repositories."
cp repo_list.json output.json
else
echo "Using raw repositories."
jq -R -s -c 'split("\n")[:-1]' < raw_repos.json > output.json
fi
- name: Set repository list output
id: set-repo-list
run: |
if [ -f output.json ]; then
echo "::set-output name=repo-list::$(cat output.json)"
else
echo "::set-output name=repo-list::[]"
fi
- name: Print repository list
run: |
echo "Repository list contents:"
cat output.json || echo "No repositories found"
- name: Upload repository list
uses: actions/upload-artifact@v3
with:
name: repo-list
path: output.json
retention-days: 1
- name: Upload raw repository list
uses: actions/upload-artifact@v3
with:
name: raw-repos
path: raw_repos.json
retention-days: 1
- name: Update SARIFLASTRUNTIMESTAMP
run: |
if [ "${{ env.UPDATE_TIMESTAMP }}" = "true" ]; then
echo "Setting SARIFLASTRUNTIMESTAMP variable"
NEW_TIMESTAMP=$(date -u +%Y-%m-%dT%H:%M:%SZ)
echo "Generated new timestamp: $NEW_TIMESTAMP"
gh api \
-H "Authorization: Bearer $GH_TOKEN" \
-X PATCH \
/repos/${{ github.repository }}/actions/variables/SARIFLASTRUNTIMESTAMP \
-f name=SARIFLASTRUNTIMESTAMP \
-f value="$NEW_TIMESTAMP"
echo "Verifying the variable was set"
gh api \
-H "Authorization: Bearer $GH_TOKEN" \
/repos/${{ github.repository }}/actions/variables \
| grep SARIFLASTRUNTIMESTAMP
else
echo "UPDATE_TIMESTAMP is set to false. Skipping timestamp update."
fi
env:
GH_TOKEN: ${{ steps.access_token.outputs.token }}
UPDATE_TIMESTAMP: true # Set to false to disable the timestamp update
#log-repo-list:
# needs: fetch-repos
# runs-on: ubuntu-latest
# steps:
# - name: Log Repository List
# run: echo "${{ needs.fetch-repos.outputs.repo-list }}"
run-scorecard:
needs: fetch-repos
runs-on: ubuntu-latest
if: ${{ needs.fetch-repos.outputs.repo-list != '[]' }}
strategy:
matrix:
repo: ${{ fromJson(needs.fetch-repos.outputs.repo-list) }}
steps:
- name: Print repository list
run: echo "${{ needs.fetch-repos.outputs.repo-list }}"
echo "The repo in the matrix is ${{ matrix.repo }}"
- name: Generate JWT
id: jwt
run: |
python - <<EOF
import os
import time
import jwt
private_key = os.getenv('SECDEVOPS_APP_PRIVATE_KEY', '').replace('\\n', '\n')
payload = {
'iat': int(time.time()),
'exp': int(time.time()) + (10 * 60),
'iss': os.getenv('SECDEVOPS_APP_ID')
}
token = jwt.encode(payload, private_key, algorithm='RS256')
print(f"::set-output name=token::{token}")
EOF
env:
SECDEVOPS_APP_PRIVATE_KEY: ${{ secrets.SECDEVOPS_APP_PRIVATE_KEY }}
SECDEVOPS_APP_ID: ${{ secrets.SECDEVOPS_APP_ID }}
- name: Get Installation Access Token
id: access_token
run: |
curl -X POST \
-H "Authorization: Bearer ${{ steps.jwt.outputs.token }}" \
-H "Accept: application/vnd.github.v3+json" \
https://api.github.com/app/installations/${{ secrets.SECDEVOPS_APP_INSTALLATION_ID }}/access_tokens \
> response.json
token=$(jq -r .token < response.json)
echo "::set-output name=token::$token"
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: "Checkout code"
id: checkout
uses: actions/checkout@v2
with:
repository: ${{ matrix.repo }}
token: ${{ steps.access_token.outputs.token }}
continue-on-error: true
- name: Confirm checkout and list directory
run: |
echo "Checkout directory: $(pwd)"
echo "Listing contents of the checked-out directory:"
ls -lah
- name: Check for checkout error
id: check_checkout
run: |
if [ ! -d .git ]; then
echo "checkout_failed=true" >> $GITHUB_ENV
echo "::warning::Checkout failed for ${{ matrix.repo }}"
fi
- name: Set Directory Permissions
run: sudo chmod -R 777 ${{ github.workspace }}
if: ${{ env.checkout_failed != 'true' }}
continue-on-error: true
- name: Extract repository name
id: extract_repo_name
run: |
echo "Matrix repo: ${{ matrix.repo }}"
repo_name=$(basename "${{ matrix.repo }}")
echo "Extracted repo name: $repo_name"
echo "::set-output name=repo_name::$repo_name"
if: ${{ env.checkout_failed != 'true' }}
continue-on-error: true
- name: Test GITHUB_ENV
run: |
echo "TEST_VAR=TestValue" >> $GITHUB_ENV
- name: Show TEST_VAR
run: |
echo "Test Variable: $TEST_VAR"
- name: Test modify GITHUB_ENV
run: |
echo "TEST_VAR=TestValue-${{ matrix.repo }}" >> $GITHUB_ENV
- name: Show modded TEST_VAR
run: |
echo "Test Variable: $TEST_VAR"
- name: Display event.json content
run: cat /home/runner/work/_temp/_github_workflow/event.json
- name: Show original variables
run: |
echo "Original GITHUB_REPOSITORY: $GITHUB_REPOSITORY"
echo "Original GITHUB_WORKSPACE: $GITHUB_WORKSPACE"
echo "Original GITHUB_EVENT_NAME: $GITHUB_EVENT_NAME"
echo "Original GITHUB_EVENT_PATH: $GITHUB_EVENT_PATH"
echo "Original GITHUB_REF: $GITHUB_REF"
echo "Original GITHUB_API_URL: $GITHUB_API_URL"
- name: "Run analysis"
uses: ossf/scorecard-action@v2.3.1
with:
results_file: results.sarif
results_format: sarif
repo_token: ${{ secrets.GITHUB_TOKEN }}
publish_results: false
- name: Debug SARIF file
run: |
echo "Contents of results.sarif:"
cat results.sarif
Metadata
Metadata
Assignees
Labels
No labels