Dynamically retrieve GitHub Actions secret

In case this can help, after reading the above answers which truly helped, the strategy I decided to use consists of storing my secrets as follow:

DB_USER_MASTER
DB_PASSWORD_MASTER
DB_USER_TEST
DB_PASSWORD_TEST

Where MASTER is the master branch for the prod environment and TEST is the test branch for the test environment.

Then, using the suggested solutions in this thread, the key is to dynamically generate the keys of the secrets variable. Those keys are generated via an intermediate step (called vars in the sample below) using outputs:

name: Pulumi up
on:
  push:
    branches:
      - master
      - test
jobs:
  up:
    name: Update
    runs-on: ubuntu-latest
    steps:
      - name: Create variables
        id: vars 
        run: |
          branch=${GITHUB_REF##*/} 
          echo "::set-output name=DB_USER::DB_USER_${branch^^}"
          echo "::set-output name=DB_PASSWORD::DB_PASSWORD_${branch^^}"
      - uses: actions/checkout@v2
        with:
          fetch-depth: 1
      - uses: docker://pulumi/actions
        with:
          args: up -s ${GITHUB_REF##*/} -y
        env:
          AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
          AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          GOOGLE_CREDENTIALS: ${{ secrets.GOOGLE_CREDENTIALS }}
          PULUMI_ACCESS_TOKEN: ${{ secrets.PULUMI_ACCESS_TOKEN }}  
          DB_USER: ${{ secrets[steps.vars.outputs.DB_USER] }}  
          DB_PASSWORD: ${{ secrets[steps.vars.outputs.DB_PASSWORD] }}

Notice the hack to get the branch on uppercase: ${branch^^}. This is required because GitHub forces secrets to uppercase.

Update - July 2021

I found a better way to prepare dynamic secrets in a job, and then consume those secrets as environment variables in other jobs.

Here's how it looks like in GitHub Actions.

My assumption is that each secret should be fetched according to the branch name. I'm getting the branch's name with this action rlespinasse/github-slug-action.

Go through the inline comments to understand how it all works together.

name: Dynamic Secret Names

# Assumption:
# You've created the following GitHub secrets in your repository:
# AWS_ACCESS_KEY_ID_master
# AWS_SECRET_ACCESS_KEY_master

on:
  push:

env:
  AWS_REGION: "eu-west-1"

jobs:
  prepare:
    name: Prepare
    runs-on: ubuntu-20.04
    steps:
      - uses: actions/checkout@v2
      - name: Inject slug/short variables
        uses: rlespinasse/[email protected]
      - name: Prepare Outputs
        id: prepare-step
        # Sets this step's outputs, that later on will be exported as the job's outputs
        run: |
          echo "::set-output name=aws_access_key_id_name::AWS_ACCESS_KEY_ID_${GITHUB_REF_SLUG}";
          echo "::set-output name=aws_secret_access_key_name::AWS_SECRET_ACCESS_KEY_${GITHUB_REF_SLUG}";
    # Sets this job's, that will be consumed by other jobs
    # https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#jobsjob_idoutputs
    outputs:
      aws_access_key_id_name: ${{ steps.prepare-step.outputs.aws_access_key_id_name }}
      aws_secret_access_key_name: ${{ steps.prepare-step.outputs.aws_secret_access_key_name }}

  test:
    name: Test
    # Must wait for `prepare` to complete so it can use `${{ needs.prepare.outputs.{output_name} }}`
    # https://docs.github.com/en/actions/reference/context-and-expression-syntax-for-github-actions#needs-context
    needs:
      - prepare
    runs-on: ubuntu-20.04
    env:
      # Get secret names
      AWS_ACCESS_KEY_ID_NAME: ${{ needs.prepare.outputs.aws_access_key_id_name }}
      AWS_SECRET_ACCESS_KEY_NAME: ${{ needs.prepare.outputs.aws_secret_access_key_name }}
    steps:
      - uses: actions/checkout@v2
      - name: Test Application
        env:
          # Inject secret values to environment variables
          AWS_ACCESS_KEY_ID: ${{ secrets[env.AWS_ACCESS_KEY_ID_NAME] }}
          AWS_SECRET_ACCESS_KEY: ${{ secrets[env.AWS_SECRET_ACCESS_KEY_NAME] }}
        run: |
          printenv | grep AWS_
          aws s3 ls

Update - August 2020

Following some hands-on experience with this project terraform-monorepo, here's an example of how I managed to use secret names dynamically

Secrets names are aligned with environments names and branches names - development, staging and production
$GITHUB_REF_SLUG comes from the Slug GitHub Action which fetches the name of the branch
The commands which perform the parsing are

      - name: set-aws-credentials
        run: |
          echo "::set-env name=AWS_ACCESS_KEY_ID_SECRET_NAME::AWS_ACCESS_KEY_ID_${GITHUB_REF_SLUG}"
          echo "::set-env name=AWS_SECRET_ACCESS_KEY_SECRET_NAME::AWS_SECRET_ACCESS_KEY_${GITHUB_REF_SLUG}"
      - name: terraform-apply
        run: |
          export AWS_ACCESS_KEY_ID=${{ secrets[env.AWS_ACCESS_KEY_ID_SECRET_NAME] }}
          export AWS_SECRET_ACCESS_KEY=${{ secrets[env.AWS_SECRET_ACCESS_KEY_SECRET_NAME] }}

Full example

name: pipeline

on:
  push:
    branches: [development, staging, production]
    paths-ignore:
      - "README.md"

jobs:
  terraform:
    runs-on: ubuntu-latest

    env:
      ### -----------------------
      ### Available in all steps, change app_name to your app_name
      TF_VAR_app_name: tfmonorepo
      ### -----------------------

    steps:
      - uses: actions/checkout@v2
      - name: Inject slug/short variables
        uses: rlespinasse/[email protected]
      - name: prepare-files-folders
        run: |
          mkdir -p ${GITHUB_REF_SLUG}/
          cp live/*.${GITHUB_REF_SLUG} ${GITHUB_REF_SLUG}/
          cp live/*.tf ${GITHUB_REF_SLUG}/
          cp live/*.tpl ${GITHUB_REF_SLUG}/ 2>/dev/null || true
          mv ${GITHUB_REF_SLUG}/backend.tf.${GITHUB_REF_SLUG} ${GITHUB_REF_SLUG}/backend.tf
      - name: install-terraform
        uses: little-core-labs/install-terraform@v1
        with:
          version: 0.12.28
      - name: set-aws-credentials
        run: |
          echo "::set-env name=AWS_ACCESS_KEY_ID_SECRET_NAME::AWS_ACCESS_KEY_ID_${GITHUB_REF_SLUG}"
          echo "::set-env name=AWS_SECRET_ACCESS_KEY_SECRET_NAME::AWS_SECRET_ACCESS_KEY_${GITHUB_REF_SLUG}"
      - name: terraform-apply
        run: |
          export AWS_ACCESS_KEY_ID=${{ secrets[env.AWS_ACCESS_KEY_ID_SECRET_NAME] }}
          export AWS_SECRET_ACCESS_KEY=${{ secrets[env.AWS_SECRET_ACCESS_KEY_SECRET_NAME] }}
          cd ${GITHUB_REF_SLUG}/
          terraform version
          rm -rf .terraform
          terraform init -input=false
          terraform get
          terraform validate
          terraform plan -out=plan.tfout -var environment=${GITHUB_REF_SLUG}
          terraform apply -auto-approve plan.tfout 
          rm -rf .terraform

After reading this - Context and expression syntax for GitHub Actions , focusing on env object, I found out that:

As part of an expression, you may access context information using one of two syntaxes.

Index syntax: github['sha']

Property dereference syntax: github.sha

So the same behavior applies to secrets, you can do secrets[secret_name], so you can do the following

    - name: Run a multi-line script
      env:
        SECRET_NAME: A_FRUIT_NAME
      run: |
        echo "SECRET_NAME = $SECRET_NAME"
        echo "SECRET_NAME = ${{ env.SECRET_NAME }}"
        SECRET_VALUE=${{ secrets[env.SECRET_NAME] }}
        echo "SECRET_VALUE = $SECRET_VALUE"

Which results in

SECRET_NAME = A_FRUIT_NAME
SECRET_NAME = A_FRUIT_NAME
SECRET_VALUE = ***

Since the SECRET_VALUE is redacted, we can assume that the real secret was fetched.

Things that I learned -

You can't reference env from another env, so this won't work
```
env:
  SECRET_PREFIX: A
  SECRET_NAME: ${{ env.SECRET_PREFIX }}_FRUIT_NAME
```
The result of SECRET_NAME is _FRUIT_NAME, not good
You can use context expressions in your code, not only in env, you can see that in SECRET_VALUE=${{ secrets[env.SECRET_NAME] }}, which is cool

And of course - here's the workflow that I tested - https://github.com/unfor19/gha-play/runs/595345435?check_suite_focus=true - check the Run a multi-line script step

There is a much cleaner option to achieve this using the format function.

Given set secrets DEV_A and TEST_A, the following two jobs will use those two secrets:

name: Secrets

on: [push]

jobs:

  dev:
    name: dev
    runs-on: ubuntu-18.04
    env:
      ENVIRONMENT: DEV
    steps:
      - run: echo ${{ secrets[format('{0}_A', env.ENVIRONMENT)] }}

  test:
    name: test
    runs-on: ubuntu-18.04
    env:
      ENVIRONMENT: TEST
    steps:
      - run: echo ${{ secrets[format('{0}_A', env.ENVIRONMENT)] }}

This also works with input provided through manual workflows (the workflow_dispatch event):

name: Secrets

on:
  workflow_dispatch:
    inputs:
      env:
        description: "Environment to deploy to"
        required: true

jobs:
  secrets:
    name: secrets
    runs-on: ubuntu-18.04
    steps:
      - run: echo ${{ secrets[format('{0}_A', github.event.inputs.env)] }}

Dynamically retrieve GitHub Actions secret

Update - July 2021

Update - August 2020

Full example

Tags:

Github

Github Actions

Related

Recent Posts