Setup GitHub Actions and Terraform for a new GCP project

5 min readJan 10, 2023

Overview

When you start a new GCP project, it’s common to set up a GitHub Actions workflow to manage GCP resources. In this post, I’ll share how to easily set up a GitHub Actions with a Service Account that utilizes Workload identity federation. Workload identity federation uses short-lived access token instead of service account key.

We’ll create the following resources step by step:

A new GCP project
GCS bucket that stores Terraform state for the new GCP project
Service Account with workload identity provider for GitHub Actions
GitHub Actions to manage the newly create GCP project with the Service Account

Steps

1. Create a new project

gcloud projects create <new_project_id>

For more details, you can reference the doc of the command.

(requires roles/resourcemanager.projectCreator role. Ref: Creating a project)

2. Link the project to your billing account

gcloud alpha billing accounts projects link <new_project_id> --billing-account=0X0X0X-0X0X0X-0X0X0X

Ref: gcloud alpha billing accounts projects link

(requires roles/billing.user role. Ref: Billing Access)

3. Set up Terraform for the new project

3.1. Prepare gcloud cli config

PROJECT_ID=<new_project_id>
gcloud auth application-default login --project $PROJECT_ID
gcloud config set project $PROJECT_ID

Confirm gcloud is configured with <new_project_id>

gcloud config list

[core]
account = your@email.com
disable_usage_reporting = False
project = <new_project_id>

Your active configuration is: [default]

3.2. Create a GCS bucket for Terraform backend

The following command creates a bucket named ${PROJECT_ID}-terraform but you can name it as you like. You can also specify a preferred location e.g. asia-northeast1

gcloud storage buckets create gs://${PROJECT_ID}-terraform --project $PROJECT_ID --location <location>

After executing the command, you can confirm the bucket is created with the following command:

gcloud storage ls --project $PROJECT_ID

3.3. Create Terraform codes.

terraform.tf

terraform {
  required_providers {
    google = {
      source  = "hashicorp/google"
      version = "~> 4.0" # OIDC https://github.com/hashicorp/terraform-provider-google/releases/tag/v3.61.0
    }
  }

  backend "gcs" {
    bucket = "<project-name>-terraform" # need to update with the bucket name
    prefix = "state"
  }
}

provider "google" {
  project = var.project
  region  = var.region
}

variables.tf

variable "project" {
  type    = string
  default = "<project name>"
}

variable "region" {
  type    = string
  default = "asia-northeast1"
}

enabled-apis.tf

locals {
  services = toset([
    # MUST-HAVE for GitHub Actions setup
    "iam.googleapis.com",                  # Identity and Access Management (IAM) API
    "iamcredentials.googleapis.com",       # IAM Service Account Credentials API
    "cloudresourcemanager.googleapis.com", # Cloud Resource Manager API
    "sts.googleapis.com",                  # Security Token Service API

    # You can add more apis to enable in the project
  ])
}

resource "google_project_service" "service" {
  for_each = local.services
  project  = var.project
  service  = each.value
}

3.4. Apply Terraform codes

Check the Terraform version:

terraform -v

Initialize Terraform

terraform init

Check GCS

gsutil ls -p $PROJECT_ID gs://${PROJECT_ID}-terraform/state/

You’ll see gs://<new_project_id>-terraform/state/default.tfstate file.

Plan and apply

terraform plan
terraform apply

4. Set up Service Account for GitHub Actions

4.1. Create github-actions-setup.tf

locals {
  roles = [
    "roles/resourcemanager.projectIamAdmin", # GitHub Actions identity
    "roles/editor", # allow to manage all resources
  ]
  github_repository_name = "<your github repo name>" # e.g. yourname/yourrepo
}

resource "google_service_account" "github_actions" {
  project      = var.project
  account_id   = "github-actions"
  display_name = "github actions"
  description  = "link to Workload Identity Pool used by GitHub Actions"
}

# Allow to access all resources
resource "google_project_iam_member" "roles" {
  project = var.project
  for_each = {
    for role in local.roles : role => role
  }
  role   = each.value
  member = "serviceAccount:${google_service_account.github_actions.email}"
}

resource "google_iam_workload_identity_pool" "github" {
  provider                  = google-beta
  project                   = var.project
  workload_identity_pool_id = "github"
  display_name              = "github"
  description               = "for GitHub Actions"
}

resource "google_iam_workload_identity_pool_provider" "github" {
  provider                           = google-beta
  project                            = var.project
  workload_identity_pool_id          = google_iam_workload_identity_pool.github.workload_identity_pool_id
  workload_identity_pool_provider_id = "github-provider"
  display_name                       = "github actions provider"
  description                        = "OIDC identity pool provider for execute GitHub Actions"
  # See. https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/about-security-hardening-with-openid-connect#understanding-the-oidc-token
  attribute_mapping = {
    "google.subject"       = "assertion.sub"
    "attribute.repository" = "assertion.repository"
    "attribute.owner"      = "assertion.repository_owner"
    "attribute.refs"       = "assertion.ref"
  }

  oidc {
    issuer_uri = "https://token.actions.githubusercontent.com"
  }
}

resource "google_service_account_iam_member" "github_actions" {
  service_account_id = google_service_account.github_actions.name
  role               = "roles/iam.workloadIdentityUser"
  member             = "principalSet://iam.googleapis.com/${google_iam_workload_identity_pool.github.name}/attribute.repository/${local.github_repository_name}"
}

output "service_account_github_actions_email" {
  description = "Service Account used by GitHub Actions"
  value       = google_service_account.github_actions.email
}

output "google_iam_workload_identity_pool_provider_github_name" {
  description = "Workload Identity Pood Provider ID"
  value       = google_iam_workload_identity_pool_provider.github.name
}

4.2. Plan and Apply

terraform init
terraform plan
terraform apply

4.3. Get Terraform output

terraform output
google_iam_workload_identity_pool_provider_github_name = "projects/<org_id>/locations/global/workloadIdentityPools/github/providers/github-provider"
service_account_github_actions_email = "github-actions@<new_project_id>.iam.gserviceaccount.com"

You’ll use this value in the GitHub Actions’ yaml file.

5. Create GitHub Actions

Add the following code .github/workflows/gcp-<new_project_id>.yml.

You also need to update the following values with your owns:

on.pull_request.paths
env.TERRAFORM_VERSION
env.WORKING_DIR
workload_identity_provider: terraform output from the last step
service_account: terraform output from the last step

name: gcp-<new_project_id>
on:
  pull_request:
    # paths: # setup paths if necessary
    branches:
      - main
    types:
      - opened # default
      - synchronize # default
      - reopened # default
      - closed

env:
  TERRAFORM_VERSION: 1.3.7
  WORKING_DIR: path/to/your/code # relative path under which your terraform codes are

jobs:
  terraform:
    # Add "id-token" with the intended permissions.
    permissions:
      contents: read
      id-token: write
      pull-requests: write
    runs-on: ubuntu-latest
    defaults:
      run:
        working-directory: ${{ env.WORKING_DIR }}
    steps:
      - uses: actions/checkout@v3

      - id: 'auth'
        name: 'Authenticate to Google Cloud'
        uses: google-github-actions/auth@v0.7.0
        with:
          create_credentials_file: 'true'
          workload_identity_provider: <write what you got from terraform output>
          service_account: <write what you got from terrafrom output>

      - uses: hashicorp/setup-terraform@v1

      - name: Terraform fmt
        id: fmt
        run: terraform fmt -check

      - name: Terraform Init
        id: init
        run: terraform init

      - name: Terraform Validate
        id: validate
        run: terraform validate -no-color

      - name: Terraform Plan
        id: plan
        run: terraform plan -no-color

      - uses: actions/github-script@v6
        if: github.event.pull_request.merged != true
        with:
          github-token: ${{ secrets.GITHUB_TOKEN }}
          script: |
            const output = `#### Terraform Format and Style 🖌\`${{ steps.fmt.outcome }}\`
            #### Terraform Initialization ⚙️\`${{ steps.init.outcome }}\`
            #### Terraform Validation 🤖\`${{ steps.validate.outcome }}\`
            <details><summary>Validation Output</summary>
            \`\`\`\n
            ${{ steps.validate.outputs.stdout }}
            \`\`\`
            </details>
            #### Terraform Plan 📖\`${{ steps.plan.outcome }}\`
            <details><summary>Show Plan</summary>
            \`\`\`\n
            ${{ steps.plan.outputs.stdout }}
            \`\`\`
            </details>
            *Pusher: @${{ github.actor }}, Action: \`${{ github.event_name }}\`, Working Directory: \`${{ env.tf_actions_working_dir }}\`, Workflow: \`${{ github.workflow }}\`*`;
            github.rest.issues.createComment({
              issue_number: context.issue.number,
              owner: context.repo.owner,
              repo: context.repo.repo,
              body: output
            })

      - name: Terraform Apply
        id: apply
        if: github.event.pull_request.merged == true
        run: terraform apply -auto-approve -input=false
      - uses: actions/github-script@v6
        if: github.event.pull_request.merged == true
        with:
          script: |
            const output = `#### Terraform Format and Style 🖌\`${{ steps.fmt.outcome }}\`
            #### Terraform Initialization ⚙️\`${{ steps.init.outcome }}\`
            #### Terraform Validation 🤖\`${{ steps.validate.outcome }}\`
            #### Terraform Apply 📖\`${{ steps.apply.outcome }}\`
            <details><summary>Show Apply</summary>
            \`\`\`\n
            ${{ steps.apply.outputs.stdout }}
            \`\`\`
            </details>
            *Pusher: @${{ github.actor }}, Action: \`${{ github.event_name }}\`, Working Directory: \`${{ env.tf_actions_working_dir }}\`, Workflow: \`${{ github.workflow }}\`*`;
            github.rest.issues.createComment({
              issue_number: context.issue.number,
              owner: context.repo.owner,
              repo: context.repo.repo,
              body: output
            })

6. Commit, push and create a PR

Now you can commit all the files (Terraform codes & GitHub Actions yaml files), push to your branch (not the main branch), and create a PR to the main branch.

The new GitHub Actions gcp-<new_project_id>will be triggered and you’ll see a comment from GitHub Actions

6. Add more resources with Terraform codes

Now the pipeline to apply changes to GCP resources with Terraform is ready, which automatically executes plan in a PR and apply when a PR is merged to main branch.

Summary

We need to set up a CI/CD pipeline for GCP resources every time we create a new project. It’s better to make a fixed steps for setup. I shared a way to set up a new GCP project with GitHub Actions easily. Hopefully, it’ll help your daily work!