Optimizing Costs with GitHub Actions: Dynamically Starting and Stopping Self-Hosted Runner VMs

4 min readNov 25, 2024

## Overview

In the realm of modern software development, using self-hosted runners for GitHub Actions can significantly streamline your CI/CD processes, especially when you need to operate within a specific network or access resources that only allow connections from a fixed IP. However, if your GitHub Actions aren’t triggered frequently, you may end up incurring costs for idle resources.

In this post, we’ll explore how to optimize costs by dynamically starting and stopping a Google Cloud Platform (GCP) Virtual Machine (VM) that acts as a self-hosted runner, using GitHub Actions. This approach is particularly suitable for smaller projects where more complex solutions like https://github.com/actions/actions-runner-controller might be overkill.

## Steps

### 1. Set Up the Trigger for Starting and Stopping the VM

We’ll use the workflow_run event in GitHub Actions to start the VM when a specified workflow is triggered and stop it after the workflow completes. Here’s how you can configure the trigger:

on:
  workflow_run: # to start/stop vm instance when workflow has been started/completed.
    workflows: [some-workflow-using-self-hosted-runner]
    types: 
      - requested # start
      - completed # completed

### 2. Determine the Action (Start or Stop)

Next, you’ll want to determine whether to start or stop the VM based on the event type. Here’s how to implement this in your workflow:

      - name: Determine action
        id: determine-action
        run: |
          if [[ "${{ github.event.action }}" == "requested" ]];then
            echo "action=start" >> "$GITHUB_OUTPUT"
            echo "MESSAGE=VM ${VM_NAME} has been stgarted." >> "$GITHUB_OUTPUT"
          else
            echo "action=stop" >> "$GITHUB_OUTPUT"
            echo "MESSAGE=VM ${VM_NAME} has been stopped." >> "$GITHUB_OUTPUT"
          fi

### 3. Authenticate and Manage the GCP VM

To manage your GCP resources, you need to authenticate using OpenID Connect and set up the Google Cloud SDK. Here’s how to do that:

Environment variables for the workload identity provider and service account:

env:
  WORKLOAD_IDENTITY_PROVIDER: xxxx
  SERVICE_ACCOUNT: xxx

The steps in the GitHub Actions workflow with google-github-actions/auth and google-github-actions/setup-gcloud:

      - name: Authenticate to Google Cloud
        uses: google-github-actions/auth@v2
        with:
          workload_identity_provider: ${{ env.WORKLOAD_IDENTITY_PROVIDER }}
          service_account: ${{ env.SERVICE_ACCOUNT }}

      - name: Set up Google Cloud SDK
        uses: google-github-actions/setup-gcloud@v2

      - name: "${{ steps.determine-action.outputs.action }} instance"
        if: steps.check-status.outputs.need_to_update == 'true'
        run: |
          echo "${{ steps.determine-action.outputs.action }} instance..."
          gcloud compute instances "${{ steps.determine-action.outputs.action }}" "${{ env.VM_NAME }}" --zone "${{ env.VM_ZONE }}" --project ${{ env.GCP_PROJECT }} --async

### 4. Check VM Status and Execute Start/Stop Commands

Before starting or stopping the VM, check its current status to avoid unnecessary commands:

      - name: Check the status of the instance
        id: check-status
        run: |
          status=$(gcloud compute instances describe "${{ env.VM_NAME }}" --zone "${{ env.VM_ZONE }}" --format="get(status)" --project ${{ env.GCP_PROJECT }})
          # set need to update desired status
          if [ "${{ steps.determine-action.outputs.action }}" == 'start' ] && [ "$status" == 'TERMINATED' ]; then
            echo "need_to_update=true" >> "$GITHUB_OUTPUT"
          elif [ "${{ steps.determine-action.outputs.action }}" == 'stop' ] && [ "$status" == 'RUNNING' ]; then
            echo "need_to_update=true" >> "$GITHUB_OUTPUT"
          else
            echo "need_to_update=false" >> "$GITHUB_OUTPUT"
          fi

### 5. Notify via Slack

To keep your team informed, send a notification to Slack whenever the VM is started or stopped:

      - name: Send slack notification
        if: steps.check-status.outputs.need_to_update == 'true'
        run: |
          curl -X POST -H 'Content-type: application/json' --data "{\"text\":\"${{ steps.determine-action.outputs.message }}\"}" ${{ secrets.SLACK_WEBHOOK_URL }}

### Final Workflow Example

Here’s how the complete workflow YAML would look after incorporating all the steps:

name: start-and-stop-self-hosted-runner

on:
  workflow_run: # to start/stop vm instance when workflow has been started/completed.
    workflows: [some-workflow-using-self-hosted-runner]
    types: 
      - requested # start
      - completed # completed

concurrency:
  group: start-and-stop-self-hosted-runner
  cancel-in-progress: true

env:
  WORKLOAD_IDENTITY_PROVIDER: projects/<project-number>/locations/global/workloadIdentityPools/<pool-name>/providers/<provider-name>
  SERVICE_ACCOUNT: github-actions@<your-gcp-project>.iam.gserviceaccount.com
  VM_NAME: test-instance-ubuntu
  VM_ZONE: asia-northeast1-c
  GCP_PROJECT: <your-gcp-project>

jobs:
  action:
    runs-on: ubuntu-latest
    permissions:
      id-token: write
      contents: read
      actions: read
    steps:
      - name: Determine action
        id: determine-action
        run: |
          if [[ "${{ github.event.action }}" == "requested" ]];then
            echo "action=start" >> "$GITHUB_OUTPUT"
            echo "MESSAGE=VM ${VM_NAME} has been stgarted." >> "$GITHUB_OUTPUT"
          else
            echo "action=stop" >> "$GITHUB_OUTPUT"
            echo "MESSAGE=VM ${VM_NAME} has been stopped." >> "$GITHUB_OUTPUT"
          fi

      - name: Authenticate to Google Cloud
        uses: google-github-actions/auth@v2
        with:
          workload_identity_provider: ${{ env.WORKLOAD_IDENTITY_PROVIDER }}
          service_account: ${{ env.SERVICE_ACCOUNT }}

      - name: Set up Google Cloud SDK
        uses: google-github-actions/setup-gcloud@v2

      - name: Check the status of the instance
        id: check-status
        run: |
          status=$(gcloud compute instances describe "${{ env.VM_NAME }}" --zone "${{ env.VM_ZONE }}" --format="get(status)" --project ${{ env.GCP_PROJECT }})
          # set need to update desired status
          if [ "${{ steps.determine-action.outputs.action }}" == 'start' ] && [ "$status" == 'TERMINATED' ]; then
            echo "need_to_update=true" >> "$GITHUB_OUTPUT"
          elif [ "${{ steps.determine-action.outputs.action }}" == 'stop' ] && [ "$status" == 'RUNNING' ]; then
            echo "need_to_update=true" >> "$GITHUB_OUTPUT"
          else
            echo "need_to_update=false" >> "$GITHUB_OUTPUT"
          fi

      - name: "${{ steps.determine-action.outputs.action }} instance"
        if: steps.check-status.outputs.need_to_update == 'true'
        run: |
          echo "${{ steps.determine-action.outputs.action }} instance..."
          gcloud compute instances "${{ steps.determine-action.outputs.action }}" "${{ env.VM_NAME }}" --zone "${{ env.VM_ZONE }}" --project ${{ env.GCP_PROJECT }} --async

      - name: Send slack notification
        if: steps.check-status.outputs.need_to_update == 'true'
        run: |
          curl -X POST -H 'Content-type: application/json' --data "{\"text\":\"${{ steps.determine-action.outputs.message }}\"}" ${{ secrets.SLACK_WEBHOOK_URL }}

## Summary

In this article, we explored how to efficiently manage a self-hosted runner on GCP by using GitHub Actions to start and stop the VM based on workflow triggers. This method is ideal for smaller projects that don’t require the complexity of more extensive scaling solutions.

By implementing this strategy, you can reduce costs associated with idle resources while maintaining the flexibility of self-hosted runners. As your project scales, you may want to consider more robust solutions, but for many use cases, this approach will serve you well.