Build golang code that uses modules in private repositories with cloudbuild | DSS

Build golang code that uses modules in private repositories with cloudbuild

Table of Contents

Problem

If you build golang source code that uses any modules hosted in the github private repository with cloudbuild, cloudbuild will be failed in go build because it will not be able to get private repository’s module.
Sample cloudbuild.yaml:

options:
  env:
    - GO111MODULE=on
  volumes:
    - name: go-modules
      path: /go

steps:
  - name: golang:1.11
    dir: .
    args: ["go", "test", "./..."]

  - name: golang:1.11
    dir: .
    args: ["go", "build", "-o", {build target file path}]
    env: ["CGO_ENABLED=0"]

When you execute this cloudbuild.yaml, you will get an following error.

optionsStep #0: go: github.com/dssolutioninc/{private repository name}@{version}: git fetch -f origin refs/heads/*:refs/heads/* refs/tags/*:refs/tags/* in /go/pkg/mod/cache/vcs/960bac95cc27711ae0971346df7b6c2f5b0d755d5c6f5e45bea54c22cf6501c6: exit status 128:
Step #0:        fatal: could not read Username for 'https://github.com': terminal prompts disabled
Step #0: go: error loading module requirements
Finished Step #0

How to resolve this problem

Generate a key pair
Add the publich key to private repository as a deploy key
Generate encrypted private key using Cloud KMS :
https://cloud.google.com/cloud-build/docs/access-private-github-repos
Add following configuration to cloudbuild.yaml
a) Decrypt encrypted private key
b) Add github’ signature to
~/.ssh/known_hosts
c) Add git config to use ssh access instead of https access

optionsStepoptions:
  env:
    - GO111MODULE=on
  volumes:
    - name: go-modules
      path: /go
    - name: "ssh"
      path: /root/.ssh

steps:
  - name: 'gcr.io/cloud-builders/gcloud'
    dir: .
    entrypoint: 'bash'
    args:
      - '-c'
      - |
        gsutil cp gs://{encrypted private key file path} dev_id_rsa.enc
        gcloud kms decrypt --ciphertext-file=./dev_id_rsa.enc --plaintext-file=/root/.ssh/id_rsa --location={location of key ring} --keyring={keyring name} --key={key name}

  - name: 'gcr.io/cloud-builders/git'
    dir: .
    entrypoint: 'bash'
    args:
      - '-c'
      - |
        chmod 600 /root/.ssh/id_rsa
        cat </root/.ssh/config
        Hostname github.com
        IdentityFile /root/.ssh/id_rsa
        EOF
        mv ./kms/dev/known_hosts /root/.ssh/known_hosts
        cat /root/.ssh/id_rsa
        git config --global url."git@github.com:".insteadOf "https://github.com/"

  - name: golang:1.11
    dir: .
    args: ["go", "test", "./..."]

  - name: golang:1.11
    dir: .
    args: ["go", "build", "-o", {build target file path}]
    env: ["CGO_ENABLED=0"]