Skip to content
Inviting nominations for the Champions of Scale - Nominate a leader building internet-scale data infastructure today! Nominate someone
Database ( latest 8.1.0 )
/Tools/Secret agent/Secret manager/GCP

GCP Secret Manager

Secret Agent can fetch secrets from GCP Secrets Manager in the following ways:

Attaching a Service Account

This is the recommended way to fetch secrets from GCP, because it does not require any GCP credentials to be explicitly specified in the configuration file. When Secret Agent runs on a GCP VM with a service account attached to it, it automatically uses the credentials of the attached service account. The attached service account must have a Secret Manager Secret Accessor role attached to it.

Use the following steps to configure Secret Agent on GCP VM and fetch the secrets:

  1. Create a secret in the GCP secret manager. In this example, the secret is named TestingSecret.
  2. Create a service account with the Secret Manager Secret Accessor role. In this example, the service account is named test-service-account@xxx.iam.gserviceaccount.com.
  3. Create a VM with the test-service-account@xxx.iam.gserviceaccount.com service account attached to it. From the GCP console, follow this sequence of steps:
    1. Click Create Instance in VM Instances under Compute Engine
    2. Go to Identity and API access
    3. Select test-service-account@xxx.iam.gserviceaccount.com in Service account
  4. Install Secret Agent on the GCP VM.
  5. Configure Secret Agent to fetch secrets from GCP Secrets Manager.
  6. Start the Secret Agent.
  7. Use the Secret Agent endpoint to fetch secrets.

The following is a sample configuration file for Secret Agent:

service:
  tcp:
    endpoint: 0.0.0.0:3005


secret-manager:
  gcp:
    resources:
      TestRes: projects/xxxxxxxxxxxx/secrets/TestingSecret/versions/latest


log:
  level: info

Application Default Credentials

Secret Agent can also run on any machine with GCP static credentials configured in the configuration file. You can use Application Default Credentials even when Secret Agent is running on an GCP VM instance without any service account attached to it. Application Default Credentials are also useful when Secret Agent uses a different service account from the one attached to the GCP VM instance.

Secret Agent looks for Application Default Credentials in the following order:

  1. Secret Agent configuration file.
  2. The environment variable GOOGLE_APPLICATION_CREDENTIALS.
  3. The default credentials file ~/.config/gcloud/application_default_credentials.json.
  4. GCP VM attached service account credentials.

Use the following procedure to configure Secret Agent to use GCP service account’s Application Default Credentials from GCP console:

  1. Create a service account with the Secret Manager Secret Accessor role. In this example, the service account is named test-service-account@xxx.iam.gserviceaccount.com.
  2. Create a secret in the GCP secret manager. In this example, the secret is named TestingSecret.
  3. Download the key file in JSON format for the test-service-account@xxx.iam.gserviceaccount.com service account.
  4. Install Secret Agent on the GCP VM instance.
  5. Configure Secret Agent and specify the downloaded service account credential file path in the config file by setting the credential-file config.
  6. Start Secret Agent.
  7. Use the Secret Agent endpoint to fetch secrets.

The following is a sample configuration file for Secret Agent:

service:
  tcp:
    endpoint: 0.0.0.0:3005


secret-manager:
  gcp:
    resources:
      TestRes: projects/xxxxxxxxxxxx/secrets/TestingSecret/versions/latest
    credential-file: <path-to-service-account-credential-file>


log:
  level: info

Service Account Impersonation

You can use service account Impersonation mode on top of the previous two modes (Attaching a Service Account & Application Default Credentials). When you use service account impersonation, you start with an authenticated principal (your Google Account or a service account) and request short-lived credentials for a service account that has the authorization for accessing secrets. The authenticated principal must have the necessary permissions to impersonate the service account.

Service account impersonation is more secure than using a service account key because service account impersonation requires a prior authenticated identity, and the credentials that are created by using impersonation do not persist. In comparison, authenticating with a service account key requires no prior authentication, and the persistent key is a high risk credential if exposed.

Use the following procedure to configure Secret Agent to fetch secrets by using an impersonated service account:

  1. Create a secret in the GCP secret manager. In this example, the secret is named TestingSecret.
  2. Create a service account with Service Account Token Creator role attached to it. In this example, this service account is named base-service-account@xxx.iam.gserviceaccount.com.
  3. Create a service account with a Secret Manager Secret Accessor role attached to it. In this example, this service account is named test-service-account@xxx.iam.gserviceaccount.com.
  4. Make sure that IAM Service Account Credentials API is enabled for GCP project.
  5. Create a GCP VM instance with base-service-account@xxx.iam.gserviceaccount.com service account attached to it. To do it from the GCP console follow this sequence of steps:
    1. Click Create Instance in VM Instances under Compute Engine
    2. Go to Identity and API access
    3. Select base-service-account@xxx.iam.gserviceaccount.com in Service account
  6. Install Secret Agent on the GCP VM instance.
  7. Configure Secret Agent to fetch secrets from GCP Secrets Manager. Add impersonate: test-service-account@xxx.iam.gserviceaccount.com in the config file to impersonate this service account which has permissions to access secrets.
  8. Start Secret Agent.
  9. Use the Secret Agent endpoint to fetch secrets.

The following is a sample configuration file for Secret Agent:

service:
  tcp:
    endpoint: 0.0.0.0:3005


secret-manager:
  gcp:
    resources:
      TestRes: projects/xxxxxxxxxxxx/secrets/TestingSecret/versions/latest
    credential-file: <path-to-base-service-account-credential-file>
    impersonate: test-service-account@xxx.iam.gserviceaccount.com


log:
  level: info

Accessing Secrets across projects using the same service account

You can access secrets across projects using the same service account credentials with the following steps.

  1. Create a service account with the Secret Manager Secret Accessor role in Project1. In this example, the service account from Project1 is named test-service-account-project1@xxx.iam.gserviceaccount.com.
  2. Create a secret in another GCP project, for example Project2. In this example, the secret from Project2 is named TestingSecretProject2.
  3. Download the key file in JSON format for test-service-account_project1@xxx.iam.gserviceaccount.com service account.
  4. Grant access to test-service-account-project1@xxx.iam.gserviceaccount.com in Project2 as Secret Manager Secret Accessor. From the GCP console follow these steps as
    1. Select Project2 from project list.
    2. Go to Grant Access in IAM
    3. Add test-service-account-project1@xxx.iam.gserviceaccount.com as principal and Assign role as Secret Manager Secret Accessor.
  5. Install Secret Agent on the GCP VM instance.
  6. Configure Secret Agent and specify the downloaded service account credential file path in the config file by setting credential-file configuration parameter.
  7. Start Secret Agent.
  8. Use the Secret Agent endpoint to fetch secrets.

The following is a sample configuration file for Secret Agent to access secrets across projects:

service:
  tcp:
    endpoint: 0.0.0.0:3005


secret-manager:
  gcp:
    resources:
      TestResProj2: projects/yyyyyyyyyyyy/secrets/TestingSecretProject2/versions/latest
    credential-file: <path-to-service-account-credential-file>  #Credentials file for `test-service-account-project1@xxx.iam.gserviceaccount.com`.


log:
  level: info

Accessing multiple versions of secrets

The value of the Secret in GCP Secret Manager can have multiple versions. To access multiple versions of the secret, specify them in the configuration file as shown in the following example:

service:
  tcp:
    endpoint: 0.0.0.0:3005


secret-manager:
  gcp:
    resources:
      TestResV1: projects/yyyyyyyyyyyy/secrets/TestingSecret/versions/1
      TestResV2: projects/yyyyyyyyyyyy/secrets/TestingSecret/versions/2
      TestResLatest: projects/yyyyyyyyyyyy/secrets/TestingSecret/versions/latest


log:
  level: info
Feedback

Was this page helpful?

What type of feedback are you giving?

What would you like us to know?

+Capture screenshot

Can we reach out to you?