Handling Credential Zero

Posted on by Dustin

Managing secrets is a criicial part of any infrastructure. In this post, we’ll discuss how to handle the “credential zero” problem in Infrastructure as Code.

What is Credential Zero?

Credential Zero is the concept that there is a point in your infrastructure where you need to provide a secret to get access to other secrets. This is the “zero” secret that you need to bootstrap your infrastructure. Sometimes this can be managed with a machine/service identity, as with many Cloud Service Providers. Examples include: Azure Managed Identities or an AWS IAM Role. Workload Identity Federation (based on OIDC) is another way to handle this problem.

Sometimes this cannot easily mitigate the concern.

Create minimally opinionated configurations

If there is ever an opportunity to eliminate configuration for such details, this is generally ideal. For instance, when working with Terraform, a provider configuration can be managed with attributes in the provider block and even take variable inputs to make the configuration more dynamic. However, this level of opinionation is walking down a predefined path for which authentication method to use.

For instance, using client_id, client_secret, tenant_id, and subscription_id properties in the AzureRM provider block means that any choice to implement a different authentication method means an update to the configuration. If the code has been written to be highly reusable, numerous workspaces could use the same code but they would all need to use the same authentication methods:

...
variable "client_secret" {
  description = "The client secret for the Service Principal"
  sensitive = true
  type      = string
}
...
provider "azurerm" {
  features {}

  client_id       = var.client_id
  client_secret   = var.client_secret
  tenant_id       = var.tenant_id
  subscription_id = var.subscription_id
}
...

Despite otherwise using good practices, it creates challenges. Instead, use an empty provider block (if supported by the provider; if it doesn’t, provide feedback requesting the feature) and use environment variables to provide the necessary information:

provider "azurerm" {
  features {}
}

In the shell, the following environment variables can be set for the same method of authentication:

export ARM_CLIENT_ID="00000000-0000-0000-0000-000000000000"
export ARM_CLIENT_SECRET="00000000-0000-0000-0000-exmplesecret"
export ARM_TENANT_ID="00000000-0000-0000-0000-000000000000"
export ARM_SUBSCRIPTION_ID="00000000-0000-0000-0000-000000000000"

If the authentication method should change to Workload Identity Federation (WIF) with an Azure Managed Identity, the code doesn’t need to change. The environment variables can be provided:

export ARM_USE_MSI="true"
export ARM_CLIENT_ID="00000000-0000-0000-0000-000000000000"
export ARM_TENANT_ID="00000000-0000-0000-0000-000000000000"
export ARM_SUBSCRIPTION_ID="00000000-0000-0000-0000-000000000000"

This would also necessitate the workload be executed from a platform that supports WIF.

Accessing secrets from HashiCorp Vault is no different. An initial secret must be used to authenticate to Vault.

Using attributes with variables in the provider block:

provider "vault" {
  address = var.vault_address
  token   = var.vault_token
}

An empty Vault provider block can be used with environment variables, as well (although, machine identities are the preferred mechanism):

provider "vault" {}

In the shell, the following environment variables can be set:

export VAULT_ADDR="https://vault.example.com:8200"
export VAULT_TOKEN="00000000-0000-0000-0000-000000000000"

If Vault is implementing the Userpass auth method, these additional environment variables can be set appropriately:

export TERRAFORM_VAULT_USERNAME="test-user"
export TERRAFORM_VAULT_PASSWORD="test-password"
export TERRAFORM_VAULT_SKIP_CHILD_TOKEN=true

Configurations for Vault

Similar to provider configuration in Terraform, Vault configurations may also need details for their own authentication. When Vault is initialized, it will generate unseal or recovery keys. In the case of recovery keys, these exist in the event that an auto-unseal configuration fails and remediation is required.

In order to configure the user of the HSM auto-unseal method, the following configuration is outlined in the documentation:

seal "pkcs11" {
  lib            = "/usr/vault/lib/libCryptoki2_64.so"
  slot           = "2305843009213693953"
  pin            = "AAAA-BBBB-CCCC-DDDD"
  key_label      = "vault-hsm-key"
  hmac_key_label = "vault-hsm-hmac-key"
}

This method still generate the keys from the Shamir Secret Sharing algorithm. However, the whole key is stored within the hardware HSM which requires a PIN and other details to retreive it.

Just as the Terraform provider situation, this is rather opinionated, but it often less problematic than a Terraform configuration. However, the details can all be supplied via environment variables:

export VAULT_SEAL_TYPE="pkcs11"
export VAULT_HSM_LIB="/usr/vault/lib/libCryptoki2_64.so"
export VAULT_HSM_SLOT="2305843009213693953"
export VAULT_HSM_KEY_LABEL="vault-hsm-key"
export VAULT_HSM_HMAC_KEY_LABEL= "vault-hsm-hmac-key"
export VAULT_HSM_PIN="AAAA-BBBB-CCCC-DDDD"

Providing these details means that no unseal stanza for the Vault configuration is even required.

This can all then be managed at run-time providing additional flexbility for the configuration to be reused.

Further, if generating the configuration with a template, not only does this prevent the secret from being hardcoded, it also simplifies the template and any necessary code to create it.

Conclusion

Handling Credential Zero is a challenge in Infrastructure as Code. However, by using environment variables, the configuration can be made more dynamic and reusable. This is especially important when the authentication method may change. By using environment variables, the configuration can be more easily managed and reused across multiple workspaces.

Leave a comment