Vault · Software Engineering Wiki

Cheatsheet #

Task	Command
Server state	`vault status`
Who is this token	`vault token lookup`
What can it do	`vault token capabilities <path>`
Log in with OIDC	`vault login -method=oidc`
Read a KV v2 secret	`vault kv get -mount=secret app/db`
One field only	`vault kv get -mount=secret -field=password app/db`
Write without shell history	`vault kv put -mount=secret app/db password=-` (reads stdin)
Previous version	`vault kv get -mount=secret -version=3 app/db`
Undelete	`vault kv undelete -mount=secret -versions=3 app/db`
List engines	`vault secrets list -detailed`
List auth methods	`vault auth list`
Read a policy	`vault policy read myapp`
Dynamic database credential	`vault read database/creds/readonly`
Renew a lease	`vault lease renew <lease_id>`
Revoke everything under a prefix	`vault lease revoke -prefix database/creds/readonly`

Seal, unseal and tokens #

Vault starts sealed: the master key is split with Shamir’s scheme (or held by a KMS with auto-unseal) and the data is unreadable until enough shares are provided. Every request afterwards carries a token, and every token has policies, a TTL and a parent — revoking a parent revokes the tree.

vault status                                  # sealed, HA mode, version, storage
vault operator unseal                         # repeat until threshold is met
vault operator seal                           # emergency stop: makes all data unreadable
vault token lookup                            # policies, TTL, renewable, entity
vault token capabilities secret/data/app/db   # exactly what this token may do at that path
vault token revoke -self

Auto-unseal with a cloud KMS removes the manual step and is the norm for production. Keep the recovery keys for it as carefully as you would keep unseal keys.

Auth methods #

Humans authenticate with OIDC or LDAP; machines should never use a static token.

vault auth enable oidc
vault auth enable kubernetes
vault auth enable approle

# Kubernetes: the pod's service account token is the credential
vault write auth/kubernetes/config kubernetes_host=https://kubernetes.default.svc
vault write auth/kubernetes/role/myapp \
  bound_service_account_names=api \
  bound_service_account_namespaces=apps \
  policies=myapp ttl=1h

# AppRole: for workloads outside Kubernetes
vault write auth/approle/role/ci token_policies=ci token_ttl=20m secret_id_ttl=10m
vault read auth/approle/role/ci/role-id
vault write -f auth/approle/role/ci/secret-id

Kubernetes auth exchanges the pod’s projected service account token for a Vault token, so nothing secret is stored in the manifest or the image. That property is the reason to prefer it.

Policies #

Policies are deny-by-default path rules. Capabilities are create, read, update, delete, list, sudo and deny; a deny anywhere wins.

# myapp policy
path "secret/data/app/*" {
  capabilities = ["read"]
}

path "secret/metadata/app/*" {
  capabilities = ["list"]
}

path "database/creds/readonly" {
  capabilities = ["read"]
}

path "secret/data/app/admin" {
  capabilities = ["deny"]
}

vault policy write myapp myapp.hcl
vault policy read myapp
vault token capabilities <token> secret/data/app/db

KV v2 splits the API path from the logical path: the secret at secret/app/db is read through secret/data/app/db and listed through secret/metadata/app/db. Policies written against secret/app/* silently grant nothing.

KV secrets #

vault secrets enable -path=secret -version=2 kv
vault kv put -mount=secret app/db username=api password="$PASS"
vault kv get -mount=secret -format=json app/db | jq -r '.data.data.password'
vault kv patch -mount=secret app/db password="$NEW"     # update one field, keep the rest
vault kv metadata get -mount=secret app/db              # versions, delete markers
vault kv delete -mount=secret app/db                    # soft delete of the latest version
vault kv undelete -mount=secret -versions=3 app/db
vault kv destroy -mount=secret -versions=3 app/db       # irreversible
vault kv metadata delete -mount=secret app/db           # removes every version

Passing a secret as a command argument puts it in shell history and the process list. Use key=- to read from stdin or key=@file.

Dynamic secrets #

Vault creates the credential on demand and deletes it when the lease expires, so a leaked credential has a bounded lifetime and an owner in the audit log.

vault secrets enable database
vault write database/config/prod \
  plugin_name=postgresql-database-plugin \
  allowed_roles=readonly \
  connection_url='postgresql://{{username}}:{{password}}@db.internal:5432/app?sslmode=require' \
  username=vault-admin password="$ADMIN_PASS"

vault write database/roles/readonly \
  db_name=prod \
  creation_statements="CREATE ROLE \"{{name}}\" WITH LOGIN PASSWORD '{{password}}' VALID UNTIL '{{expiration}}'; GRANT SELECT ON ALL TABLES IN SCHEMA public TO \"{{name}}\";" \
  default_ttl=1h max_ttl=24h

vault read database/creds/readonly        # returns username, password, lease_id
vault lease renew <lease_id>
vault lease revoke <lease_id>

The same pattern exists for AWS (aws/creds/<role>), cloud providers and SSH. Rotate the root credential Vault itself uses with vault write -f database/rotate-root/prod so nobody else knows it.

PKI #

vault secrets enable pki
vault secrets tune -max-lease-ttl=87600h pki
vault write pki/root/generate/internal common_name="Example Internal Root" ttl=87600h
vault write pki/roles/internal allowed_domains=example.internal allow_subdomains=true max_ttl=720h
vault write pki/issue/internal common_name=api.example.internal ttl=720h

Short-lived certificates issued on demand remove renewal as an operational task — see TLS for what the issued material means.

Transit #

Encryption as a service: the key never leaves Vault, so an application can encrypt without ever holding key material.

vault secrets enable transit
vault write -f transit/keys/app
vault write transit/encrypt/app plaintext="$(base64 <<< 'card number')"
vault write transit/decrypt/app ciphertext='vault:v1:...'
vault write -f transit/keys/app/rotate          # new version; old ciphertext still decrypts
vault write transit/rewrap/app ciphertext='vault:v1:...'

Troubleshooting #

Symptom	Cause
`permission denied`	Policy path wrong — KV v2 needs `data/` and `metadata/` prefixes
`missing client token`	`VAULT_TOKEN` unset, or the agent’s sink file not mounted
`Vault is sealed`	Restarted without auto-unseal, or a seal was triggered
Credentials stop working after an hour	Lease expired and nothing renewed it
`connection refused` on 8200	`VAULT_ADDR` wrong, or TLS expected and `http://` used
Works for me, fails for the app	Different auth method, different policies — compare `token lookup`

VAULT_ADDR=https://vault.example.internal:8200 vault status
vault token lookup -format=json | jq '{policies, ttl, renewable}'
vault read sys/internal/ui/mounts/secret/data/app/db     # which mount and capabilities apply
vault audit list                                          # confirm auditing is on

Enable at least one audit device in production. Vault refuses requests when every audit device fails, which is deliberate: no audit, no access.

Oneliners #

# Export a secret into the environment without touching disk
export DB_PASSWORD=$(vault kv get -mount=secret -field=password app/db)

# All keys under a path, recursively
vault kv list -mount=secret -format=json app | jq -r '.[]'

# Check what a policy allows at a path
vault token capabilities "$(vault print token)" secret/data/app/db

# Leases about to expire
vault list sys/leases/lookup/database/creds/readonly

# Revoke every credential a role issued
vault lease revoke -prefix database/creds/readonly

# Compare two policies
diff <(vault policy read app-a) <(vault policy read app-b)

# Render a config file from secrets
vault kv get -mount=secret -format=json app/db | jq -r '.data.data | to_entries[] | "\(.key)=\(.value)"' > .env

# Who is authenticated as what
vault list identity/entity/id | tail -n +3 | xargs -n1 -I{} vault read -format=json identity/entity/id/{} | jq -r '.data | [.name, (.policies//[]|join(","))] | @tsv'

# Confirm a Kubernetes role binds the service account you expect
vault read auth/kubernetes/role/myapp

# Test a login without saving the token
VAULT_TOKEN= vault write -field=token auth/approle/login role_id="$RID" secret_id="$SID"

# Seal status across an HA cluster
for h in vault-{1..3}.internal; do printf '%s ' "$h"; VAULT_ADDR="https://$h:8200" vault status -format=json | jq -r '[.sealed, .ha_mode] | @tsv'; done