Shipsafershipsafer.ai
Security Tools

HashiCorp Vault Tutorial: Secrets Management for Production

Learn HashiCorp Vault's dynamic secrets, PKI, Kubernetes auth, transit encryption, and policies for secure secrets management in production environments.

March 9, 20266 min readShipSafer Team

HashiCorp Vault Tutorial: Secrets Management for Production

HashiCorp Vault is the leading open-source secrets management platform. It centralizes and controls access to tokens, passwords, certificates, and encryption keys — replacing hardcoded secrets, long-lived credentials, and scattered .env files with dynamic, short-lived, audited secrets. This guide covers the Vault features most critical for production use: dynamic secrets, PKI management, Kubernetes authentication, transit encryption, and policy design.

Core Concepts

Before diving in, three concepts are essential:

  • Secret engines — plugins that generate or store secrets. The KV engine stores static key-value secrets. The database engine generates dynamic database credentials. The PKI engine issues certificates.
  • Auth methods — how clients authenticate to Vault. AppRole, Kubernetes, AWS IAM, and GitHub are common choices.
  • Policies — HCL documents that control which secrets a token can read, write, or delete.

Vault's design principle is zero-trust by default: nothing has access unless explicitly granted by a policy attached to an authenticated identity.

Dynamic Database Secrets

The most impactful Vault capability for most applications is dynamic database credentials. Instead of a static DB_PASSWORD environment variable shared across your whole fleet, Vault generates unique, time-limited credentials per client.

Enable the database secret engine:

vault secrets enable database

Configure a PostgreSQL connection:

vault write database/config/myapp-db \
  plugin_name=postgresql-database-plugin \
  allowed_roles="myapp-readonly,myapp-readwrite" \
  connection_url="postgresql://{{username}}:{{password}}@db.example.com:5432/myapp" \
  username="vault-root" \
  password="vault-root-password"

Create a role with a credential template:

vault write database/roles/myapp-readwrite \
  db_name=myapp-db \
  creation_statements="CREATE ROLE \"{{name}}\" WITH LOGIN PASSWORD '{{password}}' VALID UNTIL '{{expiration}}';
    GRANT SELECT, INSERT, UPDATE ON ALL TABLES IN SCHEMA public TO \"{{name}}\";
    GRANT USAGE, SELECT ON ALL SEQUENCES IN SCHEMA public TO \"{{name}}\";" \
  default_ttl="1h" \
  max_ttl="24h"

Generate credentials:

vault read database/creds/myapp-readwrite

Output:

Key                Value
---                -----
lease_id           database/creds/myapp-readwrite/xKq3...
lease_duration     1h
lease_renewable    true
password           A1a-xK3mNq7...
username           v-token-myapp-rw-xKq3...

These credentials are valid for one hour and revoked automatically at expiry. If your application is compromised and the credentials leak, the blast radius is limited to one hour. Revoke the lease immediately with vault lease revoke <lease_id> to invalidate them instantly.

PKI: Internal Certificate Authority

Vault's PKI engine acts as an internal CA, issuing short-lived TLS certificates for service-to-service mTLS. This eliminates the need for long-lived wildcard certificates and manual cert rotation.

Enable PKI and generate a root CA:

vault secrets enable -path=pki pki
vault secrets tune -max-lease-ttl=87600h pki

# Generate root CA (store the certificate for distribution)
vault write -field=certificate pki/root/generate/internal \
  common_name="Internal Root CA" \
  ttl=87600h > root-ca.crt

Enable an intermediate CA (best practice — do not use root for leaf certs):

vault secrets enable -path=pki_int pki
vault secrets tune -max-lease-ttl=43800h pki_int

# Generate CSR
vault write -format=json pki_int/intermediate/generate/internal \
  common_name="Internal Intermediate CA" | jq -r '.data.csr' > pki_int.csr

# Sign with root CA
vault write -format=json pki/root/sign-intermediate \
  csr=@pki_int.csr format=pem_bundle ttl=43800h | jq -r '.data.certificate' > signed_cert.pem

# Import signed certificate
vault write pki_int/intermediate/set-signed certificate=@signed_cert.pem

Create a role for issuing certificates:

vault write pki_int/roles/myapp-service \
  allowed_domains="internal.example.com" \
  allow_subdomains=true \
  max_ttl=24h \
  generate_lease=true

Issue a certificate:

vault write pki_int/issue/myapp-service \
  common_name="api.internal.example.com" \
  ttl=8h

Applications can renew certificates before expiry using Vault Agent or the Vault SDK, enabling zero-downtime certificate rotation with 24-hour TTLs.

Kubernetes Authentication

In Kubernetes, Vault Auth with the Kubernetes method allows pods to authenticate using their service account JWT — no static secrets required.

Enable the Kubernetes auth method:

vault auth enable kubernetes

vault write auth/kubernetes/config \
  kubernetes_host="https://$KUBERNETES_PORT_443_TCP_ADDR:443" \
  token_reviewer_jwt="$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)" \
  kubernetes_ca_cert=@/var/run/secrets/kubernetes.io/serviceaccount/ca.crt \
  issuer="https://kubernetes.default.svc.cluster.local"

Create a Vault role binding a Kubernetes service account to a Vault policy:

vault write auth/kubernetes/role/myapp \
  bound_service_account_names=myapp-sa \
  bound_service_account_namespaces=production \
  policies=myapp-policy \
  ttl=1h

Deploy Vault Agent as a sidecar. Vault Agent handles authentication and secret injection, so your application code never calls Vault directly:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp
spec:
  template:
    metadata:
      annotations:
        vault.hashicorp.com/agent-inject: "true"
        vault.hashicorp.com/role: "myapp"
        vault.hashicorp.com/agent-inject-secret-db-creds: "database/creds/myapp-readwrite"
        vault.hashicorp.com/agent-inject-template-db-creds: |
          {{- with secret "database/creds/myapp-readwrite" -}}
          DB_USERNAME={{ .Data.username }}
          DB_PASSWORD={{ .Data.password }}
          {{- end -}}
    spec:
      serviceAccountName: myapp-sa

The Vault Agent sidecar writes the rendered template to /vault/secrets/db-creds as a file. Your application reads this file for database credentials. When the lease expires, Vault Agent renews it and rewrites the file automatically.

Transit Encryption: Encryption as a Service

The Transit secret engine provides encryption-as-a-service. Your application sends plaintext to Vault and receives ciphertext — the encryption key never leaves Vault.

Enable transit and create a key:

vault secrets enable transit
vault write transit/keys/user-pii type=aes256-gcm96

Encrypt data:

vault write transit/encrypt/user-pii \
  plaintext=$(echo -n "192.168.1.1 user@example.com" | base64)

Output:

ciphertext    vault:v1:8SDkp…

Decrypt data:

vault write transit/decrypt/user-pii \
  ciphertext="vault:v1:8SDkp…" | jq -r '.data.plaintext' | base64 -d

Key rotation is non-destructive. Rotating a transit key generates a new key version. New encryptions use the latest version. Old ciphertexts (marked vault:v1:) can still be decrypted using the previous key version. Force re-encryption of old data with vault write transit/rewrap/user-pii ciphertext="vault:v1:...".

For GDPR compliance, you can delete a transit key to cryptographically erase all data encrypted with it — useful for right-to-erasure use cases where encrypted data is stored in a database.

Writing Policies

Vault policies follow a least-privilege model. A policy grants explicit read, write, create, delete, list, or sudo access to specific paths.

Application read-only policy:

# myapp-policy.hcl

# Allow reading database credentials
path "database/creds/myapp-readwrite" {
  capabilities = ["read"]
}

# Allow using transit encryption
path "transit/encrypt/user-pii" {
  capabilities = ["update"]
}

path "transit/decrypt/user-pii" {
  capabilities = ["update"]
}

# Allow reading application config secrets
path "secret/data/myapp/*" {
  capabilities = ["read", "list"]
}

# Deny access to other namespaces
path "secret/data/other-team/*" {
  capabilities = ["deny"]
}

Apply the policy:

vault policy write myapp-policy myapp-policy.hcl

Admin policy for the CI/CD pipeline (write-only, no read):

# ci-deploy-policy.hcl
path "secret/data/myapp/*" {
  capabilities = ["create", "update", "delete", "list"]
}

# Deny reading secrets — deploy only, not read
path "secret/data/myapp/prod-*" {
  capabilities = ["deny"]
}

Separating write-only deploy access from read access ensures that even if your CI/CD pipeline is compromised, an attacker cannot exfiltrate production secrets — they can only overwrite them.

Audit Logging

Vault's audit log records every request and response with the caller's identity. Enable file-based audit logging:

vault audit enable file file_path=/var/log/vault/audit.log

Logs are HMAC-hashed to prevent accidental exposure of secrets in logs while retaining the ability to verify whether a specific value appeared in a request.

High Availability and Storage Backends

For production, run Vault with a replicated storage backend. The Integrated Storage (Raft) backend is the current recommendation — no external dependency required:

# vault.hcl
storage "raft" {
  path    = "/vault/data"
  node_id = "vault-node-1"

  retry_join {
    leader_api_addr = "https://vault-node-2:8200"
  }
  retry_join {
    leader_api_addr = "https://vault-node-3:8200"
  }
}

listener "tcp" {
  address     = "0.0.0.0:8200"
  tls_cert_file = "/vault/tls/vault.crt"
  tls_key_file  = "/vault/tls/vault.key"
}

api_addr     = "https://vault-node-1:8200"
cluster_addr = "https://vault-node-1:8201"

seal "awskms" {
  region     = "us-east-1"
  kms_key_id = "alias/vault-unseal"
}

The awskms seal (auto-unseal) uses AWS KMS to automatically unseal Vault on restart — critical for production availability. Similar options exist for Azure Key Vault, GCP Cloud KMS, and HashiCorp-managed keys.

Check Your Security Score — Free

See exactly how your domain scores on DMARC, TLS, HTTP headers, and 25+ other automated security checks in under 60 seconds.