Shipsafershipsafer.ai
Web Security

Secrets Rotation Automation: How to Rotate API Keys and Credentials Safely

Zero-downtime rotation strategies, AWS Secrets Manager auto-rotation Lambda patterns, GitHub Actions secret scanning, and GitGuardian integration for leaked credential response.

March 9, 20267 min readShipSafer Team

Secrets Rotation Automation: How to Rotate API Keys and Credentials Safely

Leaked credentials are the leading initial access vector in cloud breaches. GitHub's own research found that millions of secrets are exposed in public repositories every year, and many organizations have no automated process for detecting or rotating them. Manual rotation is error-prone, infrequent, and often skipped entirely because of the operational burden. Automating secrets rotation removes that friction and turns credential hygiene from a quarterly audit item into a continuous background process.

Why Rotation Matters

A secret that never changes has unlimited exposure time. If it is leaked — through a misconfigured log, a former employee, a compromised third-party service, or a git history mistake — the attacker has indefinite access until someone manually discovers and rotates the credential. Rotation bounds the attacker's window to the rotation interval.

For regulated environments, rotation requirements are explicit: PCI-DSS requires periodic rotation of service account passwords and API keys, and HIPAA technical safeguards require regular review of access credentials.

Zero-Downtime Rotation: The Two-Version Pattern

The most common mistake in rotation is rotating a secret before all consumers have switched to the new value. This causes downtime. The safe pattern is:

  1. Create the new credential alongside the old one.
  2. Deploy application configuration to use the new credential.
  3. Verify the new credential works in production.
  4. Revoke the old credential.

This requires that the secret provider supports two simultaneous active credentials. Most do:

  • Database passwords: Create a second DB user, grant identical permissions, rotate all connection strings to the new user, then drop the old user.
  • API keys with multiple versions: AWS IAM allows two active access keys per user; rotate the inactive one.
  • Stripe API keys: Stripe shows the key once; store in Secrets Manager and create a new one before revoking.
  • OAuth client secrets: Most IdPs allow two active client secrets during a transition window.

AWS Secrets Manager Auto-Rotation

AWS Secrets Manager has native rotation support via Lambda functions. For RDS databases, AWS provides managed rotation Lambdas. For custom credentials, you write a Lambda implementing a four-step interface.

RDS PostgreSQL auto-rotation (managed)

aws secretsmanager rotate-secret \
  --secret-id prod/myapp/db-password \
  --rotation-rules AutomaticallyAfterDays=30 \
  --rotate-immediately

AWS will handle the rotation schedule, invoke the built-in RDS rotation Lambda, and update the secret value atomically. Applications using the Secrets Manager SDK fetch the current value on each connection and automatically get the new password after rotation.

Custom rotation Lambda

For non-RDS services, implement a rotation Lambda:

// lambda/rotate-api-key/index.ts
import {
  SecretsManagerClient,
  GetSecretValueCommand,
  PutSecretValueCommand,
  DescribeSecretCommand,
  UpdateSecretVersionStageCommand,
} from '@aws-sdk/client-secrets-manager';

const client = new SecretsManagerClient({});

interface RotationEvent {
  SecretId: string;
  ClientRequestToken: string;
  Step: 'createSecret' | 'setSecret' | 'testSecret' | 'finishSecret';
}

export async function handler(event: RotationEvent): Promise<void> {
  const { SecretId, ClientRequestToken, Step } = event;

  switch (Step) {
    case 'createSecret':
      // Generate a new API key and store it as AWSPENDING
      const newApiKey = await generateNewApiKeyFromProvider();
      await client.send(new PutSecretValueCommand({
        SecretId,
        ClientRequestToken,
        SecretString: JSON.stringify({ apiKey: newApiKey }),
        VersionStages: ['AWSPENDING'],
      }));
      break;

    case 'setSecret':
      // Activate the AWSPENDING key in the external service
      const pending = await getSecretVersion(SecretId, 'AWSPENDING');
      await activateKeyInExternalService(pending.apiKey);
      break;

    case 'testSecret':
      // Verify the new key works
      const testPending = await getSecretVersion(SecretId, 'AWSPENDING');
      const ok = await testApiKey(testPending.apiKey);
      if (!ok) throw new Error('New API key test failed');
      break;

    case 'finishSecret':
      // Promote AWSPENDING to AWSCURRENT and demote old AWSCURRENT to AWSPREVIOUS
      const desc = await client.send(new DescribeSecretCommand({ SecretId }));
      const currentVersion = Object.entries(desc.VersionIdsToStages ?? {})
        .find(([, stages]) => stages.includes('AWSCURRENT'))?.[0];

      await client.send(new UpdateSecretVersionStageCommand({
        SecretId,
        VersionStage: 'AWSCURRENT',
        MoveToVersionId: ClientRequestToken,
        RemoveFromVersionId: currentVersion,
      }));

      // Revoke old key from the external service after a safe delay
      await revokeOldKeyFromExternalService(currentVersion);
      break;
  }
}

Wire this to a Secrets Manager rotation schedule:

aws secretsmanager rotate-secret \
  --secret-id prod/myapp/third-party-api-key \
  --rotation-lambda-arn arn:aws:lambda:us-east-1:123456789:function:rotate-api-key \
  --rotation-rules AutomaticallyAfterDays=90

Fetching secrets in your application

Never hard-code secrets or read them only at startup. Fetch from Secrets Manager with caching:

import { SecretsManagerClient, GetSecretValueCommand } from '@aws-sdk/client-secrets-manager';

const client = new SecretsManagerClient({});

interface CachedSecret {
  value: string;
  fetchedAt: number;
}

const cache = new Map<string, CachedSecret>();
const TTL_MS = 5 * 60 * 1000; // 5 minutes

async function getSecret(secretId: string): Promise<string> {
  const cached = cache.get(secretId);
  if (cached && Date.now() - cached.fetchedAt < TTL_MS) {
    return cached.value;
  }

  const response = await client.send(new GetSecretValueCommand({ SecretId: secretId }));
  const value = response.SecretString ?? '';
  cache.set(secretId, { value, fetchedAt: Date.now() });
  return value;
}

GitHub Actions Secret Scanning

GitHub scans every push to public repositories for known secret patterns (API keys, tokens, connection strings) across 200+ providers. For private repositories, enable GitHub Advanced Security secret scanning.

Setting up secret scanning on GitHub

# Enable via GitHub API (organization level)
gh api -X PATCH /orgs/{org}/secret-scanning \
  --field secret_scanning=true \
  --field secret_scanning_push_protection=true

Push protection is the key feature: it blocks the push before the secret reaches the remote, rather than alerting after the fact. Enable it for all repositories.

Responding to a secret scanning alert

When GitHub detects a leaked secret, the alert appears under Security > Secret scanning. Your immediate response should be:

  1. Revoke the secret immediately — do this before investigating how it leaked.
  2. Audit usage logs — check if the secret was used by anyone other than your application.
  3. Issue a new credential and deploy it.
  4. Scrub git history if the secret is in a commit (use git filter-repo, not git filter-branch).
  5. Add to .gitignore and verify your CI/CD does not log environment variables.

GitGuardian for Continuous Monitoring

GitGuardian provides real-time scanning of commits across your entire git history, CI/CD pipelines, and connected repositories. It detects 350+ secret types including custom internal patterns.

Integration with GitHub Actions

# .github/workflows/secrets-scan.yml
name: Secrets Scan
on: [push, pull_request]

jobs:
  gitguardian:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0  # Full history scan

      - name: GitGuardian scan
        uses: GitGuardian/ggshield-action@v1
        env:
          GITHUB_PUSH_BEFORE_SHA: ${{ github.event.before }}
          GITHUB_PUSH_BASE_SHA: ${{ github.event.base }}
          GITHUB_DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
          GITGUARDIAN_API_KEY: ${{ secrets.GITGUARDIAN_API_KEY }}

Custom secret patterns

For internal credentials with non-standard formats, define custom detectors:

# .gitguardian.yaml
version: 2
extra-headers:
  - name: X-Internal-Prefix
    regex: 'MYAPP_KEY_[A-Z0-9]{32}'
    keywords:
      - 'MYAPP_KEY_'

Rotation for Common Secret Types

Database connection strings

# Rotate RDS master password
aws rds modify-db-instance \
  --db-instance-identifier prod-db \
  --master-user-password "$(openssl rand -base64 32)" \
  --apply-immediately

# Update Secrets Manager
aws secretsmanager put-secret-value \
  --secret-id prod/myapp/db \
  --secret-string '{"password":"<new-password>","username":"myapp"}'

JWT signing secrets

For JWT secrets, generate a new secret and support both for the TTL of existing tokens:

// Support old and new secrets during rotation window
const SIGNING_SECRETS = [
  process.env.JWT_SECRET_NEW,    // Sign new tokens with this
  process.env.JWT_SECRET_OLD,    // Also accept tokens signed with this
].filter((s): s is string => Boolean(s));

function verifyToken(token: string): JWTPayload {
  for (const secret of SIGNING_SECRETS) {
    try {
      return jwt.verify(token, secret) as JWTPayload;
    } catch {
      // Try next secret
    }
  }
  throw new Error('Invalid token');
}

function signToken(payload: JWTPayload): string {
  return jwt.sign(payload, SIGNING_SECRETS[0], { expiresIn: '7d' });
}

After all existing tokens have expired (7 days in this case), remove JWT_SECRET_OLD.

Detecting Exposed Secrets Before They Reach Git

Pre-commit hooks with detect-secrets or truffleHog catch secrets before they are committed:

# Install detect-secrets
pip install detect-secrets
detect-secrets scan > .secrets.baseline

# Add to .pre-commit-config.yaml
repos:
  - repo: https://github.com/Yelp/detect-secrets
    rev: v1.4.0
    hooks:
      - id: detect-secrets
        args: ['--baseline', '.secrets.baseline']

Automating secrets rotation from creation through detection, rotation, and revocation closes the window of exposure to minutes rather than months. The investment in automation pays for itself the first time it catches a leaked credential before an attacker does.

Check Your Security Score — Free

See exactly how your domain scores on DMARC, TLS, HTTP headers, and 25+ other automated security checks in under 60 seconds.