Secrets Management: From Hardcoded to Vault

The Secrets Problem

Every application has secrets: API keys, database passwords, encryption keys, OAuth tokens. The question is: where do you put them?

Common approaches (from worst to best):

Hardcoded in source code — Please don't

In config files — Better, but still risky

Environment variables — Common, but has limitations

Secrets manager — The right way

Let's talk about why you should migrate to a proper secrets manager and how to do it.

Why Environment Variables Aren't Enough

Environment variables are better than hardcoded secrets, but they have problems:

Visible in process listings — ps aux can expose them

Logged accidentally — Debug logs often dump the environment

Inherited by child processes — Secrets leak to subprocesses

No rotation mechanism — Changing secrets requires redeployment

No audit trail — No record of who accessed what

No encryption at rest — Stored in plain text

Enter HashiCorp Vault

Vault is the industry standard for secrets management. It provides:

Centralized secrets storage — One source of truth

Dynamic secrets — Short-lived credentials generated on demand

Encryption as a service — Encrypt data without managing keys

Audit logging — Full trail of secret access

Fine-grained access control — Who can access what secrets

Alternative Options

AWS Secrets Manager — Great if you're all-in on AWS

GCP Secret Manager — Same, for GCP

Azure Key Vault — Same, for Azure

Kubernetes Secrets + Sealed Secrets — K8s-native approach

Migration Strategy

Phase 1: Inventory Your Secrets

Before migrating, you need to know what secrets you have:

Scan your repos for hardcoded secrets (use gitleaks or trufflehog)

Review environment variables across all environments

Check config files for embedded credentials

Document each secret: What is it? Who owns it? Where is it used?

Phase 2: Set Up Your Secrets Manager

For Vault:

# Enable the KV secrets engine
vault secrets enable -path=secret kv-v2
# Create a policy for your application
vault policy write myapp - <path "secret/data/myapp/*" {
  capabilities = ["read"]
}
EOF# Create an auth method (e.g., Kubernetes)
vault auth enable kubernetes

Phase 3: Migrate Secrets

Start with non-production — Test the migration first

Migrate one secret at a time — Don't big-bang it

Update application code to fetch from Vault

Remove old secrets after confirming the new path works

Rotate the secret — The old one may be compromised

Phase 4: Implement Dynamic Secrets

Where possible, use dynamic secrets instead of static ones:

Database credentials:

# Configure database secrets engine
vault secrets enable database
vault write database/config/postgres   plugin_name=postgresql-database-plugin   connection_url="postgresql://{{username}}:{{password}}@postgres:5432/mydb"   allowed_roles="myapp"   username="vault"   password="vaultpassword"vault write database/roles/myapp   db_name=postgres   creation_statements="CREATE ROLE \"{{name}}\" WITH LOGIN PASSWORD '{{password}}' VALID UNTIL '{{expiration}}'; GRANT SELECT ON ALL TABLES IN SCHEMA public TO \"{{name}}\";"   default_ttl="1h"   max_ttl="24h"

Now your application gets short-lived database credentials that automatically expire.

Application Integration Patterns

Pattern 1: Vault Agent Sidecar

Vault Agent runs alongside your application and handles authentication and secret retrieval:

# Kubernetes example
annotations:
  vault.hashicorp.com/agent-inject: "true"
  vault.hashicorp.com/role: "myapp"
  vault.hashicorp.com/agent-inject-secret-db: "secret/data/myapp/database"

Pattern 2: Direct API Integration

Your application authenticates with Vault and fetches secrets directly:

import hvac
client = hvac.Client(url='https://vault.example.com')
client.auth.kubernetes.login(role='myapp', jwt=service_account_token)secret = client.secrets.kv.v2.read_secret_version(path='myapp/database')
db_password = secret['data']['data']['password']

Pattern 3: External Secrets Operator (Kubernetes)

Sync secrets from Vault to Kubernetes Secrets:

apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: myapp-secrets
spec:
  refreshInterval: 1h
  secretStoreRef:
    name: vault-backend
    kind: ClusterSecretStore
  target:
    name: myapp-secrets
  data:
  - secretKey: DB_PASSWORD
    remoteRef:
      key: secret/data/myapp/database
      property: password

Common Pitfalls

Not rotating secrets after migration — If a secret was ever in git, it's compromised

Over-permissive policies — Apply least privilege to secret access

No backup strategy — Vault data needs to be backed up

Single point of failure — Run Vault in HA mode

Logging secrets — Make sure your logging doesn't capture secret values

Migration Checklist

Need Help?

Secrets management migration can be complex, especially in production environments. If you want expert help implementing Vault or another secrets manager, let's talk.