supraj.dev

Interview questions & answers

HashiCorp Vault

Secrets management, encryption, PKI, and access control.

  • 13 questions
  • 2 easy
  • 6 medium
  • 5 hard

13 questions

Question 1Easy

What is HashiCorp Vault and what problems does it solve?

#
Show answer

Vault is a secrets management platform that securely stores, controls access to, and audited access of secrets (API keys, passwords, certificates, encryption keys). Core problems: (1) Centralized secrets storage — no more secrets in config files or env vars. (2) Dynamic secrets — generate credentials on-demand with TTL (no long-lived static creds). (3) Encryption as a Service — encrypt/decrypt data with centralized key management. (4) Identity-based access — authenticate via Kubernetes, AWS IAM, LDAP, JWT/OIDC.

Question 2Easy

What is a Vault secrets engine?

#
Show answer

A secrets engine handles reading, generating, or encrypting secrets. Types: (1) KV (Key-Value) — stores static secrets at versioned paths. (2) AWS/GCP/Azure — generates dynamic cloud credentials with IAM roles. (3) Database — generates dynamic DB credentials with TTL (PostgreSQL, MySQL, MongoDB). (4) PKI — generates on-demand TLS certificates. (5) Transit — encryption-as-a-service (data stays in app, keys in Vault). (6) TOTP — generates time-based one-time passwords. Engines are enabled at paths: vault secrets enable -path=aws aws.

Question 3Medium

What is a Vault policy and how is it structured?

#
Show answer

Vault policies grant permissions to paths using HCL: path "secret/data/myapp/*" { capabilities = ["read", "list"] }; path "aws/creds/myrole" { capabilities = ["read"] }. Capabilities: read, create, update, delete, list, sudo, deny. Policies are additive — a deny overrides other capabilities. Attach policies to auth methods: vault write auth/kubernetes/role/myrole bound_service_account_names=myapp policies=myapp-policy. Latest Vault recommends policy-as-code in version-controlled HCL files.

Question 4Medium

How does Vault's seal/unseal mechanism work?

#
Show answer

When Vault starts, it is sealed — it cannot read or serve any secrets. Unsealing requires a threshold of unseal keys (Shamir's Secret Sharing). Default: 5 key shares, 3 key threshold. Each unseal key decrypts part of the master key. Once the threshold is met, Vault loads the master key into memory and becomes operational. Auto-unseal: use cloud KMS (AWS KMS, GCP CKMS, Azure KeyVault) or a transit engine from another Vault cluster — eliminates manual unseal. The master key is encrypted by the KMS key; Vault automatically unseals using cloud KMS. Recommended for production.

Question 5Medium

How does Vault integrate with Kubernetes for pod authentication?

#
Show answer

The Vault Kubernetes auth method validates JWTs from the Kubernetes API server. Workflow: (1) App pod runs with a ServiceAccount. (2) Vault Agent Injector (mutating webhook) annotates the pod: vault.hashicorp.com/agent-inject: true. (3) Vault Agent sidecar authenticates using the pod's JWT at /var/run/secrets/kubernetes.io/serviceaccount/token. (4) Vault validates the JWT against the K8s API server, checks bound ServiceAccount/namespace, and issues a Vault token. (5) Agent sidecar fetches secrets and writes them to a shared volume (or injects as env vars). No static secrets in pod specs.

Question 6Hard

How does Vault's database secrets engine dynamically generate credentials?

#
Show answer

Configure a database connection: vault write database/config/postgresql plugin_name=postgresql-database-plugin allowed_roles=myrole connection_url="postgresql://{{username}}:{{password}}@host:5432/db". Create a role with TTL: vault write database/roles/myrole db_name=postgresql creation_statements="CREATE USER {{name}} WITH PASSWORD {{password}} VALID UNTIL {{expiration}}; GRANT SELECT ON ALL TABLES IN SCHEMA public TO {{name}};" default_ttl=1h max_ttl=24h. When apps read database/creds/myrole, Vault creates a new DB user, returns the credentials, and schedules automatic revocation after the TTL. Benefits: no static DB passwords, each app instance gets unique creds, automatic rotation.

Question 7Hard

How does Vault's PKI secrets engine work for internal TLS certificate management?

#
Show answer

Enable PKI: vault secrets enable pki. Generate a root CA (self-signed) or intermediate CA. Create a role: vault write pki/roles/internal allowed_domains=myapp.internal allow_subdomains=true max_ttl=72h key_type=ec. Apps request certificates: vault write pki/issue/internal common_name=api.myapp.internal ttl=24h. Vault returns signed cert + private key + CA chain. Automatic renewal: use Vault Agent template with renewal_ttl to keep certs fresh. Benefits: (1) Short-lived certs (hours, not months). (2) Automatic revocation via TTL expiry. (3) No manual CSR process. (4) EC or RSA key types. (5) Integration with cert-manager for Kubernetes: cert-manager Issuer of type Vault.

Question 8Hard

How do you back up and restore Vault in production?

#
Show answer

Vault stores data in the storage backend (Consul, Raft, Integrated Storage, file). Backup strategies: (1) Raft snapshot: vault operator raft snapshot save backup.snap (works with integrated storage/Consul). (2) File backend: snapshot the storage file. (3) Consul: consul snapshot save backup.snap. Restore: vault operator raft snapshot restore backup.snap. DR: For Vault Enterprise, Performance Replicas and Disaster Recovery clusters replicate data across regions. RPO/RTO considerations: Raft snapshots provide point-in-time recovery. Always back up the unseal keys or auto-unseal KMS configuration — without them, the backup is unrecoverable.

Question 9Medium

What is the Vault Agent and Vault Agent Injector?

#
Show answer

Vault Agent is a sidecar that authenticates with Vault, fetches/secrets, and manages token lifecycle. It runs alongside the app container. Agent Injector is a Kubernetes mutating admission webhook that automatically injects Vault Agent sidecars into pods based on annotations. Annotations: vault.hashicorp.com/agent-inject: true, vault.hashicorp.com/role: myapp, vault.hashicorp.com/agent-inject-secret-config.txt: secret/data/myapp. Agent renders secrets into a shared volume. Benefits: (1) No Vault SDK in the app. (2) Automatic token renewal. (3) Template-based secret injection. (4) Zero code changes for existing apps.

Question 10MediumEncryption

When would you use Vault's Transit engine instead of encrypting in the application?

#

Several services store customer PII and each implements its own encryption with keys in config files.

Show answer

Transit provides encryption as a service: an application sends plaintext to Vault and gets ciphertext back, prefixed with the key version (vault:v1:...). The keys never leave Vault, access is controlled by policy, and every use is audited.

Rotating a key adds a new version; old data still decrypts, and rewrap moves ciphertext to the latest version without the application seeing plaintext. You can raise the minimum decryption version after rewrapping. For large data, request a data key and encrypt locally (envelope encryption) so you do not send big payloads to Vault.

Trade-offs

Centralised keys simplify rotation and audit, but encryption now depends on Vault's availability and latency.

Common mistake

Caching plaintext keys in applications, which defeats the point of keeping them in Vault.

Follow-up

How would you rotate a Transit key and rewrap millions of records safely?

Question 11MediumKubernetes

Agent Injector, CSI provider or Vault Secrets Operator: how do you get Vault secrets into Pods?

#

A platform team must pick one default pattern for 80 services.

Show answer

The Agent Injector adds a Vault Agent sidecar that authenticates and renders secrets into a shared volume; secrets never become Kubernetes Secrets, and templates can format them. The CSI provider mounts secrets as files through the Secrets Store CSI driver, without a sidecar per Pod.

The Vault Secrets Operator syncs Vault secrets into Kubernetes Secrets and can restart workloads on change. Apps read them as normal Secrets, but the values now live in etcd, so encryption at rest and RBAC on Secrets matter.

Trade-offs

Operator sync is simplest for apps; the injector keeps secrets out of etcd at the cost of a sidecar per Pod.

Common mistake

Choosing the operator and leaving etcd unencrypted with broad Secret read access.

Follow-up

How would each option handle a database credential that rotates every hour?

Question 12HardAuthentication

CI needs Vault access. How do you avoid a long-lived Vault token in CI variables?

#

A root-like token stored in CI settings is used by every pipeline.

Show answer

Use the JWT/OIDC auth method. CI systems such as GitHub Actions and GitLab issue a signed token per job; Vault validates it against the issuer and grants a role only when bound_claims match, for example the repository, branch or environment.

Each role maps to a narrow policy, so the main branch of one repository can read only its deployment secrets. Tokens are short-lived and audited per job. Revoke the old static token afterwards.

Trade-offs

Per-repository roles take more setup, but loose claim bindings would let any pipeline read any secret.

Common mistake

Binding a role only to the issuer, which trusts every repository on that CI platform.

Follow-up

Which claims would you bind for production deploys from protected branches only?

Question 13HardOperations

Vault performance is degrading and the lease count keeps climbing. What happened?

#

After a new service launched, active leases went from 20,000 to 2 million.

Show answer

Every dynamic secret and service token creates a lease, and leases cost storage and expiry work. A spike usually means a client requests new credentials or logs in again on every request instead of reusing them, often combined with long TTLs, so leases pile up faster than they expire.

Find the source in the audit logs and the lease metrics per mount. Fix the client: cache credentials and tokens, for example with Vault Agent, renew rather than re-request, and set TTLs that match real use. Clean up with prefix revocation for the offending role once the client is fixed.

Trade-offs

Short TTLs limit exposure but increase renewal traffic; pick values per use case.

Common mistake

Adding Vault nodes instead of fixing the client that creates the leases.

Follow-up

How would you set a guardrail so one service cannot create unbounded leases again?