Plan normal and emergency rotation
Routine rotation can use a short two-key overlap only when the provider supports it. Emergency rotation after exposure may need rapid revocation and service disruption handling. Test both plans. Never copy a raw key into a ticket or log; track it by key ID. Use a secret store that can show which deployment received the new value. After cutover, remove old values from config and confirm that an old-key request is denied.
Test cases and proof
| Case | Expected result | Proof to keep |
|---|---|---|
| New key is wrong in one worker | Restore old key during overlap | Worker success rate |
| Old key used after revocation | Deny and alert | Gateway log |
| Key appears in application log | Remove and rotate | Log review |
Synthetic example
Issue key B while key A still works. Move a worker to B and check its calls. After every worker uses B, revoke A and test one old request. If B fails during overlap, switch consumers back to A before A is revoked.
Evidence to keep
Keep a call ledger for each consumer: key ID used, endpoint, response, and time. Do not record the secret itself. After old-key revocation, one controlled call with that key should fail, while the new key still works. Save the secret-store version and deployment ID so a failed worker can be traced to the configuration it received.
Choose the moment of revocation
A zero-downtime rotation needs a period when both keys work. That period is also a time when either key can be abused. Keep it short and track old-key calls by consumer. Do not infer migration from a deployment success message; a dormant worker may not call until tomorrow. Test scheduled tasks and failure recovery paths before revocation. Emergency rotation after a leak is different: revoke quickly, notify owners of broken callers, and use a planned fallback. A key ID in logs is useful; a raw key in logs creates another secret leak.
Related reading
Why these checks matter
NIST’s guidance covers cryptographic key lifecycle and is background context, not an API-token overlap contract. OWASP’s secrets guide calls for storage, auditing, rotation, and revocation. The overlap window in this plan is an operational method: it gives each consumer time to switch while usage by key ID shows whether the old key can safely be removed.
Check whether overlap exists
Before planning two active keys, check the provider’s actual key model. Some services issue separate scoped credentials; others replace a single key immediately. Record that behavior in a staging exercise. For single-key replacement, coordinate cutover, stop or queue work, update all consumers, and retry safe operations after recovery. A two-key plan cannot create overlap that the provider does not support.
Inventory dormant consumers as well as busy ones: scheduled settlement import, manual support retry, old deployment, and disaster-recovery worker. A quiet K1 metric does not prove a monthly job has moved. Run each consumer with K2 and its least-used required endpoint. Compare denied as well as accepted operations so K2 gains no unplanned scope.
Keep compromised keys out of rollback
Before K1 is revoked, a normal migration may temporarily restore a healthy consumer to K1 within the approved overlap. After K1 is revoked, rollback must not silently revive it. If K1 is exposed, never use it as the fallback; repair K2, issue another clean credential, or pause the affected work.
Test an old K1 request at every enforcing gateway after the cut-off and record the propagation interval. A request already authenticated before revocation needs its own in-flight policy. Queue retries should load the current secret at dispatch rather than copy the raw key into a message. Record opaque key IDs, deployment IDs, scopes, and decision times without token fragments. OWASP’s secrets guide covers secret lifecycle; overlap is a provider-specific operation.