How webhook callback delivery operates on Nigerian gateways
Nigerian payment processing relies heavily on asynchronous notifications. When a user completes a payment using card payment or USSD, gateways like Paystack or Flutterwave send a POST callback to your designated webhook endpoint. The payload contains details like the transaction reference and the transaction status.
These gateways require a successful HTTP status code response (usually a 200 OK) from your server within a short timeframe. If your server fails to respond quickly or returns a server error, the gateway retries delivery. Paystack retries up to 5 times with increasing delays, while Monnify and Flutterwave use similar retry queues. This retry mechanism provides reliable delivery during temporary network drops, but it also creates scenarios where multiple duplicate webhooks are delivered for a single transaction.
The database idempotency gap
An API endpoint is idempotent if processing the same request multiple times does not change the system state beyond the initial call. In many fintech systems, webhooks are processed by performing a sequence of verification steps:
- Verify the payload's cryptographic signature (using header values like
x-paystack-signature). - Check if the transaction status is marked as
success. - Check the database to verify if the transaction reference is already credited, and if not, update the customer's wallet balance.
The vulnerability lies in the gap between the database lookup (checking if the transaction reference was already processed) and the balance update (crediting the wallet). If two requests with the same reference arrive at the server concurrently, the database lookup for both requests runs in parallel. Because neither request has completed its write operation, both lookups see the transaction reference as unprocessed. Both requests then execute the update, resulting in the wallet being credited twice.
The concurrent replay attack path
Attackers exploit this race condition window deliberately. Rather than waiting for legitimate network retries, they simulate concurrency artificially to double or triple their wallet credits.
The attack follows a specific sequence:
- The attacker signs up for the fintech service and initiates a wallet funding request of ₦2,000.
- The attacker pays the ₦2,000 through the Paystack checkout modal and obtains the valid transaction reference.
- The attacker captures the webhook callback payload sent by Paystack to the fintech's server. This is done by monitoring their own account traffic or by extracting the payload from a public sandbox environment.
- The attacker writes a script that replays the captured webhook payload by sending 20 concurrent HTTP POST requests to the fintech's webhook endpoint in parallel.
- If the fintech's database check
SELECT * FROM transactions WHERE ref = ?and wallet updateUPDATE wallets SET balance = balance + 2000do not execute within an isolated atomic transaction, multiple parallel requests bypass the duplicate check. - The server credits the user's wallet multiple times. In this scenario, the attacker's wallet is credited with ₦40,000 for a single ₦2,000 payment.
Below is a basic bash script showing how concurrent requests are executed against the webhook endpoint:
# Replaying 20 concurrent webhook requests with the captured payload
for i in {1..20}; do
curl -X POST https://api.fintech.com.ng/v1/webhooks/paystack \
-H "Content-Type: application/json" \
-H "x-paystack-signature: d2f5e8...[captured_hash]" \
-d '{
"event": "charge.success",
"data": {
"reference": "ref_908310931_simpa",
"amount": 200000,
"status": "success"
}
}' &
done
wait
Why local network latency extends the exploit window
The viability of a race condition depends on the duration of the database transaction window. The slower the database query response, the wider the window for concurrent requests to land.
In high-performance infrastructure (such as AWS US-East), query execution times often measure under 2 milliseconds, making concurrent exploitation difficult to time. However, many Nigerian fintechs run their backend services on local VPS hosting providers or hybrid cloud architectures that connect to local databases with high network lag.
If the application server is hosted remotely while the database is on-premise, or if the database is running on a poorly configured instance, network round-trip times suffer. A simple query that takes 1 millisecond in a unified cloud zone can take 35 to 80 milliseconds across local routes. This lag stretches the race window, giving the attacker a wider opportunity to send concurrent requests that execute before the database commits the first balance update.
The signature verification false comfort
A common security misconception among backend engineers is that verifying the signature on the webhook prevents this class of attacks. It does not.
When Paystack signs a webhook, it calculates an HMAC-SHA512 hash using the merchant's secret key and the raw JSON request body. This signature guarantees that the payload is authentic and was not tampered with in transit.
However, during a concurrent replay attack, the attacker does not alter the payload or the signature header. They send the exact, unaltered JSON body alongside the valid cryptographic signature. The server's signature verification function will recalculate the HMAC hash and find that it matches the header. The check passes successfully on all 20 concurrent requests. Recalculating the signature validates data integrity, not request uniqueness.
The fix: Row-level locking and strict validation
To resolve the double-crediting issue, you must implement strict idempotency checks using database-level serialization on the transaction reference.
First, process the webhook within an isolated database transaction using row-level locking. In PostgreSQL, this is done by appending a FOR UPDATE clause to your select query. By querying the transaction reference with a lock, you block subsequent queries trying to read or modify that specific row until the current transaction commits or rolls back. Adding SKIP LOCKED prevents other concurrent queries from stalling.
Second, always validate the payment amount. Do not rely on the amount returned in the webhook payload. Compare the webhook payload amount with the amount recorded in your database when the payment reference was originally generated.
Here is the Node.js + PostgreSQL database transaction pattern using Knex.js:
await db.transaction(async (trx) => {
// Lock the reference row immediately
const tx = await trx('payment_refs')
.where({ ref: payload.reference })
.forUpdate()
.first();
if (!tx) {
throw new Error('Transaction reference not found');
}
// Reject if already credited
if (tx.credited) {
return { status: 'already_processed' };
}
// Compare payload amount to stored database amount
if (Number(payload.amount) !== Number(tx.amount)) {
throw new Error('Payload amount does not match stored amount');
}
// Credit user wallet balance
await trx('wallets')
.where({ userId: tx.userId })
.increment('balance', tx.amount);
// Mark reference as credited
await trx('payment_refs')
.where({ ref: payload.reference })
.update({ credited: true, processed_at: new Date() });
});
Third, if your application runs on a horizontally scaled cluster, database locks are the primary defense, but a distributed lock adds a clean layer of isolation. Use Redis to set a distributed lock on the transaction reference with a short time-to-live (TTL).
// Setting a distributed lock in Redis
const lockKey = 'lock:webhook:' + payload.reference;
const acquired = await redis.set(lockKey, 'processing', 'NX', 'EX', 10);
if (!acquired) {
return { status: 'rate_limited_or_locked' };
}
Concurrent Webhook Replay Double-Crediting
During a security review, we sent 15 concurrent POST requests with a valid captured webhook payload to a fintech's /webhooks/paystack endpoint. The endpoint returned a 200 OK status code on all 15 requests and credited the user wallet 15 times, leading to a total credit of ₦750,000 for a single ₦50,000 payment. The duplicate check was a SELECT query that ran before an UPDATE query, without row-level locking. Fix priority was set to critical.
Are your webhook endpoints ready to reject concurrent replay attempts?
Get a security reviewFrequently asked questions
Does Paystack's webhook retry mechanism cause real duplicates in practice?
Yes. Paystack retries webhook delivery on 5xx responses and timeouts. If your endpoint is slow (>30 seconds), Paystack may deliver the same webhook while you are still processing the first delivery. The duplicate is legitimate from Paystack's perspective.
Is this different from the signature replay protection Paystack mentions?
Yes. Paystack's documentation mentions verifying the signature and recommends idempotency, but the responsibility for idempotency implementation is entirely on the fintech side. Paystack does not prevent replay.
How do we test this ourselves before a review?
Store one valid webhook payload from your Paystack test environment. Write a script that fires 10 concurrent POST requests to your own /webhooks endpoint with that payload. Check your database: was the wallet credited once or multiple times?
Related reading
Blog: Webhook Security for Payment Platforms · Rate Limiting and Anti-Fraud Patterns for Payment APIs
Services: API security testing · Penetration testing · For payment gateways