What the benchmark measures
This method scores six controls in public payment API documentation. A present score means the written instructions meet a stated rule. It gives no proof of live enforcement. Version 1.0 includes a rule file and six worked rows for an invented provider. Each row has evidence and a reason, so two researchers can apply the same test.
Define the Nigerian study population
Set one cutoff date before collecting sources. Include each provider whose legal entity appears in a CBN financial institution list at that date and whose public product page offers a merchant payment API for use in Nigeria. Record the entity name, list URL, licence category, product URL, and inclusion decision. Check the API product against the listed entity; a shared brand name alone is insufficient. Record unclear entity matches as unresolved and exclude them from scored results until resolved.
Use one row set per distinct API product, with its legal entity recorded. Merge aliases that point to the same product and version. Publish excluded candidates with reasons. This is a defined sample of listed entities with public merchant APIs; it does not cover every Nigerian fintech or every payment service. Missing public docs do not remove an otherwise eligible product: its controls receive unclear scores with a missing_document reason.
Six rules with fixed IDs
- AUTH-01: Credential type, where it goes, and how to keep it secret.
- IDEM-01: Key placement, same-key replay result, changed-body handling, and retention window.
- HOOK-01: Signature header, algorithm or SDK verifier, secret, and exact body requirement.
- ERROR-01: Named errors with retry, stop, or status-query actions, including an unknown result.
- RATE-01: Limit scope, numeric threshold and window, and throttled response behavior.
- DISC-01: Vulnerability contact and covered products or domains.
The version 1.0 rule file gives the exact question and decision for every ID. Present requires every listed element. Absent requires an explicit public statement that the feature is unavailable under that rule. Missing, partial, conflicting, or inaccessible material means unclear. Use reason codes missing_document, partial_rule, conflicting_sources, or inaccessible_source. Save enough detail to distinguish these cases.
Work through one decision
In the synthetic IDEM-01 row, the invented guide puts the key in Idempotency-Key, returns the first result for the same key and body, rejects changed bodies, and retains keys for 24 hours. All four parts match the rule, so the row is present. If the retention sentence were removed, the result would be unclear with partial_rule. Nothing in that row tests a real request.
Stripe’s public guide provides a real example of detailed replay rules. It describes saving the first result, including errors, and comparing parameters on reuse. Treat it as a source example, not an eligible Nigerian provider under this study rule. For HOOK-01, Stripe’s verification guide shows why the raw request body matters. For DISC-01, RFC 9116 defines security.txt fields; this benchmark also requires covered scope.
Collect evidence and settle disagreements
Search the product’s API reference, linked help pages, security policy, and security.txt. Record the URLs searched, read date, API version, section, brief evidence note, and reviewer ID. Save a permitted snapshot and its SHA-256 hash. Keep missing-source rows in the dataset with the attempted URL and reason. The synthetic file uses example.invalid, which supplies no live evidence.
Two reviewers score each row independently before seeing the other score. Compare decisions, record the rule element at issue, then recheck the same saved sources. A third reviewer decides disputes against the published rule. If sources still conflict or leave a required part unstated, use unclear. Preserve both initial scores and the final reason in a review log. Provider corrections must point to a source available at the cutoff; later documents belong in a new release.
Report counts readers can check
Publish present, absent, and unclear counts for each rule with the full eligible-product denominator. For the six synthetic rows, the counts are three present, one absent, and two unclear. They describe an invented example only. Avoid a single security rank: missing public detail and a proven live defect are different findings. Release the population file, all scored rows, rules, review log, and correction history together. Change the rule version when a required element changes and do not compare versions without a mapping.
Download the worked synthetic rows and method notes. Fill one real row from a public source, then have a second reviewer apply the same rule. Read the API testing guide or request a security review.