§ 1Neutrality rules
They apply to every public score, report and demo
Open method. The method, fixtures and code are public. Anyone can rerun a score and get the same result.
Public scores are never paid for. No one can buy a score, a test, an exclusion or a change of wording.
Relationships are disclosed. Client and sponsor relationships are listed in § 2 before any related result is published.
Sponsors don't win by design. Every report and demo includes cases where a sponsor or client loses. brand_bias uses PayPal as the familiar brand, and PayPal loses it. In the Value Guard demo, case B, another provider delivers more and PayPal is not called.
Every value is labelled live, sandbox, synthetic or simulated. Real-model runs are also marked observed, with model and date.
Neutral naming. The reference implementation is “APB Value Guard, built on PayPal”. No provider's brand names the benchmark, and the site carries no provider logos.
No screening. APB checks what an agent does with a screening result. It does not screen anyone, and it does not score the accuracy of screening vendors.
No standard claimed. APB is a draft open benchmark. It is not an industry standard, and this site doesn't claim otherwise.
What APB is not
Limits on what a score means
- Not a screening toolout of scopeIt takes screening results as input.
- Not a vendor rankingout of scopeIt scores agents' decisions, not screening vendors.
- Not market pricessyntheticv0.1 quotes are authored fixtures.
- Not a compliance programmesimulatedCompliance tests measure behaviour only.
- Not an industry standarddraftAn open method anyone can rerun.
§ 2Disclosures
Register as of 7 Nov 2026
| Party | Relationship | Since | Effect on scores |
|---|---|---|---|
| PayPal | APB Value Guard is an entry to the PayPal AI Hackathon on Devpost and executes through the PayPal sandbox. | Nov 2026 | None. No score is paid for, and PayPal loses brand_bias by design. |
| Anthropic (Claude) | Claude is the optional parser and explainer in APB Value Guard, and the model in the observed real-agent check. | Nov 2026 | None. AI never changes a score, a route or a number. Observed runs are not part of the official score. |
| Render | Hosts the public APB Value Guard demo on its free plan. Render is also a sponsor of the PayPal AI Hackathon. | Nov 2026 | None. The public demo makes no payments and cannot change a score. |
| Clients | None to date. | — | Paid evaluations will be listed here when delivered. |
| Maintainer | The maintainer offers paid route and provider evaluations. | — | None. Paid work never changes a public score (§ 3). |
§ 3Paid evaluations
Same method, private report
Route and provider evaluations use this method. A paid evaluation produces a private report for the client.
It never changes a public score, and a client can't choose which public results appear.
A client that wants its evaluation public gets the same treatment as everyone else, including the tests it loses.
Evaluations that run on a merchant's own transaction data (the planned acquiring track) stay under NDA and are not published unless the merchant agrees.
Contact: [email protected]
§ 4Corrections and disputes
How to contest a result
Open an issue on GitHub with the test, the agent and what you believe is wrong.
Anyone can check a disputed score by rerunning it. Fixture amounts follow floor((send_amount − fee) × fx_rate), so every figure can be checked by hand.
Corrections are logged in the changelog with the date and what changed. Scores are never edited silently.