Evaluate a click fraud detector by the activity it can identify, the legitimate users it can misclassify and the evidence behind its actions. A sensor list, an AI label or a large blocked-traffic total cannot answer those questions alone.
The same evaluation applies to standalone software and protection included with an agency service. Bundling may simplify operations. It does not establish technical accuracy or financial benefit, and neither does charging separately.
Start with the claim you want to test
Detecting basic automation, recognizing sophisticated browser automation, identifying suspicious paid clicks and preventing spend are different claims.
For example, the WebDriver standard defines a browser property for reporting automation control. Recognizing that signal can be useful. A test consisting only of cooperative automation cannot establish coverage against software trying to hide its automation.
Likewise, finding an automated scraper on an unpaid visit does not establish click fraud. Connect the detection evidence to the paid-click cohort before assigning commercial meaning.
Use two deliberately different test populations
Run controlled tests on your own fixture pages or staging site, without generating test clicks on live advertisements. Record software versions, settings, network conditions and expected labels.
| Test population | What to include | What to measure |
|---|---|---|
| Known legitimate use | Manual desktop and mobile sessions, returning visits, privacy tools, shared networks and relevant in-app browsers | False flags, collection failures and access or measurement impact |
| Known automation | Basic request clients, ordinary browser automation and more realistic automation appropriate to the claimed coverage | Detection rate by test family, missed cases and unavailable evidence |
Do not collapse these into one accuracy percentage. A detector can catch every simple bot while flagging too many real users, or look accurate because the test contains very little difficult automation.
A controlled suite measures performance on those cases. It does not establish the prevalence of automation in a customer's campaigns.
Report missing evidence explicitly
A browser probe can pass, fail, be unsupported or never arrive. Those states need separate counts. If blocking or script-loading decisions reduce observation, the unseen follow-up cannot be treated as evidence that the visitors were bad.
Publish the denominator for each test family, collection coverage and uncertainty around the measured rates. Keep a held-out set outside threshold tuning, and rerun relevant cases when sensors or collection code change.
Button's browser-consistency research illustrates why a narrow signal needs its own eligibility conditions and interpretation. Our proxy investigation also records a collection limitation that affected what could be concluded.
Ask to inspect one decision
A useful explanation identifies the observed evidence, classifier version, competing explanations, unresolved collection, linked paid click and subsequent action.
The provider need not reveal every detection secret publicly. It should still provide enough detail for an authorized customer to understand why a decision was made and how it can be challenged. Multiple indicators derived from the same underlying property should not be presented as independent confirmations.
Evaluate operations and business impact separately
After offline testing, review action scope, update failures, expiry, reversal and audit history. A technically correct detection can still trigger an overly broad response.
Then examine real campaign outcomes over a declared horizon. Use independent lead or order assessment, check attribution loss, and preserve negative or inconclusive results. Before-and-after improvement alone cannot isolate the detector from changes in bids, demand or the website.
For a bundled service, also ask whether you can export the evidence and action history, see which judgments were automated or made by an account manager, and retain your records if you change providers.
Use our guide to bot-click evidence for the acquisition question, our IP-exclusion guide for action limits, and our savings-report guide for financial claims. These are standards readers should apply to Button too.
AI disclosure
ChatGPT (AI) drafted this evaluation guide using the linked technical reference and existing Button research. Prepared for editorial review by Nathan Pankowsky. This is a proposed evaluation method, not a completed benchmark or a claim of superiority over another product.