Bot Probe
See if advertising crawlers can actually download your ads.txt. We request the file with several User-Agents (browsers, Googlebot-style, SSP-style) and report who gets blocked.
Cet outil est en bêta — les résultats peuvent évoluer au fil des améliorations.
We hit the same URL with multiple User-Agents and summarize HTTP status + robots.txt hints. Use this when an SSP says they “cannot see” your ads.txt.
How this tool works
- 1 Enter the publisher domain or a direct ads.txt URL.
- 2 We request the file with several User-Agents (browser, Googlebot-like, SSP-like).
- 3 For each agent we report success/failure and, when available, robots.txt guidance.
- 4 If Google or an SSP is blocked here, they will not authorize inventory from your site — fix hosting/WAF/robots before chasing yield.
FAQ
How do Google and SSPs find my ads.txt? +
Crawlers request https://<your-root-domain>/ads.txt (and sometimes app-ads.txt). They use their own User-Agents, respect or ignore robots.txt depending on policy, and cache the file for hours or days. Subdomains usually inherit the root ads.txt unless you publish a separate file or use subdomain= directives.
Why can I open ads.txt in Chrome but an SSP says they cannot see it? +
Your browser is allowlisted; bot IPs or User-Agents may be blocked by Cloudflare, WAF, geo rules, or robots.txt. Bot Probe repeats the same URL with several UAs so you can see who gets HTTP 200 vs 403/Challenge.
Where should ads.txt live? +
At the domain root over HTTPS: https://example.com/ads.txt — not /blog/ads.txt, not only on www if inventory runs on the apex (or vice versa). Redirects are usually followed, but fragile CDN rules often break bot fetches. Prefer a stable 200 text/plain response.
Does robots.txt block ads.txt crawlers? +
It can. Disallowing /* or disallowing ads.txt for Googlebot / AdsBot / partner bots prevents authorization even if humans can download the file. Allow ads.txt explicitly for major crawlers, or keep it outside broad Disallow rules.
How often do bots re-crawl ads.txt? +
Typically from a few times a day to every few days, depending on the buyer. After you publish a fix, wait for re-crawl before judging revenue impact. Some platforms let you force a refresh in their UI; otherwise Bot Probe only proves reachability now, not their cache state.
What is Bot Probe? +
A diagnostic that GETs your ads.txt with multiple User-Agents (browser, Googlebot-like, SSP-like) and reports success/failure. It does not modify your file. Use it whenever a partner claims they “cannot see” your ads.txt.
Which bots should succeed? +
At minimum a normal browser UA and a Googlebot-like UA should receive HTTP 200 with a text body. If only browsers work, expect delayed or missing ads.txt authorization in GAM and SSPs — a direct SPO / demand problem.
app-ads.txt vs ads.txt — do bots need both? +
Web inventory uses ads.txt; mobile app inventory uses app-ads.txt at the developer domain declared in the store listing. Bot Probe can target either URL. Do not confuse them: fixing only the web file will not authorize in-app demand.