Yes — you can block AI training crawlers on Blogger using its built-in Custom robots.txt setting: flip the switch under Settings > Crawlers and indexing, paste named-bot rules for GPTBot, ClaudeBot, Google-Extended and friends, and Blogger serves your file at yourblog.blogspot.com/robots.txt. The whole job takes about fifteen minutes, no plugin or code required. The catch is knowing Blogger's limits: you get robots.txt control, per-post robots tags, and not much else — no server firewall, no WAF rules, no IP blocking. This guide walks through the exact clicks, the copy-paste AI-bot rules, and what to do when robots.txt isn't enough.
What Blogger gives you (and what it doesn't)
Blogger is a fully managed platform, which is both the good news and the limitation. You don't get server access, so you can't install a firewall rule or edit server configuration. But Google built crawler controls directly into the dashboard, and for AI-crawler management those controls are enough to handle the honest bots — which, for training crawlers like GPTBot and ClaudeBot, covers the ones whose operators document robots.txt compliance.
What you get:
- Custom robots.txt — full control of the file Blogger serves at your domain's /robots.txt
- Custom robots header tags — per-section meta directives (all, noindex, nofollow, noarchive, nosnippet) for your homepage, archive/search pages, and posts
- Per-post robots settings — in the post editor's side panel
What you don't get: user-agent blocking at the server level, IP deny lists, Cloudflare-style bot management on the blogspot.com domain, or .htaccess-style rules. If a crawler ignores robots.txt, Blogger gives you no second lever on blogspot.com — the workarounds are below.
Step 1: Turn on Custom robots.txt
In your Blogger dashboard: Settings, then scroll to Crawlers and indexing. Find Custom robots.txt — it is off by default. Switch it on and Blogger opens a text box for your file. The default file Blogger serves (before you customize) blocks only internal search pages and includes your sitemap reference — it looks roughly like a wildcard group with Disallow: /search, Allow: /, plus a Sitemap: line. Open yourblog.blogspot.com/robots.txt in a browser first and read your own current file; whatever you paste should preserve its working parts.
One Blogger quirk worth knowing: the custom file applies per-blog, and it applies to the blog's canonical domain — the domain your blog actually serves. If you later move to a custom domain, check the robots.txt on that domain and re-verify everything.
Step 2: Paste your base file with AI-bot rules
Build your base file with our free robots.txt generator (select your rules, copy the output), or adapt this template — it keeps Blogger's required structure and adds the four biggest training crawlers:
User-agent: *
Disallow: /search
Allow: /
User-agent: GPTBot
Disallow: /
User-agent: ClaudeBot
Disallow: /
User-agent: Google-Extended
Disallow: /
User-agent: CCBot
Disallow: /
Sitemap: https://yourblog.blogspot.com/sitemap.xmlReplace the sitemap URL with your blog's real address (or your custom domain if you use one). Then click Save. What each part does:
- The
*group preserves Blogger's default behavior: internal search/label-search pages stay out of the crawl (they're thin duplicate content — blocking them is also an AdSense-friendliness move), everything else allowed, Googlebot included. - The four named groups opt out of the major AI training crawlers while leaving search and answer-engine retrieval untouched.
Google-Extendedonly governs Gemini training — your Google Search presence is unaffected. - If you also want to block Meta's training crawler, add
User-agent: meta-externalagentwithDisallow: /. Note that Meta's preview/link bot (facebookexternalhit) and its AI citation crawler are separate tokens — blocking the training agent doesn't kill your Facebook link previews. Our guide to blocking Meta's AI crawlers explains the split.
Do not add a blanket Disallow: / under User-agent: * — that's a whole-site block that would remove you from Google. And don't paste these examples into a blog post expecting them to work: they must go in Settings > Crawlers and indexing > Custom robots.txt. (Yes, people do this — there's a reason it's worth saying.)
Step 3: Set your robots header tags sensibly
Back in Settings > Crawlers and indexing, Enable custom robots header tags opens per-section controls. The safe setup most SEO guides recommend:
- Home page: all (let search engines index and follow normally)
- Archive and search pages: noindex (these are duplicate-content traps)
- Default for posts and pages: all
A classic Blogger foot-gun: someone sets posts to noindex while testing a theme and forgets to undo it — then wonders why nothing gets indexed. If your posts aren't appearing in Google, this setting is the first thing to check, before you blame robots.txt at all. Also check the per-post setting in the editor's side panel: a single post can carry "Custom robots tags" that override the defaults, so one accidental tick hides one post with no site-wide signal.
Step 4: Test the file before you trust it
Paste your final file into our robots.txt tester and check it the way crawlers parse it: every group, your homepage URL, a post URL, and your sitemap line. Our pre-publish testing guide walks through URL-by-URL checks — on Blogger this matters more than usual, because you can't tail a server log to notice a mistake; the tester is your only preview.
After saving in Blogger, open your live /robots.txt URL in an incognito window and confirm you see your rules. Then, in Google Search Console, verify the file fetches cleanly (Settings > robots.txt) and submit your sitemap if you haven't.
The limits: what robots.txt can't do on Blogger
Three honest limits, so you don't waste time looking for settings that don't exist:
1. No control over non-compliant crawlers. Robots.txt is honored by the documented training bots (GPTBot, ClaudeBot, Google-Extended, CCBot all say they comply) but ignored by scrapers that pretend to be browsers. On WordPress you'd add a WAF or server rule; on a blogspot.com domain you can't. If aggressive unknown bots are hammering your blog, your options are: move to a custom domain and put Cloudflare in front of it (Cloudflare's bot management and AI-crawler rules then apply — see our Cloudflare AI-bot blocking guide), or accept that you can only steer the honest ones.
2. No crawl-rate control. There's no crawl-delay equivalent in Blogger's settings, and Googlebot ignores Crawl-delay in robots.txt anyway. For Google, crawl rate is self-managed; for aggressive AI bots, robots.txt's Disallow is your throttle — an all-or-nothing one.
3. No llms.txt / ai.txt hosting control. You can't upload arbitrary files to a blogspot.com root, so you can't publish an llms.txt at your domain root the way a self-hosted site can. (Honest note: the latest log studies suggest AI crawlers almost never fetch llms.txt anyway — the data is here — so this is a smaller loss than it sounds.)
Blogger vs the other platforms
How does Blogger's control compare? WordPress gives you everything — plugins, server rules, full file control (our WordPress guide). Wix, Squarespace, and Webflow sit in the middle with partial robots.txt editors. Blogger is near WordPress's level for robots.txt specifically — full file control — but uniquely weak on everything below robots.txt: no server, no WAF, no logs. If your main concern is documented AI training bots, Blogger's robots.txt is genuinely sufficient. If you're fighting bad bots, Blogger is the wrong platform to fight from — and a custom domain plus Cloudflare changes the equation.
Monitoring: how to tell it's working
Blogger doesn't give you server logs, so you verify differently than a self-hosted site owner would. Three checks, all free:
- Search Console > Settings > robots.txt: confirms Google fetched your file and shows the last fetch time. If the fetch fails or shows an old version, your edit didn't take.
- Search Console crawl stats: over the following weeks, watch the "by response" breakdown. A healthy Blogger site shows mostly 200s; a sudden spike in "blocked by robots.txt" for pages you care about means a rule is too broad.
- Spot-check AI answers: ask ChatGPT or Claude about a topic your blog covers well and see whether your pages get cited. This is anecdotal, not scientific — citation depends on far more than crawl access — but a total disappearance after a robots.txt change is worth investigating.
One caveat that applies everywhere, not just Blogger: robots.txt tells compliant crawlers to stay out, but it doesn't make your content invisible. The file itself is public — anyone can read your /robots.txt and see which paths you consider sensitive. Never rely on robots.txt for anything that must actually stay private; on Blogger that means using the platform's post visibility settings (draft, or link-only sharing) for sensitive content.
FAQ
Will blocking GPTBot on Blogger hurt my Google rankings?
No. GPTBot is OpenAI's training crawler; it has nothing to do with Google Search rankings. Your Google visibility depends on Googlebot, which your wildcard group explicitly allows. The only ranking-adjacent risk is miswriting the file — e.g., a stray site-wide Disallow — which is why you test before saving.
Does blocking GPTBot remove me from ChatGPT answers?
Not automatically. ChatGPT citations come from a different crawler, OAI-SearchBot. Blocking GPTBot while allowing everything else keeps you eligible for citations. (Independent studies note some sites block both by accident — check your file for an OAI-SearchBot rule if citations matter to you.)
Can I block Perplexity on Blogger?
For its indexing crawler, yes: add User-agent: PerplexityBot with Disallow: /. But know that Perplexity's user-triggered fetcher (Perplexity-User) explicitly ignores robots.txt — it treats a user's question as the request. Robots.txt can't stop that on any platform, Blogger or otherwise.
My changes aren't showing in robots.txt — what went wrong?
Three usual causes: you edited the wrong blog (multi-blog accounts — check the dashboard switcher), you pasted into a blog post instead of the settings box, or you're viewing a cached copy (hard-refresh or check in incognito). The settings page must show the switch ON and your text in the box; the live /robots.txt URL is the source of truth.
Should I block AI crawlers at all, or leave everything open?
Depends on your goal. If you want maximum distribution — citations in ChatGPT, Perplexity, Gemini — leave training bots allowed; they're how models learn your content exists. If you'd rather not feed model training without compensation, block the training tokens and keep the retrieval bots open. Our pros-and-cons guide walks through the full decision.
Does Blogger's custom robots.txt work with a custom domain?
Yes — the setting lives on the blog, not the domain. After connecting a custom domain, open yourdomain.com/robots.txt and confirm your rules serve there, and re-verify the property in Search Console since the URL changed.
