Articles about "block ClaudeBot"

Block ClaudeBot with two lines of robots.txt. Which of Anthropic's three crawlers to stop, copy-paste rules, verification steps, and trade-offs explained.

Anthropic runs three crawlers — ClaudeBot, Claude-SearchBot and Claude-User — and blocking the wrong one erases you from Claude's answers. What each does, what blocking costs, and the exact robots.txt setup for no-training-yes-citations.