Articles about "Claude-User"

Block ClaudeBot with two lines of robots.txt. Which of Anthropic's three crawlers to stop, copy-paste rules, verification steps, and trade-offs explained.

Anthropic runs three crawlers — ClaudeBot, Claude-SearchBot and Claude-User — and blocking the wrong one erases you from Claude's answers. What each does, what blocking costs, and the exact robots.txt setup for no-training-yes-citations.

Get cited by Claude without feeding its training data: allow Claude-SearchBot and Claude-User, block ClaudeBot, and write pages that are easy to quote.