Amazonbot: what Amazon crawls your site for, and how to check

Amazonbot is Amazon's web crawler. According to Amazon's documentation, it crawls pages to improve Amazon products and services, and the content it collects "may be used to train Amazon AI models". It honors robots.txt allow and disallow rules but does not support Crawl-delay. You can see its visits in Clicktag on the Crawlers tab by connecting a read-only Cloudflare token.

What is Amazonbot, and how does it differ from Amzn-SearchBot and Amzn-User?

Amazon documents three separate user agents on its Amazonbot page. Each one serves a different function and handles robots.txt differently.

When you decide how to handle Amazon's crawlers, separating the three tokens lets you block AI model training while keeping your pages available when someone asks Alexa a question. You can compare how other companies split their bots on our AI crawler list.

How to block Amazonbot in robots.txt

Because Amazonbot honors standard robots.txt directives, you can stop it from crawling your site by disallowing Amazonbot while leaving Amzn-SearchBot and Amzn-User open. Amazon notes that changes to your robots.txt settings may take about 24 hours to take effect.

Put these lines in the /robots.txt file at the root of your domain:

User-agent: Amazonbot
Disallow: /

User-agent: Amzn-SearchBot
Allow: /

User-agent: Amzn-User
Allow: /

If you want to block all three bots from a specific directory, such as /private/, list them together above the same Disallow line. Don't rely on Crawl-delay for Amazonbot, since Amazon does not support it. For rules covering OpenAI, Anthropic and Google, see our guide to block AI crawlers.

Blocking or managing Amazonbot in Cloudflare

If your domain is proxied through Cloudflare, you can also control Amazonbot from your Cloudflare dashboard without editing a file by hand:

• Managed robots.txt: Available on all Cloudflare plans. When Bot Preference Sync is turned on and your Training, Search or Agent policy is set to block, Cloudflare prepends managed directives (including an Amazonbot rule) and a Content Signals Policy with the default signal search=yes, ai-train=no to your /robots.txt.
• AI Crawl Control: Available on all plans. It monitors AI crawler activity and lets you set allow or block rules per crawler.
• AI bot policies: Cloudflare's older one-click "Block AI bots" setting is deprecating on September 15, 2026. Its replacement, Security Settings > Configure AI bot policies, has Search, Agent and Training presets, each set to Block, Block on pages with ads, or Allow. From that date, new domains default to blocking Training and Agent bots on pages that display ads, while allowing Search bots.

Crawlers follow robots.txt voluntarily, so check your crawler hits after you add a disallow rule to confirm a bot actually stopped fetching your pages.

Why normal web analytics miss Amazonbot

Client-side trackers, including Clicktag's own latest.js script (about 4 KB), run in a visitor's browser, and Clicktag excludes bots from visits and pageviews by design. AI crawlers like GPTBot and ClaudeBot fetch raw HTML and never run JavaScript, so the tracking script never fires for them at all.

To see crawler traffic, you need to read requests at the edge or server level. Clicktag provides AI crawler analytics by reading request data from Cloudflare through a read-only API token.

How to check Amazonbot hits on your site in Clicktag

If your site is verified in Clicktag and its domain is proxied through Cloudflare (any Cloudflare plan works, including Free), you can connect it in three steps. There is no code to add to your site.

1. Open your site's settings in Clicktag, go to Integrations, and find Cloudflare.
2. Click Create a read-only token. Cloudflare opens with Zone Read and Analytics Read pre-selected for your zones. Create the token and copy it.
3. Paste the token into Paste the token and click Connect.

Clicktag imports the last 30 days of crawler history on connect, then runs a sync every 15 minutes. While Cloudflare drops this analytics data after 31 days, Clicktag stores it per hour so your history keeps growing over time. You can read the full technical details in the AI crawlers docs.

What you see on the Crawlers tab

Once the initial import finishes, open the Crawlers tab in your dashboard. Amazonbot is one of 22 bots Clicktag recognizes by name, typed as AI training, and it appears in the Bots panel once it has verified hits.

For Amazonbot and every other recognized bot, the report shows:

• Verified hits: Requests that Cloudflare verified in its AI Crawler, AI Search, AI Assistant or Search Engine Crawler category. Clicktag also checks published IP lists for OpenAI, Perplexity, Google, Microsoft, Apple, Common Crawl and DuckDuckGo during the sync (IP addresses are never stored). For Amazonbot, a hit counts as verified when Cloudflare verifies it.
• Unverified hits: Requests that put Amazonbot in the user agent string without passing verification. Keeping these separate stops spoofed scrapers from polluting your numbers.
• Blocked count: Verified requests your site answered with an HTTP 401, 403 or 429 status code, so you can see right away if a Cloudflare rule is turning Amazonbot away.
• Pages: The top 100 paths on your hostname (and its www twin) that crawlers fetched, including /robots.txt and /llms.txt. Static assets like scripts, styles, fonts and images are excluded.

The live demo shows Clicktag's visitor reports with synthetic sample data. Clicktag shows crawler activity and status codes; it does not block bots itself.

FAQ

Does Amazonbot train AI models?

Yes. Amazon states in its developer documentation that data crawled by Amazonbot improves Amazon products and services and "may be used to train Amazon AI models". Its other two bots, Amzn-SearchBot and Amzn-User, are used for search experiences and live user actions like Alexa answers, and are not used for generative AI training.

Does Amazonbot respect Crawl-delay in robots.txt?

No. Amazonbot honors standard Allow and Disallow rules in robots.txt, and updates take roughly 24 hours to apply, but it does not support the Crawl-delay directive. Also note that Amzn-User, which fetches pages for live user requests, may not follow all robots.txt directives.

Can I block Amazonbot with Clicktag?

No, Clicktag is an analytics tool and does not block traffic or modify your responses. You block Amazonbot by adding a Disallow rule in your /robots.txt file or by turning on a block rule in Cloudflare's AI Crawl Control, and then use Clicktag's Crawlers tab to verify whether hits drop or start returning 403 status codes.

Do I need a paid Cloudflare or Clicktag plan to track Amazonbot?

No. Clicktag is free with no credit card required, and you can track up to 50 websites per account. The Cloudflare integration works on all Cloudflare plans, including the Cloudflare Free plan, as long as your domain is proxied through Cloudflare and verified in Clicktag.

Does Clicktag recognize Amzn-SearchBot and Amzn-User separately?

Not by name. Clicktag's built-in bot table names Amazonbot under the AI training category. When Cloudflare verifies other bots that aren't in Clicktag's main list of 22 bots, they still appear on your Crawlers tab under the name in their user agent string, labeled with the type of their Cloudflare category.