xAI-Grok
xAI · Other bots · User agent only
About this crawler
Reported as a Grok fetcher from xAI.
Usually means: A product is checking the page for a specific job: an ad review, a link preview, a feed or an SEO index.
The operator doesn't document this user agent, so treat it as a claim: anyone can send it.
User agent
xAI-Grok
MetricRaven matches this token anywhere in the request's user agent, ignoring case. Operators add version numbers and browser text around it, for example "xAI-Grok/1.0".
IP verification
xAI publishes no IP ranges for xAI-Grok, so requests are identified by user agent only and recorded as unverified. Anyone can send this user agent.
Control it in robots.txt
User-agent: xAI-Grok Disallow: /private/
Well-behaved crawlers re-read robots.txt within a day or so.
Questions
- What is xAI-Grok?
- Reported as a Grok fetcher from xAI. MetricRaven groups it under Other bots: ad checks, link previews, feed readers, SEO tools and other bots.
- How do I verify that a request really comes from xAI-Grok?
- xAI publishes no IP ranges for xAI-Grok, so only the user agent identifies it. Anyone can send that string, so treat it as a claim.
- How do I block or allow xAI-Grok?
- Add a robots.txt group for "xAI-Grok" with the paths to allow or disallow. Fetchers that act for a user may ignore robots.txt; block those at your server or firewall instead.
- How can I see when xAI-Grok visits my website?
- Most crawlers never run JavaScript, so browser analytics miss them. MetricRaven's @metricraven/ai-crawl package reports crawler requests from your server, verifies their IPs against published ranges, and shows which pages each crawler reads.
See when AI assistants and crawlers read your site
Add one line to your server and MetricRaven shows which assistants, search engines and training crawlers request your pages, which ones are IP-verified, and the URLs they expected to find, including 404s.