What is ShapBot?
Parallel · Search indexes · User agent + IP range
Parallel's crawler: it discovers and indexes sites for Parallel's search, extraction and research APIs for AI agents.
A visit usually means: The page is being discovered or refreshed so it can appear in search or AI search results.
ShapBot user agent
The full user agent string, as Parallel publishes it:
- User-Agent
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; ShapBot/0.1.0
Version numbers change, so match the token ShapBot anywhere in the user agent, ignoring case. Source: Parallel's crawler documentation, checked Nov 18, 2026.
Reading the string
- The published version is ShapBot/0.1.0. Expect the number to change.
- It starts with Mozilla/5.0, like a browser, so a filter that only checks the start of the string will miss it.
Paste a string from your logs into the user agent parser to see which crawler it is. New to the term? Read what a user agent is.
Block or allow ShapBot in robots.txt
Parallel says ShapBot follows robots.txt.
User-agent: ShapBot
Disallow: /User-agent: ShapBot
Disallow: /private/User-agent: ShapBot
Allow: /A crawler follows the most specific group that names it, so a rule here overrides your User-agent: * group for this crawler. Well-behaved crawlers re-read robots.txt within about a day.
ShapBot IP ranges and verification
Verified when the user agent and the IP range both match
1User agent
The request's user agent contains
ShapBot.2Request IP
Your server-side tracker sends the IP that made the request.
3Range check
The IP is compared with the ranges Parallel publishes, refreshed twice a day.
Range sources
ShapBot IP addresses
Signed requests (Web Bot Auth)
Parallel also signs ShapBot's requests and publishes the keys at https://shapbot.parallel.ai. MetricRaven records a request with a valid signature from that key directory as verified, even from an IP outside the published ranges.
What ShapBot means for your traffic and revenue
ShapBot builds the index Parallel draws on for AI search answers. Blocking it can keep your pages out of those answers and the clicks they send.
Most crawlers never run JavaScript, so browser analytics can't see them, and the ones that do shouldn't count as visitors. MetricRaven's tracker ignores ShapBot, and server-side crawl tracking shows which pages it reads instead. See Bots & AI crawlers.
Questions
- What is ShapBot?
- Parallel's crawler: it discovers and indexes sites for Parallel's search, extraction and research APIs for AI agents. MetricRaven groups it under Search indexes: search engines and AI search keeping their index fresh.
- What is the ShapBot user agent?
- Parallel documents this user agent: "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; ShapBot/0.1.0". Match the token "ShapBot" rather than the whole string, because version numbers change.
- How do I block ShapBot in robots.txt?
- Add "User-agent: ShapBot" followed by "Disallow: /" to your robots.txt. Parallel says ShapBot follows robots.txt.
- Does ShapBot publish IP ranges?
- Yes. Parallel publishes the addresses it sends ShapBot from: https://docs.parallel.ai/resources/shapbot.json. A request that claims to be ShapBot from any other IP is an impostor.
- Is ShapBot safe?
- The real ShapBot only requests public pages, and Parallel documents it. The risk is impostors using its name, which the published IP ranges catch. Whether to allow it depends on what you get back: the page is being discovered or refreshed so it can appear in search or AI search results.
See when AI assistants and crawlers read your site
Add one line to your server and MetricRaven shows which assistants, search engines and training crawlers request your pages, which ones are IP-verified, and the URLs they expected to find, including 404s.