MetricRaven

What is ShapBot?

Parallel · Search indexes · User agent + IP range

Parallel's crawler: it discovers and indexes sites for Parallel's search, extraction and research APIs for AI agents.

A visit usually means: The page is being discovered or refreshed so it can appear in search or AI search results.

ShapBot user agent

The full user agent string, as Parallel publishes it:

  • User-Agent
    Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; ShapBot/0.1.0

Version numbers change, so match the token ShapBot anywhere in the user agent, ignoring case. Source: Parallel's crawler documentation, checked Nov 18, 2026.

Reading the string

  • The published version is ShapBot/0.1.0. Expect the number to change.
  • It starts with Mozilla/5.0, like a browser, so a filter that only checks the start of the string will miss it.

Paste a string from your logs into the user agent parser to see which crawler it is. New to the term? Read what a user agent is.

Block or allow ShapBot in robots.txt

Parallel says ShapBot follows robots.txt.

Block everywhere
robots.txt
User-agent: ShapBot
Disallow: /
Keep out of a folder
robots.txt
User-agent: ShapBot
Disallow: /private/
Allow
robots.txt
User-agent: ShapBot
Allow: /

A crawler follows the most specific group that names it, so a rule here overrides your User-agent: * group for this crawler. Well-behaved crawlers re-read robots.txt within about a day.

ShapBot IP ranges and verification

Verified when the user agent and the IP range both match

  1. 1User agent

    The request's user agent contains ShapBot.

  2. 2Request IP

    Your server-side tracker sends the IP that made the request.

  3. 3Range check

    The IP is compared with the ranges Parallel publishes, refreshed twice a day.

Range sources

Signed requests (Web Bot Auth)

Parallel also signs ShapBot's requests and publishes the keys at https://shapbot.parallel.ai. MetricRaven records a request with a valid signature from that key directory as verified, even from an IP outside the published ranges.

What ShapBot means for your traffic and revenue

ShapBot builds the index Parallel draws on for AI search answers. Blocking it can keep your pages out of those answers and the clicks they send.

Most crawlers never run JavaScript, so browser analytics can't see them, and the ones that do shouldn't count as visitors. MetricRaven's tracker ignores ShapBot, and server-side crawl tracking shows which pages it reads instead. See Bots & AI crawlers.

Questions

What is ShapBot?
Parallel's crawler: it discovers and indexes sites for Parallel's search, extraction and research APIs for AI agents. MetricRaven groups it under Search indexes: search engines and AI search keeping their index fresh.
What is the ShapBot user agent?
Parallel documents this user agent: "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; ShapBot/0.1.0". Match the token "ShapBot" rather than the whole string, because version numbers change.
How do I block ShapBot in robots.txt?
Add "User-agent: ShapBot" followed by "Disallow: /" to your robots.txt. Parallel says ShapBot follows robots.txt.
Does ShapBot publish IP ranges?
Yes. Parallel publishes the addresses it sends ShapBot from: https://docs.parallel.ai/resources/shapbot.json. A request that claims to be ShapBot from any other IP is an impostor.
Is ShapBot safe?
The real ShapBot only requests public pages, and Parallel documents it. The risk is impostors using its name, which the published IP ranges catch. Whether to allow it depends on what you get back: the page is being discovered or refreshed so it can appear in search or AI search results.

See when AI assistants and crawlers read your site

Add one line to your server and MetricRaven shows which assistants, search engines and training crawlers request your pages, which ones are IP-verified, and the URLs they expected to find, including 404s.