MetricRaven

What is SiteAuditBot?

Semrush · Other bots · User agent only

Semrush's Site Audit crawler, which checks a site for SEO and technical issues when a Semrush user audits it.

A visit usually means: A product is checking the page for a specific job: an ad review, a link preview, a feed or an SEO index.

SiteAuditBot user agent

User-Agent token
SiteAuditBot

Semrush documents the token but not a full string. Requests contain SiteAuditBot, usually with a version number such as “SiteAuditBot/1.0” and browser text around it. Match the token anywhere in the user agent, ignoring case.

Paste a string from your logs into the user agent parser to see which crawler it is. New to the term? Read what a user agent is.

Block or allow SiteAuditBot in robots.txt

Semrush says SiteAuditBot follows robots.txt.

Block everywhere
robots.txt
User-agent: SiteAuditBot
Disallow: /
Keep out of a folder
robots.txt
User-agent: SiteAuditBot
Disallow: /private/
Allow
robots.txt
User-agent: SiteAuditBot
Allow: /

A crawler follows the most specific group that names it, so a rule here overrides your User-agent: * group for this crawler. Well-behaved crawlers re-read robots.txt within about a day.

SiteAuditBot and Semrush's other crawlers

Semrush runs 2 crawlers in this directory, each with its own robots.txt token and its own job. What blocking each one costs you:

  • SiteAuditBot · OtherSemrush's SEO reports about your site
  • SemrushBot · OtherSemrush's SEO reports about your site

SiteAuditBot IP ranges and verification

Semrush publishes no IP ranges for SiteAuditBot, so requests are identified by user agent only and recorded as unverified. Anyone can send this user agent.

What SiteAuditBot means for your traffic and revenue

SiteAuditBot feeds Semrush's SEO database. It sends no visitors and adds server load. Blocking it removes your pages from that tool's reports, including competitors' research.

Most crawlers never run JavaScript, so browser analytics can't see them, and the ones that do shouldn't count as visitors. MetricRaven's tracker ignores SiteAuditBot, and server-side crawl tracking shows which pages it reads instead. See Bots & AI crawlers.

Questions

What is SiteAuditBot?
Semrush's Site Audit crawler, which checks a site for SEO and technical issues when a Semrush user audits it. MetricRaven groups it under Other bots: ad checks, link previews, feed readers, SEO tools and other bots.
What is the SiteAuditBot user agent?
Requests contain the token "SiteAuditBot", usually with a version number such as "SiteAuditBot/1.0". Match the token anywhere in the user agent, ignoring case.
How do I block SiteAuditBot in robots.txt?
Add "User-agent: SiteAuditBot" followed by "Disallow: /" to your robots.txt. Semrush says SiteAuditBot follows robots.txt.
Does SiteAuditBot publish IP ranges?
No. Semrush publishes no IP ranges for SiteAuditBot, so only the user agent identifies it, and anyone can send that string.
Is SiteAuditBot safe?
The real SiteAuditBot only requests public pages, and Semrush documents it. The risk is impostors using its name. Whether to allow it depends on what you get back: a product is checking the page for a specific job: an ad review, a link preview, a feed or an SEO index.
What other crawlers does Semrush run?
SemrushBot (SEO). Each has its own robots.txt token, and each does a different job.

See when AI assistants and crawlers read your site

Add one line to your server and MetricRaven shows which assistants, search engines and training crawlers request your pages, which ones are IP-verified, and the URLs they expected to find, including 404s.