MetricRaven

What is ev-crawler?

Headline · Other bots · User agent only

ev-crawler is Headline's crawler. Headline, a venture capital firm, reads public sites with it to follow what's being built and find founders.

A visit usually means: A product is checking the page for a specific job: an ad review, a link preview, a feed or an SEO index.

ev-crawler user agent

The full user agent string, as Headline publishes it:

  • User-Agent
    Mozilla/5.0 (compatible; ev-crawler/1.0; +https://headline.com/web-research)

Version numbers change, so match the token ev-crawler anywhere in the user agent, ignoring case. Source: Headline's crawler documentation, checked Nov 18, 2026.

Reading the string

  • The published version is ev-crawler/1.0. Expect the number to change.
  • It starts with Mozilla/5.0, like a browser, so a filter that only checks the start of the string will miss it.
  • It carries a link, https://headline.com/web-research, so anyone reading their logs can look the crawler up.

Paste a string from your logs into the user agent parser to see which crawler it is. New to the term? Read what a user agent is.

Block or allow ev-crawler in robots.txt

Headline says ev-crawler follows robots.txt.

Block everywhere
robots.txt
User-agent: ev-crawler
Disallow: /
Keep out of a folder
robots.txt
User-agent: ev-crawler
Disallow: /private/
Allow
robots.txt
User-agent: ev-crawler
Allow: /

A crawler follows the most specific group that names it, so a rule here overrides your User-agent: * group for this crawler. Well-behaved crawlers re-read robots.txt within about a day.

ev-crawler IP ranges and verification

Headline publishes no IP ranges for ev-crawler, so requests are identified by user agent only and recorded as unverified. Anyone can send this user agent.

What ev-crawler means for your traffic and revenue

ev-crawler checks pages for one product job, such as an ad review or a feed. It isn't a visitor and shouldn't be counted as one.

Most crawlers never run JavaScript, so browser analytics can't see them, and the ones that do shouldn't count as visitors. MetricRaven's tracker ignores ev-crawler, and server-side crawl tracking shows which pages it reads instead. See Bots & AI crawlers.

Questions

What is ev-crawler?
Headline, a venture capital firm, reads public sites with it to follow what's being built and find founders. MetricRaven groups it under Other bots: ad checks, link previews, feed readers, SEO tools and other bots.
What is the ev-crawler user agent?
Headline documents this user agent: "Mozilla/5.0 (compatible; ev-crawler/1.0; +https://headline.com/web-research)". Match the token "ev-crawler" rather than the whole string, because version numbers change.
How do I block ev-crawler in robots.txt?
Add "User-agent: ev-crawler" followed by "Disallow: /" to your robots.txt. Headline says ev-crawler follows robots.txt.
Does ev-crawler publish IP ranges?
No. Headline publishes no IP ranges for ev-crawler, so only the user agent identifies it, and anyone can send that string.
Is ev-crawler safe?
The real ev-crawler only requests public pages, and Headline documents it. The risk is impostors using its name. Whether to allow it depends on what you get back: a product is checking the page for a specific job: an ad review, a link preview, a feed or an SEO index.

See when AI assistants and crawlers read your site

Add one line to your server and MetricRaven shows which assistants, search engines and training crawlers request your pages, which ones are IP-verified, and the URLs they expected to find, including 404s.