Who crawls your site

If you found a visitor called SEOZion in your server logs, that’s the crawler SEO Zion uses to run website checks. Someone asked us to check your site, usually the site’s owner or someone working for them. Older versions called it SenseiSEO; it’s the same crawler.

How to identify it

It sends this User-Agent header, starting with the request for your robots.txt:

Mozilla/5.0 (compatible; SEOZion/0.1; +https://seozion.com/who-crawls-your-site)

Look for SEOZion/ in the User-Agent column of your access logs. The version number after the slash may change; the name won’t.

If your server refuses a page (a 403 or a firewall or bot-challenge page), the crawler stops there and reports the page as blocked. It never retries under a different name.

What it does

• Reads public pages only. It doesn’t log in, submit forms, or solve CAPTCHAs or bot challenges.
• Reads your robots.txt first and skips anything it disallows. It follows a group for SEOZion or the older SenseiSEO name if you have one (both, if you have both), and otherwise your User-agent: * group.
• Waits at least 0.4 seconds between requests to your site, longer if your Crawl-delay asks for it, and makes no more than 3 requests at once.
• Stops after a set number of pages per check (200 by default).

How to limit or block it

Add a group for SEOZion to your robots.txt. To block it completely:

User-agent: SEOZion
Disallow: /

To slow it down instead:

User-agent: SEOZion
Crawl-delay: 5

Like other crawlers, it follows only the most specific group that names it. Once you add a SEOZion group, it stops reading your User-agent: * group, so copy across any Disallow lines you still want it to obey. An existing User-agent: SenseiSEO group keeps working.

Questions

If you think the crawler is misbehaving on your site, a contact address for crawler reports will be published here before launch. Read about how we handle data in our privacy policy.