Onpage Rocks User-Agent

Onpage Rocks


So have you been looking through your log files and notice that our user=agent has crawled your site.

Its either myself or The Server Log Kid – ok it’s probably him.

We don’t tend to crawl sites which aren’t our clients, so it’s unlikely for you to see it unless we have done some work in the past, but see below for a couple of reason we might be crawling you if you haven’t specifically reached out.

The Useragent

The UserAgent: “OnpageRocks https://onpage.rocks/user-agent”

Blocking Our User Agent

The bot is instructed to respect robots.txt file for:


So if you prefer we didn’t crawl your site, please update your robots file, but who can say no to the server log kid crawling your site.

We haven't employed you

A question I get a lot is that “we haven’t employed you why are you crawling our site”.

First of all, well done on analysing your server logs, very few companies actively monitor their logs so it’s great that you are.

Usually, it’s one of two reasons, we do a lot of what is known as “white label” auditing, whereby other agencies and freelancers hire me to do the work.

The other reason is some of bigger clients actually hire us to audit their competition so they can understand their weaknesses from a tech point of view to see where they have a competitive advantage.

Great articles from the Blog


Technical Seo Demystified: Martin Splitt

https://www.youtube.com/watch?v=X05ypQmLXc8Video can’t be loaded because JavaScript is disabled: Martin Splitt – Technical SEO Demystified | AngualrUP 2019 (https://www.youtube.com/watch?v=X05ypQmLXc8) This is a great talk from one

Subscribe to my newsletter

Every Month I give away on FREE technical Audit for just being on our mailing list. For Full Terms check out our Terms and Conditions page.We will never sell your email or SPAM you.

Subscribe to our mailing list

* indicates required