Can you see Google bot activity in the logs on sitemap pages? Episode 1

This is the first video in our new series, with me Andy and Corey Aka The Server Log Kid.


Today’s Question is: Can you see Google bot activity in the logs on Sitemap pages?


That is a brilliant question Corey and the answer is yes, if you look in the server logs you can see Googlebot activity on the different sitemap pages.


You can see:

  • When they last crawled
  • How many times have they been crawled
  • Which bot it was, whether that’s mobile or desktop bot


Obviously, you don’t want to rely upon Googlebot crawling the sitemap to find new pages, you want to use internal linking and external linking but you can look into the logs to see when Googlebot last hit your sitemap pages.

If you have any questions to do with Googlebot crawling sitemap pages ask in the questions below and we will answer them for you, if you have a question you would like the Server Log Kid to answer, please email them to us or comment below and it could be your question we answer next.

Why not check out the rest of the series in Server Log Kid to see some of the other questions we have answered.


Share on facebook
Share on twitter
Share on pinterest
Share on linkedin

Leave a Comment

Your email address will not be published. Required fields are marked *

On Key

More Great Articles

Google bot user agent

Google Bot User Agents

Unsurprisingly, as they never do anything by half, there are quite a few different Google user agents. But, just how many there are might surprise

Read More »
SEMRush Bot

SEMRush Bot

SEMRush is a tool which you can use to crawl your website to analyse to find technical issues with what’s wrong on the website. SEMRush

Read More »

Subscribe to my newsletter

Want to learn more about technical SEO and Server Log Analysis, sign up below and I will teach you.

Each week I will send you the most helpful articles delivered straight into your inbox as well as sharing some very useful tips and tricks to improve your technical SEO skills to Ninja level.

* indicates required