Auditing Server Logs

Server logs contain a vast amount of data, data you can’t get from else where, while there is a 70% overlap with crawling your site for errors, the 30% are pure golden nuggets which you can’t get from any third party tool.
 
This data is only available to you, your competitors can’t this data. 
 
Why are auditing logs important?
 
There is a wealth of data that you are sat on, it just needs to be analysed and then more importantly actioned upon. 
 
We take all you server logs and analyse them to determine what Google is doing on your site and what they are NOT doing on your site. Knowing what they are not doing is sometimes just as important as what they are doing.
 
This can be super helpful if you have got an expired domain or purchased a website as you might know the full domain history so could have external links that Google are following that is landing on a 404 pages. 

Price

Our pricing is quite simple, we have two options, manual audits done by ourselves, we take your log files and analyse the data and produce a report. Or we have recently launched a self-serving option, whereby you upload the log files and our dashboard reporting gives you the data back.
 
The dashboard is a cheap option, starting from just $19.99 a month, but covers the most frequent issues. However every site is different so if you have any edge cases, any unusual issues they might be missed.

Testimonials

All our reviews are published on Google

Read More Reviews on Google

Frequently Asked Questions

Can’t I just get this data from Google Search Console?

This question comes up a lot when I am speaking at conferences, Google Search Console offers some “basic” crawl data – but the data isn’t 100% accurate and its not that useful. You can’t get any real analysis off it.

Should Audit logs be maintained?

We don’t recommend you keep you logs on the server over time they can slow down your server, especially for large websites as it can quickly take up a lot of storage space. We recommend taking the files off regularly and storing else where, we like to keep around 14 months worth of log data to do trend analysis, but for us to do a decent audit we like to get hold of at least the last three months worth of data?

How long should you keep log files?

On the server, as soon as feasible to remove them, they can become quite large as I mentioned above. In terms of storing them elsewhere, I have had clients with 10 years worth of data, but its not that useful after about 14 months. Trends analysis I wouldn’t really do with more than a year – its not going to lead to useful results.

How much data do you need to do an in-depth log file audit?

This is a hard question to answer, the key data is what is happening the most recent, what Google is doing now. 
 
We like to look historically to see trends, but a 404 link from 6 months ago which has since been fixed isn’t that much of an issue anymore.
 
For new audits, we like 3 months worth of data, but if not at least the last 30 days should be good enough to get the majority of issues.

How often show logs be reviewed?

I would reply with a question back, how often do you crawl your site looking for on page technical issues, the answer is the same. For our own sites and for clients we have on retainer we do them both together, if they have signed up for weekly audits, we audit the logs weekly. If they are daily or monthly clients we do the same.

What is the use of log files?

The log file is a file which stores every time someone (human) or something (bot) requests a page from your website. It stores quite basic information like date, response, size, referrer information and a bit more. Each day a new file is created with the data.

If you have any other questions around log file analysis either email me, or join our Facebook group and I will answer them for you.