Before I get started, let’s be clear – content and links are still super important ranking factors, but there’s something even more important, that no one seems to talk about! What is it? Response codes! Yes, that’s right. When Google comes to crawl your website, they may get a 5xx response code. If they try […]
One of the biggest ranking factors that no one talks about Read More »
Google are in such a position of power that it is hard to imagine them ever being seriously challenged by a competitor. But being one of the world’s most valuable brands (fourth behind Amazon, Apple and Microsoft at the last count) doesn’t come without its problems. That includes facing up to the prospect of a
Could Apple be Google’s unlikely saviour from the US government? Read More »
Unsurprisingly, as they never do anything by half, there are quite a few different Google user agents. But, just how many there are might surprise you. I have been analysing server logs since around 2013, and, because I like to keep track, I have kept a record of all the different User-Agents I have come
Google Bot User Agents Read More »
We all know that robots were created to make life easier (or to make us lazier!), so it stands to reason that the robots.txt file on your website is meant to make someone’s job easier, and that someone is Googlebot. Getting your robots.txt file right, however, can also make your job easier, if you’re looking
What Is A Robots.txt And Why Does It Matter? Read More »
Have you been analysing your Server logs and realised you are unsure about the user agent and what it means. You can try and figure it out, but sometimes it’s not that easy or if you’re doing it on bulk this can be a wasted effort. Here, I have done all the hard work –
While Google didn’t use that exact phrase, that exactly how I imagined them saying it, especially in a British accent. Bet you just tried it, didn’t you? John Muller was recently asked on twitter why they had only indexed 30,000 pages of a site and how long would it take to index the rest of
Sorry, Sir, we’re not going to index all your pages Read More »
Whether you’ve just completed a technical audit on a site, or have spent several hours on a server log audit, one of the least talked about, but most important things to consider, is getting the developer or business owner to buy into the changes you want to make – especially if it’s going to cost
How To Get Developers On Your Side? Read More »
Understanding Googlebot and making it work for you Link building is vital for increasing brand awareness. For those that aren’t in the know, link building consists of actions you take that aim to build and increase the number of inbound links to your webpage to increase your rankings within search engines. So now you’ve got
Why You Need To Stop Link Building Read More »
How Google Changes To Link Attributes Will Affect SEO – And Why You Should Care Last September,Google made a big announcement, 14 years after introducing the nofollow link attribute, they are changing how they treat this instruction as well as adding a few new attributes – cue more stress – and more research by pretty
How Google Changes To Link Attributes Will Affect SEO – And Why You Should Care Read More »
Crawl Budget – What on EARTH is it, and How Can You Make It Work for you? Modern Search engines discover new content by crawling the web. But, pre-Google, most search engines were just basically a directory list of websites added by webmasters – simple enough, eh? Well, they were for a while, but for
Crawl Budget – What on EARTH is it, and How Can You Make It Work for you? Read More »
There are quite a few Server Log tools on the market and that can only be a good thing as more and more people realise the importance of the data that is held in the log files. I have decided to review the top 5 server log file tools to see which is best. So
Top 5 Best Server Log Analysing Tools Read More »
Some of the topics I cover on here are about advance server log analysis, but I often get asked what are some of the basic / initial things to look for in the server logs. https://www.youtube.com/watch?v=aphB6TcObgwVideo can’t be loaded because JavaScript is disabled: Some of the basic elements you should be looking out for in
Basic elements you should be looking out for in server logs Read More »
Something I get asked often is if I have Google Analytics what is the benefits of doing server log analysis? In this short video I try and explain the benefits of them both and where they are both useful.
What is the difference Server logs and Analytics Software Read More »
Back when I spoke at BrightonSEO I said after my talk when a question was asked, can you see if your site was affected by looking in the server logs. My answer at the time was the two sites I had data at that time that I had analysed had seen a spike in Googlebot
Do you see a spike in Google Activity before an update? Read More »
At a recent Google IO talk, there were some key topics which came up which I found were useful and wanted to share. The videos are below. The key things is Google bot is using the latest version of Chrome for Rendering it was on Chrome 41 over 3 years old, which is great news
Errors in Search Console Read More »
SEMRush is a tool which you can use to crawl your website to analyse to find technical issues with what’s wrong on the website. SEMRush Crawler Also, SEMRush offers a domain metric called “Authority Score” which analyses a websites backlinks. Its measured using, page score, domain score, trust score as well as some other data
I haven’t been able to verify this yet, but some people are reporting a new Google User-Agent in their logs called “Google-speakr”. The IP of the bot is 66.249.81.186 which resolves to Google. People are speculating that the bot is some sort of Home / Google Assistant crawler but there is no news from Google
New Google UserAgent – Google-speakr Read More »
This is a question I get asked a lot, I have spoken at quite a few conferences and wrote quite a lot of articles across the web on how and why you should be doing server log analysis. The most common question I get asked is: How do I convert my log files into Excel?
Convert log file to excel Read More »
I’m Corey, The Server Log Kid. Today’s question is what is the difference between access and error logs?
What is the difference between access and error logs? Epioside 10 Read More »
This article has been edited on 3rd June 2018. So today is May 25th, 2018 and a new EU law came into effect today. They are designed to give EU citizens more protection over how their data is handled, stored and used.
EU GDPR Rules and Server Log Data Read More »
This is the first video in our new series, with me Andy and Corey Aka The Server Log Kid.
Can you see Google bot activity in the logs on sitemap pages? Episode 1 Read More »
One of the first steps to Server Log analysis is actually downloading the files and if you’re using cPanel to access your server its a very simple process. https://youtu.be/uyX5gPy8Vz0 Hopefully, you will have watched the previous video on how to keep historical logs, so know you have historical files your going to want to download them
How to get your Server Logs using cPanel Read More »
If you want to analyse your server logs, then unticking this one box in cPanel is the difference between doing great server log analysis and good analysis.
Ticking this one box allows you do great Server Log Analysis Read More »
If you have a development team to handles your servers and website, then you are probably going to need to ask them to get you the data, it’s much simpler than getting the data yourself, however, you need to know what to ask for. Here are the essential items you need to request.
What to ask a developer Read More »