You might have noticed a fairly big update with the latest Screaming Frog release. They have added another API integration. PageSpeed Insights by Google.
But before you jump in and start pulling in metrics for every URL (and you should – speed is measured at a URL level, not domain level, just because your home page is fast it doesn’t mean the rest of the site is quick) there is something you should know.
The way it works it affects Google Analytics tracking (and probably other tracking tools too).
The sessions appear as direct sessions in GA, got to give a big shoutout to Screaming Frog for advising on how to fix this.
You should also care even if you don’t use Screaming Frog or the PageSpeed Insight API, your competitors could be crawling your site and artificially inflating your traffic.
I often when doing Audits will take the top 5 competitors and analyse them for speed. It’s like the old saying “you don’t have to outrun a lion, you just have to outrun at least one person.”
Your site doesn’t have to be the quickest of all the sites on the internet, just the quickest in your peer group.
So I would highly recommend you add these as filters to your Google Analytics and it’s worth pointing out Google is aware of this issue and is likely to be working on a fix.
There are two methods, the first one is the easiest but neither are 100% guaranteed to work every time.
How to Stop PageSpeed API affecting Google Analytics:
- Add a filter to exclude known Google IP addresses
- Create a custom dimension of User-Agent and exclude Google (do need GTM or access to the site to set this up)
1) The following IP address exclude filter regex pattern appears to work from our own testing. However, we recommend using with caution, setting this up in a view that isn’t important to you and testing first.
2) An outline of the steps for this method is below:
- Create a custom dimension in Analytics – this can be found under the Admin for the property and Custom Definitions > Custom Dimensions
- In GTM, create a JavaScript variable and in the settings enter navigator.userAgent
- In the Google Analytics Settings, variable add the custom dimension’s details created in the first step with the user agent variable created in the second set
The drawback with step 1 while easier is that Google IPs can change so you might still see the spikes and once the data is in Google Analytics is almost impossible to get out.
Also, potentially if you genuinely have people from Google HQ visiting your site you would block these (assuming they are on the same IP ranges – but I am not sure).
The above are just guidelines, every set up is different and while it will work for the majority you might need to tweak them for your site.
Let’s just hope Google fixes this issue pretty quick.