How to find Orphan Pages

 

 

Note: If the site doesn’t have the pages in the sitemaps or any internal links, it will be quite hard to identify them, there is where having a good development team can help.

If you don’t have a development team – you can look on your server and find the pages.

Get in touch if you need more help finding Orphan pages.

Transcript

Hi! I’m Corey, the Server Log kid.

Today we are looking at orphan pages.

So that was correct, Corey. Today we are going to be looking at orphan pages and why they’re important, but more importantly, how to identify them.

So, orphan pages are basically pages on your website that have no internal links.

The reason that these are important to find and identify is because if Google bot can’t crawl them and find links on your site, pretty much they’re not going to be in the index.

If Google can’t crawl them, they can’t index them. Simple as. This plenty of ways to find orphaned pages.

In this video we’re gonna show you an example using Screaming Frog, so you can use Screaming Frog. SEMRush has a feature if you pay for SEMRush.

I believe Ahrefs has a similar feature.

You can look in your Analytics is not as reliable because you still got to rely on pages getting traffic.

You can look in your server logs but again you need to have Google hitting these pages to be able to find them.

You can look in your site maps and, hopefully, the links are auto-populating from your site maps.

But again you’re relying on devs adding the pages into site maps correctly.

Or finally, the other way to find them, is to speak to your dev team and ask for a full list of every page they expect to be on the website. And then we can crawl it and just do a simple VLOOKUP, compound the two.

But like I said, today we’re going to be looking at Screaming Frog and how to use that tool, and that tool alone, to find orphan pages.

See I’m going to now show you how to use Screaming Frog to actually find orphaned page on your site, like you just said, there are other ways of doing this.

You can do it manually, looking in server logs, but I think this is probably the most accurate way.

Or at least the most accurate way, including speaking with your developers.

Developers, if you ask them for a list of pages, could correctly give you every single page, but they still might miss some.

So how I do it is I like to both methods speak to developer, get them to give you a list of pages, and also do it this way.

Then compare the two cause then you should cover the most angles.

So let’s get into Screaming Frog. We do need to make some changes into configuration, Spider. You want it to crawl XML site maps.

How to

1. Configure the setting in Screaming Frog to allow it crawl your sitemap
2. Set Screaming Frog to Crawl your website
3. Once Finished head to “Reports”
4. Chose “Orphan Pages”
5. Save to a location on your machine
6. Open file

You can do it auto-discover in the robots file.

But you might as well just put it in right there so there’s a site map.

Then what you’re going to do is a site crawl and it will go off and do a crawl, obviously, i have done that as I didn’t want to sit here for a few minutes and just watch the Screaming Frog go. Once you’re done simply go to reports, orphan pages, and then you want to … click what you want it on, and save it to what you want.

We’re going to save it to desktop.

That’s it, done.

And then I open the report. Here is a list of every single orphan page on the site. As you can see, I made two orphan pages.

These are all site maps, where each … you can’t count them to be orphan, there are just the site maps, but I wanted to show some real pages in here.

If you go to the website they are just, there’s nothing on them.

I’ll leave them there for a few weeks, just an orphan page and orphan page dash two. But they will get deleted because they are just a waste of time.

Yeah. That’s it. That’s as simple as it is. See there’s no reason for you not to go and do it.

This works on smaller sites and micro-sites.

Obviously, the larger the website is it takes longer for Screaming Frog to crawl it, but yeah.

Thanks for watching that on how to use Screaming Frog to find orphaned pages on your website.

Hope you found it useful. Highly recommend you go and do it.

It’s super important, like I said at the beginning of this video, for finding pages that Google might not be finding.

hen how you deal with them, is up to you.

Whether you decide to add more internal links to them, or whether you remove them, but once you know the data you can make an action towards solving it.

Thanks for watching.

You can follow us on YouTube. Where Onpage Rocks. You can subscribe to us on Twitter @OnpageRocks. He’s @ServerLogKid. We’re on Facebook.

If you feel like you’ve got any questions you want answering, fire em in, and Server Log Kid will ask them and we’ll get cracking.