Crawl depth seems off?
-
I'm reviewing my site crawl data and am seeing some very strange things such as:
- The homepage URL has a listed crawl depth of 2.
- Pages that are featured in the main site navigation (which is present on all pages, including homepage) are ranking at a crawl depth of 3.
What am I missing here? Shouldn't my homepage have a crawl depth of 0 or 1? Why would pages linked directly from my homepage have a crawl depth other than 1? (Single click from homepage to that page)?
Thank you!
-
Hi Samantha,
I set up a new campaign using the https:// version of the site and ran a new crawl, but I'm running into the same issue as before. Perhaps this is a bigger question of how site redirects work? I was under the impression that any large-scale redirects (such as from non-www to www or http to https across all pages) can affect crawl time/load time. Rereading your comment, it sounds like what you're saying is those redirects count as layers of crawl depth, as well. By the same token, I'm assuming any redirects (301's in particular) also add a layer of crawl depth.
So, my larger question then is: how can I maximize crawl depth if my site has been redirected from http to https? Will that "extra layer" of crawling always be there as long as the redirect is in place, or is there a way to compress/expedite how the crawl happens?
Thanks for your input on this!
-
Hi Samantha,
That makes sense, thank you. I'll set up a new campaign tracking with "https://" instead!
-
Hey there,
Sam from Moz's Help Team here!
So the thing to keep in mind when you set up a campaign at the root domain level is that we'll be starting the crawl from the http protocol (non-www). In this case - http://logic2020.com/. If you filter by crawl depth in your Site Crawl you'll see that URL with a crawl depth of 0.
It redirects to http://www.logic2020.com/ which has a crawl depth of 1. That URL then redirects again to https://www.logic2020.com/, which is listed with a crawl depth of 2 - hence why links we found on that page have a crawl depth of 3.
I hope this helps to clarify but let me know if you have any other questions!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Moz can't crawl our site
Moz can't crawl our site because of an error in the robots.txt, we've tried everything in the troubleshooting guide but nothing works - I believe its a server error but have no idea how to fix it pls help
Link Explorer | | SigneerHFS0 -
Moz isn't crawling all my backlinks.
Moz isn't crawling all my backlinks. It's showing only 29 referring domain when I have more than 200 referring domains linking to my website. My website URL is 360gisthub.com.ng 360gisthub.com.ng
Link Explorer | | Mustybay0 -
Crawling 4XX errors because of URL accented
Hello guys, I am experiencing crawling errors in Moz because the URLs read by spiders contain accents and special characters, which I know isn't best but yet my client needs to keep it. I know that Moz "uses percent encoding to parse the HTML in the source code, so any line breaks and spaces in your HTML links or sitemap links are converted to %0A and %20, causing a 404 error". Is there any way to avoid these errors happening in the dashboard? Or am I supposed to simply ignore it?
Link Explorer | | fran8751 -
Why are recently deleted pages still appearing in the latest MOZ crawl?
Newbie, so please forgive!! OK, so I'm doing my 1st site optimization. It is reporting errors from pages that were deleted a couple of days ago. And I JUST signed up today. Where is this info coming from? Thanks, Billy
Link Explorer | | NewSEOguy0 -
Can i Force opensiteexplorer to crawl a url?
Hello. I check the QA and didnt find an answer to this. I've check one link on my site which is indexed by google but not by mozopensite explorer. If i search the url i find it on google but not i nopen site explorer. This only means that open site explorer havent find it. IF google index that page this means i'm getting the juice? My question is if google index a page for sure it follow one of the links (follow link). This is importan for me since this is my best DA link, probably a game changer for us. Also, Is there a Way to force opensiteexplorer to crawl this page? Thank you.
Link Explorer | | ebest.cl0 -
804 error preventing website being crawled
Hi For both subdomains https://us.sagepub.com and https://uk.sagepub.com crawling is being prevented by a 804 error. I can't see any reason why this should be so as all content is served through https. Thanks
Link Explorer | | philmoorse0 -
Local listings aren't being crawled?
Hello, SEO brainiacs. Here is the problem I can't figure out:We have a lot of clients and all of them have local listings, such as Insider Pages, Yahoo local, Bing locals, Yelp, Yellow Pages etc etc etc.
Link Explorer | | seomozinator
And all of those local listing have company's website link. But none of those links are considered (found) in reports, such as opensite explorer or in Link Analysis Section of Moz PRO. I do know that MOZ crawl considers only high DA websites, but all of local listing domains have above 80-90 DA score, so they can't be NOT crawled. I will say that some local listings do appear on very few reports, but far not all of them. So, what can be the problem?0 -
How Is a Page Crawled by Moz When Moz Says 'No Links'?
As above, really. I've crawled a new client's site to find the Moz crawler has identified a handful of 404 errors. The Moz crawler says these pages have '0 linking domains', and OSE has no data for these pages. So how are these pages being crawled by Moz and what should I advise my client?
Link Explorer | | xerox4320