Crawl depth seems off?
-
I'm reviewing my site crawl data and am seeing some very strange things such as:
- The homepage URL has a listed crawl depth of 2.
- Pages that are featured in the main site navigation (which is present on all pages, including homepage) are ranking at a crawl depth of 3.
What am I missing here? Shouldn't my homepage have a crawl depth of 0 or 1? Why would pages linked directly from my homepage have a crawl depth other than 1? (Single click from homepage to that page)?
Thank you!
-
Hi Samantha,
I set up a new campaign using the https:// version of the site and ran a new crawl, but I'm running into the same issue as before. Perhaps this is a bigger question of how site redirects work? I was under the impression that any large-scale redirects (such as from non-www to www or http to https across all pages) can affect crawl time/load time. Rereading your comment, it sounds like what you're saying is those redirects count as layers of crawl depth, as well. By the same token, I'm assuming any redirects (301's in particular) also add a layer of crawl depth.
So, my larger question then is: how can I maximize crawl depth if my site has been redirected from http to https? Will that "extra layer" of crawling always be there as long as the redirect is in place, or is there a way to compress/expedite how the crawl happens?
Thanks for your input on this!
-
Hi Samantha,
That makes sense, thank you. I'll set up a new campaign tracking with "https://" instead!
-
Hey there,
Sam from Moz's Help Team here!
So the thing to keep in mind when you set up a campaign at the root domain level is that we'll be starting the crawl from the http protocol (non-www). In this case - http://logic2020.com/. If you filter by crawl depth in your Site Crawl you'll see that URL with a crawl depth of 0.
It redirects to http://www.logic2020.com/ which has a crawl depth of 1. That URL then redirects again to https://www.logic2020.com/, which is listed with a crawl depth of 2 - hence why links we found on that page have a crawl depth of 3.
I hope this helps to clarify but let me know if you have any other questions!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Why my website Backlinks not getting Crawl by Moz?
Hi, I've the query related to backlinks of my website. Some of the high authority sites give backlinks to my website but these links are still not showing in Moz. My website is 6 months old and also continuously getting backlinks from high authority sites but still these are not showing on Moz and also not improving DA and PA of my website. I've also attached the screenshot of Moz link explorer results please check and guide me what to do with website so Moz will consider it and give some authority. And also guide me How much Moz takes time to crawl website backlinks and shown in their link explorer. Website URL: https://www.welderexpert.com Moz Expert Suggestions needed. Thanks, overview?site=welderexpert.com⌖=domain
Link Explorer | | KOidue0 -
Moz can't crawl our site
Moz can't crawl our site because of an error in the robots.txt, we've tried everything in the troubleshooting guide but nothing works - I believe its a server error but have no idea how to fix it pls help
Link Explorer | | SigneerHFS0 -
Internal Equity-Passing Links not getting crawled in Moz Open Site Explorer?
Internal Equity-Passing Links not getting crawled in Moz Open Site Explorer. What is the cause of this? We've checked the robots.txt and htaccess file, but so far we can't find anything that would be blocking Moz from crawling the internal links. We manage loads of other clients on this platform and this is the first time we've run into this issue. What else can I check?
Link Explorer | | OozleMedia0 -
Sudden Spike in 404 Pages Not Found in Moz Crawl But No Errors in WMT
Recently I received a spike in errors from the Moz crawler. When I looked into the matter I noticed that all the URI's looked right but then I looked a little closer and there was a /page/2 and /page/3 in front of the URI's. I'm running a WordPress website. Immediately I thought to myself this must be some kind of caching or permalinks error. So I disabled all my plugins including W3 Total Cache and ran the Integrity Link Crawler for the Mac and found that the errors were still popping up. 404-errors-ncworkercomp.png?dl=0
Link Explorer | | NCCompLawyer0 -
Moz Crawl Issue?
So I've been looking over my latest crawl and two of each category appears, the only difference between them is a / at the end, i.e. http://thespacecollective.com/space-clothing/ http://thespacecollective.com/space-clothing One brings in a 200 code while the other brings in a 301. Given that the 301 is in place I know Google won't see it as a duplicate, but I'm curious as to why Moz picks up on this and whether or not there is an issue that needs addressing here?
Link Explorer | | moon-boots0 -
Moz cannot crawl domain. Also OSE does not work properly on this specific domain?
Hi all, Moz cannot crawl the domein http://www.hoesjescases.nl.
Link Explorer | | Guapa_zwolle
When I open the crawl report I only see one line: <colgroup><col width="229"><col width="287"><col width="420"><col width="370"><col width="141"></colgroup>
| URL | Time Crawled | Title Tag | Meta Description | HTTP Status Code |
| http://www.hoesjescases.nl | 2015-10-05T12:20:48Z | 404 : Received 404 (Not Found) error response for page. | Error attempting to request page; see title for details. | 404 | Also when running OSE on this domain, Moz only can find 4 root domains while Majestic can find 91 domains. Google seems not to have any problems. What can be the problem for MOZ? Greetings!0 -
Why is Moz not crawling my backlinks
Hi my website www.dealwithautism.com is 3 months old and has been on DA 1 and PA1 ever since, even though the site is actively developed with quality content (a couple of posts already have 1k+ fb likes acquired editorially, while that doesnt necessarily improve SERP, it sure tells you that the post is engaging). In contrast another site of mine, www.deckmymac.com which is hardly ever managed, not have more than 15 posts and just 1 backlink, has DA 14. Running an on page analysis on www.dealwithautism.com I observed that Moz has not identified any backlinks nor social signals (except G+). However, according to Webmasters, I have 57 links, 51 of them to the root. Even Majestic is able to report 32+ backlinks. So what am I missing? Certainly, at this stage my website doesn't deserve DA 1, or does it?
Link Explorer | | DealWithAutism0 -
Does OSE crawl our site often? Or do we have an other problem? (internal links)
I am wondering if i can find out when OpenSiteExplorer crawled our entire website for the last time.
Link Explorer | | wilcoXXL
A few months ago we added some internal textlinks to all of our productpages. Those are links to the brandpage of that particular product. Some of our brands do have up to 1500 products, so there should be at least 1500 internal links pointing to that brandpage. But OSE still gives me a wrong count. It says there is only 1 internal link pointing to that brandpage.
Is it because OSE never crawled our site again? Or am i missing something here?(maybe too many internal links to one specific brandpage is not okay?) I'll hope you guys can help me out with this one.0