How Is a Page Crawled by Moz When Moz Says 'No Links'?
-
As above, really. I've crawled a new client's site to find the Moz crawler has identified a handful of 404 errors.
The Moz crawler says these pages have '0 linking domains', and OSE has no data for these pages.
So how are these pages being crawled by Moz and what should I advise my client?
-
Thanks again, Peter.
-
That's great, thanks Keri.
-
Hi, no problem. Keri has answered your query re the Moz crawler not finding them.
Peter
-
If you download the CSV report, we list the referring page there. We know it isn't obvious and we're working on making this a little easier to find.
-
Hi Keri,
Thanks for getting back to me (thanks to Peter above, also).
So what is the best way of me finding where these 404 pages are linked from?
I am 99% sure they are internal links that I will ask the client to remove, but would like to avoid manually checking every link on the site if possible.
-
Hi Peter,
I searched for their domain name and about the expected number of results came up.
I searched for these individual URLs that Moz is reporting using the site: command, and they were not returned.
They are also neither in the site's sitemap (HTML page or XML file).
So how is SEOmoz Crawler finding them?
-
We have two completely different crawling systems. We have on crawler for your website, where you enter in your URL and we crawl your site looking for problems. We have a different crawler for OSE that goes and looks at the web in general and crawls for links. The two crawlers do not talk to each other, so it is actually understandable that OSE may not have data on the site. OSE doesn't have the same index size as Google, so we don't see all the links that are actually out there (though we're working all the time on improving this!).
-
Hi, if you put site:xxx.com into Google where xxx.com is your client's domain name, does Google report anything?
Peter
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google Disavow File Format and MOZ Spam Score Updates
Hi, Is there a defined file format for Google disavow file name? Does it has to be disavowlinks.txt or can we do this like domain-name-date.txt ? Also, since Google does not share their data with Moz, how does MOz updates its spam score after we disavow the bad links? Do we need to connect Google search console with Moz?
Link Explorer | | Sunil-Gupta1 -
Can't make sense of OSE and MOZ.
I checked this site on my OSE and it shows only 7 inbound links a DA of 14 and no social activity whatsoever yet when I check it on Majestic it shows ExternalBacklinks 71 ReferringDomains 20 Referring IPs 19 Referring Subnets 19 . And.. the page is #1 on google search for hypnotherapy michigan. How can it rate so poorly on MOZ, show so much more on majestic and rank so high on google? I thought MOZ data was supposed to be among the best and that top rated pages on google should also rate high on MOZ. here is the site http://hypnotherapy-detroit.com Additionally, when i look at the site, i notice that most of the backlinks are exchanged links and this site's link exchange page isn't even linked from the home page. Now I thought that kind of link exchange game was now discounted by Google. I don't get it. No social pages at all... low page rank... no new content.. so by MOZ standards there is no justification for this page to be anywhere near page one let alone at position #1. Can someone help me make sense of all this?
Link Explorer | | HypnoPro0 -
Do other sites linking to my page with a low domain authority affect my rankings?
Hello, my page authority 30, I see other page with an authority of 21 for example linking to us. Is this good or bad, does this affect our ranking? A lot of these don't have anchor text, I'm guessing we should reach out to these webmaster to include anchor text with the keywords we want to drive traffic to this page or make as descriptive as possible? Thanks! Laura
Link Explorer | | lauramrobinson320 -
Why is Moz not crawling my backlinks
Hi my website www.dealwithautism.com is 3 months old and has been on DA 1 and PA1 ever since, even though the site is actively developed with quality content (a couple of posts already have 1k+ fb likes acquired editorially, while that doesnt necessarily improve SERP, it sure tells you that the post is engaging). In contrast another site of mine, www.deckmymac.com which is hardly ever managed, not have more than 15 posts and just 1 backlink, has DA 14. Running an on page analysis on www.dealwithautism.com I observed that Moz has not identified any backlinks nor social signals (except G+). However, according to Webmasters, I have 57 links, 51 of them to the root. Even Majestic is able to report 32+ backlinks. So what am I missing? Certainly, at this stage my website doesn't deserve DA 1, or does it?
Link Explorer | | DealWithAutism0 -
Is there some way to tell the Moz crawler not to crawl URL's with particular dynamic tags such as "?redirect-to:http//" ?
We are encountering an issue where the crawler is finding a ton of pages from our wordpress login url that has this dynamic tag in it to kinds of different blog entries. It's madness. I can't figure out what is causing these URLs to generate to be crawled in the first place! Does this sound familiar to anyone out there, any constructive suggestions? Robots text or maybe meta robots tags that would resolve this crawl issue?
Link Explorer | | RegistrarCorp0 -
Why doesn't my site show up in Moz Site Explorer
I started this website about 2 months ago, www.guyetteroofing.com. I know it still needs a lot of work, but I can't find any information on it on the site explorer tool.
Link Explorer | | billyguyette1 -
Do 'Just Discovered' Links get added to the main link index?
Hi, I was wondering if the 'Just Discovered' links get added to the main link crawl index? It would seem to make sense for them to do so, as this would enable the enable the link index to be more up to date than it would otherwise be. Observing the link index it would seem that at the moment it does not do this and they are totally separate indexes (based on personal observation). Thanks
Link Explorer | | James770 -
301 Redirects: How long for Google to recognize? How long for Moz/OSE to recognize?
I redirected a few pages that had funky URL issues (capitals and underscores and useless words) to the same page renamed with proper URLs. The sitemaps were changed also, and re-submitted. Like this: mydomain.com/Not_Pertinent_Words.html >>> mydomain.com/good-words.html Google seems to have found them and changed the search engine results listing in about 8 days. But it's been about a month now and Moz and OSE still have not transferred all the strength and link data from the old URLs to the new ones. Question 1: How long does it usually take Google to transfer all the link and strength data for a 301? Question 2: How long does it take Moz and OSE to do the same? Is there something I need to do to tell them about the changes?
Link Explorer | | GregB1230