Moz Crawler not Identifying all Duplicate Pages
-
On two recent site crawls (9/27/14 and 11/4/14) for duplicate content the Moz tool did not ID the following 2 pages, which are 100% duplicate to each other:
http://www.hooksandlattice.com/planter-hampton-241212.html ; Screenshot: http://screencast.com/t/DdwWroUU
http://www.hooksandlattice.com/planter-hampton-721212.html ; Screenshot: http://screencast.com/t/8Lb1cJZmGrhX
As I'm working feverishly to re-write and update the site (goal is ZERO duplicates) I'm finding it challenging to use the Moz tool to get the project done. Does anyone have any feedback or help they can provide for how I can identify all duplicate pages associated with my domain?
Thank you!
Lindsey Pfeiffer
-
Hi Lindsey
Our engineers have confirmed that rogerbot will flag pages that are 100% identical but can sometimes miss pages that are 99% similar. The crawler is deliberately written to err on the side of not reporting false positives which means it sometimes can report false negatives which has occurred in your case. Using a combination of tools such as Webmaster tools can help isolate any pages we have missed.
Hope this helps!
-
Hey Lindsey!
I am not sure why our crawler did not flag those pages as they are 99% identical and are not sharing the same canonical URL. This is very strange and I'll send this up to our crawler engineer to obtain more insight.
Will let you know what I find out once I hear back!
-
Do you check Google Webmaster Tools? Under Search Appearance > HTML Improvements Google will list duplicate titles and descriptions among other things, which might be a help to you.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
On-Page Grader
Hello the I am trying to use On-page Grader tool on https://www.brownandkings.co.uk/ but is not allowing me to get any grade. I thought might be an issue on my hosting provider, but they said all configuration is perfect... I also went to use https://technicalseo.com/tools/robots-txt/ to test the robots and the configuration and test came perfect for the webpage. I search on Google, and all the FAQ here in Moz but no answer, how to solve this issue... so that is the reason for this question. "On-page Grader tool" some of the campaign I list here work but some don't 😞 how can I make work Looking for hearing from you!
Moz Bar | | mattdilon0 -
Page Title showing Twice when I check it through Moz Bar
Hello Experts, I have a brand new website. And I know I have to do lot of things and make changes for better SEO. With that in mind, I recently installed Mozbar. I was checking one of my posts the other day and noticed Page Title is appearing twice in Mozbar.Please see the screenshot below. Could you please guide me what could be causing this? FYI, I have installed Yoast SEO Plugin. Also, I can see "Alt Text" is picking lot many words which are from other images or posts...Is there any way to customize that as well? KhP9y
Moz Bar | | techierichard0 -
Moz Bar doesn't show any data and keeps asking me to log in when actually I'm logged in.
Hi all, I've been using Moz Bar for years. It ran well until about three weeks ago. It suddenly failed to show the DA and PA of sites that I open after I log in. And it keeps asking me to log in when I did. I tried to uninstall the Mozbar extension and reinstalled it several times. Nothing worked. I also tried to uninstall Chrome and clear the cookies, still, nothing changed. Did anyone experience this? How do you solve it and make it run on the track? Any information will be appreciated. [admin edited support category]
Moz Bar | | Bennie22339 -
Am I the only one seeing the Moz's points ranking with mixed rankings?
Hello everyone. It's been several weeks since the problem persists, Here: https://moz.com/community/users
Moz Bar | | Gaston Riera
There are rankings that are incorrect and some users that have the same ranking than other users. I've attached some pics. GR 25603b49292acd5d68f7dd6a43977f94 6d8fea557f6ed2de23466f6c7df850eb0 -
All of a sudden a number of my key pages are getting 403 errors with Moz?
One of my squarespace sites has suddenly thrown up a number of 403 access denied errors for a range of pages on the site, according to the Moz weekly report. There is no access issues for these pages and nothing has been changes wrt url, etc..... so why the errors (which I am not seeing through SEM rush for example)? Thx, Phil
Moz Bar | | bugdoctor0 -
I want to uninstall the Moz SEO toolbar. How do I do this?
I installed the Moz toolbar and I don't understand it and it covers up important parts of websites and makes them inaccessible. I want to get it off my computer. I installed it in chrome. How do I get it off?
Moz Bar | | Bonnie761 -
Can't delete items from the on page grader
I check every single box and they don't delete. This is driving me nuts. Please can you delete them for me because I am not impressed with this AT ALL. In fact I am getting so cross I am in danger of screaming hysterically which might get me the sack and it would be all your fault. That was slightly tongue in cheek, but please can you fix it please. please.
Moz Bar | | CommT0 -
Moz reporting appropriate Canonical tag usage but no canonical tag on page !?
I take it this means that the page in question has been referenced via a different pages canonical tag but that the page in question itself does not have a self referencing canonical tag (and that it should do) cheers dan
Moz Bar | | Dan-Lawrence0