Crawl diagnostics incorrectly reporting duplicate page titles
-
Hi guys,
I have a question in regards to the duplicate page titles being reported in my crawl diagnostics. It appears that the URL parameter "?ctm" is causing the crawler to think that duplicate pages exist. In GWT, we've specified to use the representative URL when that parameter is used. It appears to be working, since when I search site:http://www.causes.com/about?ctm=home, I am served a single search result for www.causes.com/about. That begs the question, why is the SEOMoz crawler saying there is duplicate page titles when Google isn't (doesn't appear under the HTML improvements for duplicate page titles)? A canonical URL is not used for this page so I'm assuming that may be one reason why. The only other thing I can think of is that Google's crawler is simply "smarter" than the Moz crawler (no offense, you guys put out an awesome product!).
Any help is greatly appreciated and I'm looking forward to being an active participant in the Q&A community!
Cheers,
Brad
-
Glad I could help, Bradley. Let us know if you need help with anything else.
-
Thanks for the thorough response Chiaryn! I figured as much but wanted to make sure I wasn't overlooking anything.
-
Hey Bradley,
You're right; Google's crawler is way more sophisticated than ours is because they have a lot more resources, be they engineers or finances, to pour into their crawler. We think our crawl provides tremendous value and is an excellent way to discover and understand the architecture of your site at scale, but it's not that strange that it wouldn't line up with exactly what a site: search reveals. We also don't always know how Google (or other search engine bots) is going to consider a set of pages, so we would rather be safe than sorry with the data we provide.
Since the page http://www.causes.com/about?ctm=home is linked to from another page on your site (www.causes.com) and resolves with a 200 status, our crawler sees it as an individual page and won't associate it with the main /about page. Instead, it just compares the code and content with the other pages we've crawled and reports back when we find duplicates.
I hope this helps clear things up. Please let me know if you have any other questions.
Chiaryn
Help Team Ninja
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Why does moz give different page authority to the same page if a visit comes from adwords vs organic search?
When clicking on an adwords ad the page the landing page has a page authority of 26. When clicking on organic search to the same exact landing page the page authority is 37. Why is this. Does moz or, more importantly Google see these as the same or separate pages? Thanks Tom
Moz Pro | | ffctas1 -
Crawl Diagnostics - Crawling way more pages than my site has?
Hello all, I'm fairly new here, more of a paid search guy dabbling in SEO on the side. I have a client that I have in SEOMoz and the Crawl Diagnostics report is showing 10,000+ pages crawled and I think the site has at most 800 pages (e-commerce site using freewebstore.org as the platform). Any reasons this would be happening?
Moz Pro | | LodestoneGen0 -
Order of urls in SEOMoz crawl report
Is there any rhyme or reason to the order of urls in the SEOMoz crawl report, or are the urls just listed in random order?
Moz Pro | | LynnMarie0 -
Who wants to help go over my crawl diagnostics via skype?
I have run a crawl diagnostic on my site and have 194 errors and most of them are 404 errors in wordpress. Not sure why, but many of my pages had name changes (possibly a permalinks issue) but I have no idea how to fix it. I had 5 duplicate page titles, and 1 tile missing or empty. 72 crawl notices found (2 permanent redirect, 17 blocked by robots, 53 rel canonical) 19 Crawl warnings were found Who wants to have some fun?
Moz Pro | | starkSEO0 -
Image Asset pages shown to have Page Authority
When looking at top pages for my site in www.opensiteexplorer.org I'm seeing a bunch of asset pages being listed to have page authority. How could this be? Is open site explorer mistaken? Here is a page with a PA: 24 http://www.minespress.com/catalogassets/thumbnails/0000437_atx_software_compatible_folders.jpg
Moz Pro | | smines0 -
Pages Crawled: 0 ?
I've been with SEO Moz for over a month and a half. Why would this weeks crawl have Pages Crawled: 0? I've made no changes since the crawl last week that had 10k pages crawled...
Moz Pro | | mr_w1 -
Why did SEOMoz only crawl 1 page?
I have multiple campaigns and on a few of them SEOMoz has only crawled one page. I think this may have to do with how I set up the campaign. How do I get SEOMoz to crawl more than one page on these campaigns.
Moz Pro | | HermanAdvertising0 -
I have another Duplicate page content Question to ask.Why does my blog tags come up as duplicates when my page gets crawled,how do I fix it?
I have a blog linked to my web page.& when rogerbot crawls my website it considers tags for my blog pages duplicate content.is there any way I can fix this? Thanks for your advice.
Moz Pro | | PCTechGuy20120