Database driven content producing false duplicate content errors
-
How do I stop the Moz crawler from creating false duplicate content errors. I have yet to submit my website to google crawler because I am waiting to fix all my site optimization issues.
Example: contactus.aspx?propid=200, contactus.aspx?propid=201.... these are the same pages but with some old url parameters stuck on them. How do I get Moz and Google not to consider these duplicates. I have looked at http://moz.com/learn/seo/duplicate-content with respect to
Rel="canonical"
and I think I am just confused.
Nick
-
All of you guys rock! I have never been involved in a community that has had the right answers every time... I used the on all my static pages such as directions, policies, contact, etc... and it removed all the parameters thereby eliminating them from standing out in the MOZ crawl.. I feel like and idiot not knowing about this HTML tag and its importance. My moz crawl now looks so so much better.
When I mean old url parameters, I just meant a few seconds old, meaning the user is on property.aspx?property=1 then when they moved to a static page such as contact, directions, policy we now have another page called contact.aspx?property=1 which if I have 150 properties times 10 static pages I basically just created 150 duplicate content errors just for the contact page alone. Because contact.aspx?property=1 or contact.aspx?property=150 and in between are all the same page... I am sure this has killed my SEO. SO THAT PROBLEM IS NOW FIXED!!
NOW to revisit what zenstorageunits says about URL rewriting which has many different ways to do it using .net, but Miketek I would not have to create subdirectories because it is done in the code... they are more like virtual directories...
zenstorageunits or anyone else for that matter, Is it worth it for me to hire somebody to create a URL rewrite app that can change the following;
http:/www.destinationbigbear.com/property_detail.aspx?propid=202 to
http://www.destinationbigbear.com/big-bear-cabin-rentals/a-true-cabin/details
and
http:/www.destinationbigbear.com/property_photos.aspx?propid=202 to
http://www.destinationbigbear.com/big-bear-cabin-rentals/a-true-cabin/photos
See everyone of my 150 cabins has these pages; info, photos, calendar, video, reviews, rates...and they all have unique cabin names... so it is basically 150 cabins x 6 pages = 900 unique pages with unique content but really only 6 pages dynamically being changed by 150 cabins.
I have been able to dynamically change all the page titles for everyone of these 900 database driven pages such as
Big-Bear-Cabin | A True Cabin Photos or Big-Bear-Cabin | A True Cabin Calendar and so on.
-
Hi Nick,
I think you've gotten some good tips here - I'd agree with Prestashop that the preferred solution would be to find where these parameters are being included in links to this page and remove them.
Failing that, zenstorageunits's advice to use rel="canonical" would be my recommendation - or a 301 redirect from the URLs that include parameters back to the core URL would work.
I wouldn't convert these parameters to subdirectories unless they are integral to the way your site works and pull up unique content - you called them "old parameters" so it sounds like they're not supposed to be there, so probably not a case where you'd want to convert these parameters to subdirectories.
Failing the above, you could utilize the Google Webmaster Tools "URL Parameters" interface to tell Googlebot to ignore these parameters.
Overall, your best course of action is to find and remove the links that include the parameters.
I'd also add that the Moz crawl report is highly sensitive to "duplicate content," and I often find it flags up issues has high/medium priority that are not actually going to have a significant impact on the site. You have to take the crawl report with a grain of salt - while duplicate content can be a serious issue for some sites (ecommerce retailers for example with duplication issues across a wide catalog of products), in most cases it has minimal impact and isn't something I'd hold up your site launch for.
Best of Luck,
Mike -
I agree zenstorageunits about using rel=canonical but one thing I would like to point out is that Moz does not create false errors. It is a simple crawler, not like google. Google will actually try to follow links that people have used before and that show up in your analytics files. moz uses no logic like that, it just jumps from page to page. If it is picking up a page with a query string like that then it is a link on your site. I would find the links and take them off.
-
You have a few options you could do. One thing I would look into is maybe doing some url rewriting to change
contactus.aspx?propid=200
to
contactus/propid/200
look at http://msdn.microsoft.com/en-us/library/ms972974.aspx on how to do that for IIS.
A better option I think if you need to keep the parameters the way they are is to use the rel canocial tag look at moz article
http://moz.com/blog/rel-confused-answers-to-your-rel-canonical-questions
but basicly you would need to add something like this to your contact.aspx page(replace example.com with your website url)
This suggest to the website crawler, like google or moz crawler, that those pages should be assoicated with the contact.aspx page.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Development Website Duplicate Content Issue
Hi, We launched a client's website around 7th January 2013 (http://rollerbannerscheap.co.uk), we originally constructed the website on a development domain (http://dev.rollerbannerscheap.co.uk) which was active for around 6-8 months (the dev site was unblocked from search engines for the first 3-4 months, but then blocked again) before we migrated dev --> live. In late Jan 2013 changed the robots.txt file to allow search engines to index the website. A week later I accidentally logged into the DEV website and also changed the robots.txt file to allow the search engines to index it. This obviously caused a duplicate content issue as both sites were identical. I realised what I had done a couple of days later and blocked the dev site from the search engines with the robots.txt file. Most of the pages from the dev site had been de-indexed from Google apart from 3, the home page (dev.rollerbannerscheap.co.uk, and two blog pages). The live site has 184 pages indexed in Google. So I thought the last 3 dev pages would disappear after a few weeks. I checked back late February and the 3 dev site pages were still indexed in Google. I decided to 301 redirect the dev site to the live site to tell Google to rank the live site and to ignore the dev site content. I also checked the robots.txt file on the dev site and this was blocking search engines too. But still the dev site is being found in Google wherever the live site should be found. When I do find the dev site in Google it displays this; Roller Banners Cheap » admin dev.rollerbannerscheap.co.uk/ A description for this result is not available because of this site's robots.txt – learn more. This is really affecting our clients SEO plan and we can't seem to remove the dev site or rank the live site in Google. In GWT I have tried to remove the sub domain. When I visit remove URLs, I enter dev.rollerbannerscheap.co.uk but then it displays the URL as http://www.rollerbannerscheap.co.uk/dev.rollerbannerscheap.co.uk. I want to remove a sub domain not a page. Can anyone help please?
Technical SEO | | SO_UK0 -
Rel=canonical overkill on duplicate content?
Our site has many different health centers - many of which contain duplicate content since there is topic crossover between health centers. I am using rel canonical to deal with this. My question is this: Is there a tipping point for duplicate content where Google might begin to penalize a site even if it has the rel canonical tags in place on cloned content? As an extreme example, a site could have 10 pieces of original content, but could then clone and organize this content in 5 different directories across the site each with a new url. This would ultimately result in the site having more "cloned" content than original content. Is this at all problematic even if the rel canonical is in place on all cloned content? Thanks in advance for any replies. Eric
Technical SEO | | Eric_Lifescript0 -
Fix duplicate content caused by tags
Hi everyone, TGIF. We are getting hundreds of duplicate content errors on our WP site by what appears to be our tags. For each tag and each post we are seeing a duplicate content error. I thought I had this fixed but apparently I do not. We are using the Genesis theme with Yoast's SEO plugin. Does anyone have the solution to what I imagine is this easy fix? Thanks in advance.
Technical SEO | | okuma0 -
Link Structure & Duplicate Content
I am struggling with how I should handle the link structure on my site. Right now most of my pages are like this: Home -> Department -> Service Groups -> Content Page For Example: Home -> IT Solutions -> IT Support & Managed Services -> IT Support Home -> IT Solutions -> IT Support & Managed Services -> Managed Services Home -> IT Solutions -> IT Support & Managed Services -> Help Desk Services Home -> IT Solutions -> Virtualization & Data Center Solutions -> Virtualization Home -> IT Solutions -> Virtualization & Data Center Solutions -> Data Center Solutions This structure lines up with our business and makes logical sense but I am not sure how to handle the department and service group pages. Right now you can click them and it just brings you to a page with a small snippet for the links below. The real content is on the content pages. What I am worried about is that the snippets on those pages are just a paragraph or two of the content that's on the content page. Will this hurt me and get considered duplicate content? What is the best practice for dealing with this? Those department/service group pages have some good content on them but it's just parts of other pages. Am I okay doing this because there are not direct duplicates of other pages just parts of a few pages? Any help on this would be great. Thanks in advance.
Technical SEO | | ZiaTG0 -
How do I get rid of duplicate content
I have a site that is new but I managed to get it to page one. Now when I scan it on SEO Moz I see that I have duplicate content. Ex: www.mysite.com, www.mysite.com/index and www.mysite.com/ How do I fix this without jeopardizing my SERPS ranking? Any tips?
Technical SEO | | bronxpad0 -
Tags and Duplicate Content
Just wondering - for a lot of our sites we use tags as a way of re-grouping articles / news / blogs so all of the info on say 'government grants' can be found on one page. These /tag pages often come up with duplicate content errors, is it a big issue, how can we minimnise that?
Technical SEO | | salemtas0 -
Duplicate content and http and https
Within my Moz crawl report, I have a ton of duplicate content caused by identical pages due to identical pages of http and https URL's. For example: http://www.bigcompany.com/accomodations https://www.bigcompany.com/accomodations The strange thing is that 99% of these URL's are not sensitive in nature and do not require any security features. No credit card information, booking, or carts. The web developer cannot explain where these extra URL's came from or provide any further information. Advice or suggestions are welcome! How do I solve this issue? THANKS MOZZERS
Technical SEO | | hawkvt10 -
I'm getting a Duplicate Content error in my Pro Dashboard for 2 versions of my Homepage. What is the best way to handle this issue?
Hi SEOMoz,I am trying to fix the final issues in my site crawl. One that confuses me is this canonical homepage URL fix. It says I have duplicate content on the following pages:http://www.accupos.com/http://www.accupos.com/index.phpWhat would be the best way to fix this problem? (...the first URL has a higher page authority by 10 points and 100+ more inbound links).Respectfully Yours,Derek M.
Technical SEO | | DerekM880