How to Solve Duplicate Page Content Issue?
-
I have created one campaign over SEOmoz tools for my website. I have found 89 duplicate content issue from report.
Please, look in to Duplicate Page Content Issue.
I am quite confuse to resolve this issue. Can any one suggest me best solution to resolve it?
-
No probs glad to help!
Best of luck!
-
Oh great. That's fine. Now, I got idea... What's wrong with my site. If I suppose to find that duplicate page in Admin so I may not able to find that pages. Right?? So, I just need to set 301 (Permanent Redirection) with help of htaccess. Now, It's clear. Thanks a lot for your prompt reply and quick discussion on my issue.
-
Are you talking about HTML sitemap or XML sitemap?
If HTML sitemap so you are right and I suppose to make it live very soon with proper structure.
If you are talking about XML sitemap so I have created it with two partition.
http://www.vistastores.com/main_sitemap.xml
http://www.vistastores.com/products_sitemap.xml
So, What you think about it?
301 redirection is final. Right?? After removal of duplicate pages.
-
The tool you used to find these pages seems to work fine ;).
It's simply a crawl so use the seomoz data as it's a crawl of your site
Set a 301 redirect with the complete list of duplicated content, you don't need to delete the pages(it's not deleted anyway since it's dynamically created pages)
/ G
-
Ok! Then we have that cleared
If you already are using URL rewrite there shouldn't be any duplicated content in the manner the list shows.
- But since there is a problem I would check so that you're not using a sitemap that dynamically crawls and creates these urls for the pages.
If that's not the issue:
- We then come back to doing the 301:s by using .htaccess.
/ Gustav
-
In addition you may want to remove "zero products" pages from index and not link to them as they are not good for users or search engines.
Yes, you are right. I want to remove zero product pages as well as all pages which were created due to human error.
As I mentioned above: Following page is not available on website for buyers. Buyers will not able to see this page any more because, not a single page available on website which is available with hyperlink to that page.
http://www.vistastores.com/126/cookwares.html
But, SEOmoz crawler detect it and when I see in excel so I quite confuse. Because, that page is working and visible with all products which are available to original page.
Users are not able to go that duplicate page but Google crawler can go there and able to detect as duplication.
I have bit knowledge about duplication and assume that Google will detect both page as duplicate. That's it. And, I want to resolve it.
-
I think, This issue is not regarding URL rewriting. I can say very sure for it. As I said, duplicate page content URLs are not available on website. If any visitors will visit website and go to each page so that visitor will not able to find duplicate page.
URL rewriting will fix issue regarding URL structure but, what about duplication which is available in my own website to compete both pages with same keyword?
-
hi again!
After reading you follow up question I know a better solution for you
Instead of going canonical which takes much manual labor or doing 301:s since both are simply just temporary fixes for the problem and not the cure.
The best way and what I always recommend our clients is working with URL rewrite. This takes care of this problem completely, however it takes some coding to implement.
If you can handle this yourself and only need hints and guides read below:
- I assume that you use Apache server?
- If so read this: http://httpd.apache.org/docs/2.0/misc/rewriteguide.html
Or:
- Contact the admin for the webshop and ask them to make an URL Rewrite based on the URL structure that you would want to use
- Good structure: root/productcategory/Product
Hope this helped you
-
I am quite confuse with 301 redirect. There are too many duplicate pages which were generated due to wrong categorization or create new same category rather edit old category.
I want to remove all duplicate pages from my website and want to set 301 redirect.
I want to give one example for it.
Original Page:
http://www.vistastores.com/125_126/kitchen-and-dining/cookwares.html
Duplicate Page:
http://www.vistastores.com/126/cookwares.html
Duplicate pages were created due to Admin issue or by human error. No one will find our land on duplicate page during website surfing.
Now, I want to detect and remove all pages which are available with duplicate content.
So, Does it matter to delete that pages from website and set 301 to associated page or home page?
-
Best practice in your case would be to implement URL canonicalisation (rel="canonical").
Watch this: http://www.google.com/support/webmasters/bin/answer.py?answer=139394 (contains explanation and examples)
In addition to this try to prevent page duplication form happening in the first place, though this may need to be done on a programming level.
It seems that string such as "6_129_130" appears based on the category and navigational path. For example if user browses from outdoor and home decor and arrive on the same page the URL will have different number.
In addition you may want to remove "zero products" pages from index and not link to them as they are not good for users or search engines.
-
Hi!
Good question, one I often come across
I would say this is not a real issue for you in the serp. But if you want to fix it and competition is fierce in your field you should of course do it
Step one: Look up your index at google: simply with site:"youdomain.com" and see if there are any duplicated content in the index(chance is slim)
Step two: look up your webshop cms, perhaps there is a function for redirecting dynamic pages to a static page(usually there is)
Step three: If it's not possible to do the 301:s from webshop admin use the .htaccess file and implement 301:s with the url:s you have in the list.
Other comments:
If you would like to clean up these URLs work with 301:s and decide which structure is most important.(base it on the current index)
There are several ways to do an 301: but in this case I would say the easiest way since you have the list is to go to the .htacces and put the 301 redirect list there.
This way could of course be bad for you if the duplicated pages are somehow necessary for the webshop so start to look at the webshop to make certain that you don't mess up any critical part by doing it by .htaccess.
Best regards!
/ Gustav
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Overdynamic Pages - How to Solve it?
Hi everyone, I'm running a classified real estate ads site, where people can publish their apartment or house they want to sell, so we use multiple filters to help people find what they want. Lately we added multiple filters to the URL to make the search more precise, things like: Prices (priceAmount=###) Bedrooms (BedroomsNumber=2) Bathrooms (BathroomsNumber=3) TotalArea (totalArea=1_50) Services (Elevator, CommonAreas, security) Among other Filters so you see the picture, all this filters are on the URL so that people can share their search on multiple social media, that makes two problems for moz crawl: Overdynamic URLs Too long URLs Now what would be a good solution for this 2 problems, would a canonical to the original page before the "?" would be ok? Example:
Technical SEO | | JoaoCJ
http://urbania.pe/buscar/venta-de-propiedades?bathroomsNumber=2&services=gas&commonAreas=solarium The problem I have with this solution is that I also have a pagination parameter (page=2), and I'm using prev and next tags, if I use a such canonical will break the prev and next tag? http://urbania.pe/buscar/venta-de-propiedades?bathroomsNumber=2&services=gas&commonAreas=solarium&page=2 Also thinking if adding a noindex on pages with paramters could also be an option. Thanks a lot, I'm trying to address this issues.0 -
Assistance with High Priority Duplicate Page Content Errors
Hi I am trying to fix the high priority duplicate content URL's from my recent MOZ crawl (6 URL's) in total. Would someone from the community be able to offer some web development advice? I had reached out on the Moz Community on the main welcome page. Samantha stated that someone in web development on Moz's Q&A forum would be better suited to assist me. I took a word press class on Lynda.com, but other than that, I am a novice. I manage my site www.rejuvalon.com on Go Daddy's managed wordpress site. Thanks so much for your help! Best, Jill
Technical SEO | | justjilly0 -
Development Website Duplicate Content Issue
Hi, We launched a client's website around 7th January 2013 (http://rollerbannerscheap.co.uk), we originally constructed the website on a development domain (http://dev.rollerbannerscheap.co.uk) which was active for around 6-8 months (the dev site was unblocked from search engines for the first 3-4 months, but then blocked again) before we migrated dev --> live. In late Jan 2013 changed the robots.txt file to allow search engines to index the website. A week later I accidentally logged into the DEV website and also changed the robots.txt file to allow the search engines to index it. This obviously caused a duplicate content issue as both sites were identical. I realised what I had done a couple of days later and blocked the dev site from the search engines with the robots.txt file. Most of the pages from the dev site had been de-indexed from Google apart from 3, the home page (dev.rollerbannerscheap.co.uk, and two blog pages). The live site has 184 pages indexed in Google. So I thought the last 3 dev pages would disappear after a few weeks. I checked back late February and the 3 dev site pages were still indexed in Google. I decided to 301 redirect the dev site to the live site to tell Google to rank the live site and to ignore the dev site content. I also checked the robots.txt file on the dev site and this was blocking search engines too. But still the dev site is being found in Google wherever the live site should be found. When I do find the dev site in Google it displays this; Roller Banners Cheap » admin dev.rollerbannerscheap.co.uk/ A description for this result is not available because of this site's robots.txt – learn more. This is really affecting our clients SEO plan and we can't seem to remove the dev site or rank the live site in Google. In GWT I have tried to remove the sub domain. When I visit remove URLs, I enter dev.rollerbannerscheap.co.uk but then it displays the URL as http://www.rollerbannerscheap.co.uk/dev.rollerbannerscheap.co.uk. I want to remove a sub domain not a page. Can anyone help please?
Technical SEO | | SO_UK0 -
Duplicates on the page
Hello SEOMOZ, I've one big question about one project. We have a page http://eb5info.com/eb5-attorneys and a lot of other similar pages. And we got a big list of errors, warnings saying that we have duplicate pages. But in real not all of them are same, they have small differences. For example - you select "State" in the left sidebar and you see a list on the right. List on the right panel is changing depending on the what you selecting on the left. But on report pages marked as duplicates. Maybe you can give some advices how to improve quality of the pages and make SEO better? Thanks Igor
Technical SEO | | usadvisors0 -
How do I fix this type of duplicate page content problem?
Sample URLs with this Duplicate Page Content URLs Internal Links External Links Page Authority Linking Root Domains http://rogerelkindlaw.com/index.html 30 0 26 1 http://www.rogerelkindlaw.com/index.html 30 0 20 1 http://www.rogerelkindlaw.com/ | 1,630 | 613 | 43 | 110 | As you can see there are three duplicate pages; http://rogerelkindlaw.com/index.html http://www.rogerelkindlaw.com/index.html http://www.rogerelkindlaw.com/ What would be the best and most efficient way to fix this problem and also how to prevent this from happening? Thank you.
Technical SEO | | brianhughes0 -
Does turning website content into PDFs for document sharing sites cause duplicate content?
Website content is 9 tutorials published to unique urls with a contents page linking to each lesson. If I make a PDF version for distribution of document sharing websites, will it create a duplicate content issue? The objective is to get a half decent link, traffic to supplementary opt-in downloads.
Technical SEO | | designquotes0 -
Are all duplicate pages bad?
I just got my first Crawl Report for my forum and it said I have almost 9,000 duplicate pages. When I looked at a sample of them though I saw that many of them were "reply" links. By this I mean the "reply" button was clicked for a topic yet since the crawler was not a member, it just brought them to the login/register screen. Since all the topics would bring you to the same login page I'm assuming it counted all these "reply" links as duplicates. Should I just ignore these or is there some way to fix it? Thanks in advance.
Technical SEO | | Xee0 -
Duplicate Page Issue
Dear All, I am facing stupid duplicate page issue, My whole site is in dynamic script and all the URLs were in dynamic, So i 've asked my programmer make the URLs user friendly using URL Rewrite, but he converted aspx pages to htm. And the whole mess begun. Now we have 3 different URLs for single page. Such as: http://www.site.com/CityTour.aspx?nodeid=4&type=4&id=47&order=0&pagesize=4&pagenum=4&val=Multi-Day+City+Tours http://www.tsite.com/CityTour.aspx?nodeid=4&type=4&id=47&order=0&pagesize=4&pagenum=4&val=multi-day-city-tours http://www.site.com/city-tour/multi-day-city-tours/page4-0.htm I think my programmer messed up the URL Rewrite in ASP.net(Nginx) or even didn't use it. So how do i overcome this problem? Should i add canonical tag in both dynamic URLs with pointing to pag4-0.htm. Will it help? Thanks!
Technical SEO | | DigitalJungle0