Does this site have a duplicate content issue?
-
Google WMT is showing me only 2 short meta descriptions under "HTML Improvements" but I believe http://www.customgia.com may have a content duplication issue. Numerous keywords are used repeatedly across many product descriptions. To make matters worse, every product page has a "Design It!" button that sends the user to a flash-based jewelry designer in which they can edit the product's appearance. I'm not sure if these "designer pages" are adding unnecessary and potentially damaging duplicate content but it's certainly a possibility.
There are many items on this site that are similar to one another but not the same. The product description tend to use the same phrases over and over again - words like crystal, Swarovski, beaded, design it, customize, change, pearl, glass beads, iridescent, pearl, drop earrings are used a lot. What I'm stuck on is whether or not I should be focusing on a content duplication issue as the primary SEO problem or if there is something bigger. Thank you for any assistance you can provide!
-
This is where things get a bit dicey - I'm not 100% sure that won't remove the main page, too (and how Google handles the trailing "/"). You might need a "/*" wild-card in the Robots.txt. Frankly, I'd ease into it with just one directory. These things never seem to work quite the way in practice that we all say they should in theory.
-
Okay, last question on this (I hope). As far as I can tell, Google's URL removal tool does not support the use of wildcards. And according to their removal requirements, I can't remove an entire directory unless that directory is already blocked in the robots.txt. So before I submit the removal request for: http://www.customgia.com/design-your-own-jewelry/classic-necklace/, I have to add: Disallow: /design-your-own-jewelry/classic-necklace/ to robots.txt. Is this the right way to do this? And again, thanks for helping me with this.
-
Oh, sorry - yeah, this is why these questions can be dangerous in the scope of Q&A. If some of the pages in that virtual folder are main nav pages/links, then it's definitely going to look weird to block them (it's a mixed signal, at best). I'm not sure I fully understand the site structure, but my gut reaction is to leave those indexed. The wild-cards should work - the other option would be to give them each their own shorter URLs and not put them in the "/design-your-own-jewelry" folder, but that can be a ton of work, depending on how your site is built (plus, you'd have to 301-redirect the old URLs, which opens up a whole new mess).
-
Thank you for the excellent advice. We put the META NOINDEX tags into place this morning. The URL removal request is next but I have a slight change to what Everett outlined above.
The six designer pages that are accessible from the homepage: /design-your-own-jewelry/classic-necklace, /design-your-own-jewelry/simple-necklace, /design-your-own-jewelry/classic-bracelet, /design-your-own-jewelry/simple-bracelet, /design-your-own-jewelry/pendant-necklace, /design-your-own-jewelry/drop-earrings are not duplicates and the Moz crawl did not show them as duplicates. All the other designer pages are considered duplicates of each other or duplicates of one of these six pages. So we put INDEX, NOFOLLOW on these six pages to keep them indexed. I think the removal request should follow suit.
What's your opinion on placing a removal request for each of the following? - /design-your-own-jewelry/classic-necklace/, /design-your-own-jewelry/simple-necklace/, /design-your-own-jewelry/classic-bracelet/, /design-your-own-jewelry/simple-bracelet/, /design-your-own-jewelry/pendant-necklace/, /design-your-own-jewelry/drop-earrings/. Correct me if I'm wrong, but this should remove all the designer pages from the index except the six that are accessible from the homepage, giving those six pages a chance to rank.
-
That sounds like a winning plan Dr. Pete, though I'd append 2.1 "Request removal of directory in Bing and Google webmaster tools".
-
I've had a lot of issues where, if pages were already indexed, Robots.txt did a poor job of removing them. Absolutely agree on the crawl budget issue and it's a whole lot easier to remove a folder in Robots.txt, but I've just had a bunch of odd problems with Robots.txt at large scale. If I actually had to do it on my own site, I'd probably:
(1) META NOINDEX the pages
(2) Monitor removal
(3) Once removal was progressing well (80%+), then add to Robots.txt
-
I agree with Dr. Pete here, though I think the easiest solution would be to simply block the entire /design-your-own-jewelry/* directory from being indexed using robots.txt and, to Dr. Pete's point, you'll want to remove that directory from the index in both Bing and Google webmaster tools, as discussed here:
http://googlewebmastercentral.blogspot.com/2007/04/requesting-removal-of-content-from-our.html (see the section under "entire directory")
Something I think about with regard to robots.txt block Vs meta robots block is crawl budget. Google has to access a page to see the meta noindex tag, while a single disallow statement in the robots.txt file can save Googlebot the hassle of visiting potentially thousands of unnecessary pages.
If down the road you figure out a way to put custom content on those pages and want to try and rank for things like "Custom Garnet Pearl Bracelet" or "Design Your Own Beaded Bracelet" then I'd look into some of the other options discussed here. Until then I feel they would just be complicating something as simple as the need to remove very thin, mostly duplicate content from the index.
-
Each of these problems may have a unique solution, so it gets complicated. Regarding the "design your own" pages, I'm seeing over 5K of those URLs in the search index, and they do probably look very similar. Since these are not the core product pages, I'd strongly consider using META NOINDEX on them. I find that Robots.txt does not do a good job of blocking content that has already been indexed, in most cases. You can add the meta tag dynamically in your code, hopefully, so that just a few lines of code will serve all of these pages.
While these pages aren't "true" duplicates, they look similar enough that, at the scale of your site, they really are diluting your ability to rank. In extreme cases, if you're also serving up product variations, paginated search results, etc., you could even run into Panda issues. Whether or not this is your core problem, from an SEO perspective, cleaning it up can't hurt, and may make it easier to find other problems.
-
Even if the 'design' part was not flash the textual content is pretty much identical. There is no benefit for it to be indexed so canonical to the main directory URL would make sense. Then add some good text to those main pages.
Personally I would only use H1 for user experience rather than keywordy as they don't carry much weight.
-
Yes, the page http://www.customgia.com/fashion-beaded-jewelry/shop-for/beaded-bracelets/classic-bracelet/AADE17E9BB964352B7A3912294BB5DF8 is a unique product page with a unique description. The designer page you referenced: http://www.customgia.com/design-your-own-jewelry/classic-bracelet/AADE17E9BB964352B7A3912294BB5DF8 is also unique because it loads the jewelry design in the product page.
On every product page the "Design It!" button opens a flash-based designer page that let's the user edit that particular design. Unfortunately, the Moz crawler (and I assume Google) considers these pages duplicates of http://www.customgia.com/design-your-own-jewelry/classic-bracelet/ (or one of the other five jewelry patterns). The fact that every designer page loads a unique jewelry design does not seem to matter. The best (and most costly) solution, I suppose would be to change all the flash code to html but that isn't happening anytime soon.
The
tag on all the designer pages is either "Classic Bracelet", "Classic Necklace", "Simple Necklace", "Simple Bracelet", "Pendant Necklace" or "Drop Earrings" (depending on which pattern was used to create the design). Maybe changing the
to something like: Redesigning "Garnet Pearl Bracelet with Silver Hearts" would help tell Google these pages indeed differ from one another - but I think the content below the
will still be considered duplicate. If I use canonical tags on these pages, is there any point in creating dynamic
tags if it doesn't improve the user experience? Thanks again!
-
On a brief view:
http://www.customgia.com/fashion-beaded-jewelry/shop-for/beaded-bracelets/classic-bracelet/AADE17E9BB964352B7A3912294BB5DF8 looks like it is a unique product page with potentially unique description? If so leave as is.
http://www.customgia.com/design-your-own-jewelry/classic-bracelet/AADE17E9BB964352B7A3912294BB5DF8 looks a duplicate of http://www.customgia.com/design-your-own-jewelry/classic-bracelet/ so I would set rel canonical on any http://www.customgia.com/design-your-own-jewelry/classic-bracelet/CODE page pointing to ...design-your-own-jewelry/classic-bracelet/
I hope this helps.
-
Initially I thought canonical tags would work best. If we use them, should the canonical tag for the page: http://www.customgia.com/design-your-own-jewelry/classic-bracelet/AADE17E9BB964352B7A3912294BB5DF8 point to the "parent" product page: http://www.customgia.com/fashion-beaded-jewelry/shop-for/beaded-bracelets/classic-bracelet/AADE17E9BB964352B7A3912294BB5DF8 or do you think it should point to the appropriate designer page: http://www.customgia.com/design-your-own-jewelry/classic-bracelet/? - (The design that appears on these pages is based on the six products that appear on the homepage.)
The first solution, I suppose, would pass authority/PageRank to the "parent" product page. Whereas the second solution would pass authority/PageRank to one of the six designer pages. I'm not sure which is a better solution but I'm favoring pointing the pages to it's "parent" product page.
The priority is to fix the duplication content issue but a bump in ranking for any of these pages is obviously a bonus. Thanks for your help!
-
If you add rules to robots.txt that does not mean those directories will be removed from the index. You will also need to remove them in Webmaster Tools >> Google Index >> Remove URLs >> set Reason to Remove Directory.
Having said that why not use canonical tags on pages like /design-your-own-jewelry/classic-bracelet/AADE17E9BB964352B7A3912294BB5DF8 - http://moz.com/blog/rel-confused-answers-to-your-rel-canonical-questions
-
You are so welcome, and as I said, technical SEO is something I've been thrust into learning because of my current situation as an in-house SEO.
I think you may be on the right track, but there are other, very talented technical SEOs here who I would ask for a second on your decision. Perhaps Dr. Pete, Ian Lurie, or Everett Sizemore could chime in with a much more accurate response on the robots.txt
Good luck!
-
Thanks for the quick response. The Crawl Diagnostics report I just rec'd identified all the designer pages as duplicate content (these are the pages that load when the "Design It!" button is clicked).
The current robots.txt file does not disallow the designer pages. There are six different types of designer pages. Each of the 967 products loads one type of designer page, depending on the jewelry pattern the item was created with. Do you think the correct solution is to disallow these designer pages in the robot.txt file? I think the new robot.txt file should look like this:
User-agent: *
Disallow: /admin/
Disallow: /design-your-own-jewelry/classic-bracelet/
Disallow: /design-your-own-jewelry/classic-necklace/
Disallow: /design-your-own-jewelry/drop-earrings/
Disallow: /design-your-own-jewelry/pendant-necklace/
Disallow: /design-your-own-jewelry/simple-bracelet/
Disallow: /design-your-own-jewelry/simple-necklace/Your help with this is very much appreciated.
-
First off, welcome!
Second, I would say don't worry at all about your call to action button. Every eCommerce site has call to action buttons on every page (i.e. "Add to Cart"). The pages that happen after that don't matter to search engines with one caveat...just make sure you have a properly configured Robots.txt file.
Third. If all of your product pages are indeed trying to capitalize on the same key terms....yes, you have a duplicate content problem. Don't wait for a tool to tell you what you already know in your heart!
Take some time, and prioritize and start to re-write you product pages to use a wider variety of keywords (particularly long-tail) that better describe the products you offer.
I hope this helps!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Two Day After Starting Moz Pro campaign i experienced Sudden Huge Traffic Drop and My site ranki Drops in my site..Please I need your help
Two Day After Starting Moz Pro campaign i experienced Sudden Huge Traffic Drop and Drops in my site..Please I need your help
On-Page Optimization | | zizutz0 -
Duplicate Content for Event Pages
Hi Folks, I have event pages for specific training courses running on certain dates, the problem I have is that MOZ indicates that I have 1040 duplicate content issues because I'm serving pages like this https://purplegriffon.com/event/2521/mop-practitioner I'm not sure how best to go about resolving this as, of course, although each event is unique in terms of it's start date, the courses and locations could be identical. Will Google penalise us for these types of pages, or will they even index them? Should I add a canonical link to the head of the document pointing to the related course page such as https://purplegriffon.com/courses/project-management/mop-management-of-portfolios/mop-practitioner. Will this solve the issue? I'm a little stuck on what to do for the best. Any advice would be much appreciated. Thanks. Kind Regards Gareth Daine
On-Page Optimization | | PurpleGriffon0 -
Duplicate pages
Hi I have recently signed up to Moz Pro and the first crawl report on my wordpress site has brought up some duplicate content issues. I don't know what to do with this data! The original page : http://www.dwliverpoolphotography.co.uk/blog/ and the duplicate content page : http://www.dwliverpoolphotography.co.uk/author/david/ If anyone can point me to a resource or explain what I need to do thanks! David.
On-Page Optimization | | WallerD0 -
Content with changing URL and duplicate content
Hi everyone, I have a question regarding content (user reviews), that are changing URL all the time. We get a lot of reviews from users that have been dining at our partner restaurants, which get posted on our site under (new) “reviews”. My worry however is that the URL for these reviews is changing all the time. The reason for this is that they start on page 1, and then get pushed down to page 2, and so on when new reviews come in. http://www.r2n.dk/restaurant-anmeldelser I’m guessing that this could cause for serious indexing problems? I can see in google that some reviews are indexed multiple times with different URLs, and some are not indexed at all. We further more have the specific reviews under each restaurant profile. I’m not sure if this could be considered duplicate content? Maybe we should tell google not to index the “new reviews section” by using robots.txt. We don’t get much traffic on these URLs anyways, and all reviews are still under each restaurant-profile. Or maybe the canonical tag can be used? I look forward to your input. Cheers, Christian
On-Page Optimization | | Christian_T2 -
Index Page Content
Mozers, I am of the believe and as a person who puts the utmost emphasis on the index page of any website I am trying to rank, especially with a new domain ... insuring content is relevant, structured, optimized and we have some link juice flowing in. I find once we get the index page ranked, Google's little bots then start to index and rank accordingly the rest of the website ... and we start producing results. We also develop websites (dare I say its where we expertise in) and unexpectantly the client has asked us to carry out SEO work additionally to their web development. Problem lies here, their index page, has absolutely no written content at all, just one large image with a logo (Fashion Website) ...Which I identify as a huge issue as per my explanation is paragraphs one or two. I am sure withe the many more qualified SEO experts and gurus within the SEOmoz community, you have also come across this issue So a few questions, if you don't mind adding advice. 1 - Am I putting too much emphasize on content within the index page, in terms of indexing and actually ranking ...yes I appreciate that terms within the website will be ranked against other pages other than the index page, but will it harm us for having no content at all within the index page 2 - If so, and yes is the answer to above, how do we handle it, we have spoke with the client and he is pretty adamant that he want the index page as is, he has been through out the whole website building process. As suggested, any advice would be really appreciated, its a difficult market to rank within a it is, and i can only see this index page making the task a lot more difficult Cheers John
On-Page Optimization | | Johnny4B0 -
Is my blog simply duplicate content of my authors' profiles?
www.example.com/blog is the full list of blog posts by various writers. The list contains the title of each article and the first paragraph from the article. In addition to /blog being indexed, each author's contribution list is being indexed separately. It's not a profile, really, just a list of articles in the same title & paragraph format of the /blog page. So if /blog a list of 10 articles written by two writers, I have three pages: /blog/author1 is a list of 4 articles /blog/author2 is a list of 6 different articles /blog is a list of 10 articles (the 4+6 from the two writers) Is this going to be considered duplicate content?
On-Page Optimization | | Brocberry0 -
Archive of content
Hi there, I have recently joined a company to look after the e-marketing side of things, anyway the company I work for have been writing articles for a website that they own for over 2 years, probably about 200 or so unique articles on that website, however over the past year or so there has been no contribution to this site and was wondering if it would be worthwhile transferring these article over to our blog?, as this is where all the attention is in terms of marketing etc Kind Regards,
On-Page Optimization | | Paul780 -
User experience regarding dulpicate content and managing this content with google.
Hi long title i know! We are moving on to magento and have chosen to allocate a specific colour to each category using corresponding tabbed navigation for user experience.All products within each of the coloured tabs then inherit the repective colour, giving the products a category identiy within the store. This layout has had a positive feedback from our "testers" As a lot of our products are seasonal and can be represented in different categories there is a significant amount of duplicate content. ATM i see our options as being: Alter the site structure so that the category is not shown in the url, therefore eliminating our duplicate products. The downside of this is that the colour co-ordination of the categories would not work at product level as its the category path that assigns the colour. create canonical links for every duplicate, can this be damaging? keep the duplicates and do nothing let google decide the most important version of a product. any guidance would be appreciated!
On-Page Optimization | | LadyApollo0