Moz Q&A is closed.
After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.
Duplicate content, although page has "noindex"
-
Hello,
I had an issue with some pages being listed as duplicate content in my weekly Moz report. I've since discussed it with my web dev team and we decided to stop the pages from being crawled. The web dev team added this coding to the pages <meta name='robots' content='max-image-preview:large, noindex dofollow' />, but the Moz report is still reporting the pages as duplicate content.
Note from the developer "So as far as I can see we've added robots to prevent the issue but maybe there is some subtle change that's needed here. You could check in Google Search Console to see how its seeing this content or you could ask Moz why they are still reporting this and see if we've missed something?"
Any help much appreciated!
-
@rj_dale have you added a rel=canonical tag to the page to make sure you're marking to Google the correct version of the page? Even if it doesn't have a duplicate page, add a self-referencing canonical tag and if you need any more help - speak to a freelance seo consultant.
-
Based on your description, it appears as though the page is already indexed in the search engine. This is being picked up by the Moz tool and being reported as duplicate content.
What you can try to do is - for the page that is flagged by the Moz tool as duplicate enter it in Google as the following
site:example.com/moz-flagged-duplicate-url
This will show if the page is already indexed.
If the page is indexed, I would recommend you to:
a) Review the two or more similar URLs flagged as duplicates and see if they are actual duplicates or a tool-based error. If it is a tool based error you can ignore the issue. If you think the pages are actual duplicates you can evaluate pages in terms of backlinks or incoming traffic and choose the preferred page version that should be indexed in search engines. Place a canonical tag of the preferred page version on all the pages flagged as duplicate.
b) Please remove the noindex tag. This is not the right technique to handle duplicate content for SEO results.
c) You would have to wait for Google to crawl and update results. You can promote your new page on social media, email marketing campaigns, or build backlinks to the page. With time search engines will pick up these changes and updates its indexed results.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Is it Ok to have multiple domains (separate website different content) rank for similar keywords?
Is it 'OK' to have multiple domains in the following instance? Does Google actively discourage multiple (but completely different sites) domains from the same company appearing in the search results for the same and or similar keywords if the content is slightly different? This is where the 'main site' has the details, and you can purchase product, and the second site is a blog site only. We are creating a separate content blogsite; which would be on a second domain that will be related to one portion of content on main site. They would be linking back and forth, or maybe the blog site would just link over to the main site so they can purchase said product. This would be a similar scenario to give you an idea of how it would be structured: MAIN SITE: describes a few products, and you can purchase from this site SECOND SITE, different domain: a blog site that contains personal experiences with one of the products. BOTH sites will be linked back and forth....or as mentioned maybe the blog site could just link over to the 'main site' Logo would be a modified version of the main logo and look and feel of the sight would be similar but not exactly the same. MORE INFO: the main site has existed for way over 10 years, starting to gain some traction in an extremely competitive market, but does not rank super high, is gaining traction due to improvements in speed, content, onpage SEO, etc... So in addition to my main question of is this 'ok' to have this second domain, also will it hurt the rankings or negatively affect the 'main' site? Wondering about duplicate content issues, except it will be slightly different...
SEO Tactics | | fourwhitesocks0 -
Duplicate Content and Subdirectories
Hi there and thank you in advance for your help! I'm seeking guidance on how to structure a resources directory (white papers, webinars, etc.) while avoiding duplicate content penalties. If you go to /resources on our site, there is filter function. If you filter for webinars, the URL becomes /resources/?type=webinar We didn't want that dynamic URL to be the primary URL for webinars, so we created a new page with the URL /resources/webinar that lists all of our webinars and includes a featured webinar up top. However, the same webinar titles now appear on the /resources page and the /resources/webinar page. Will that cause duplicate content issues? P.S. Not sure if it matters, but we also changed the URLs for the individual resource pages to include the resource type. For example, one of our webinar URLs is /resources/webinar/forecasting-your-revenue Thank you!
Technical SEO | | SAIM_Marketing0 -
"Noindex, follow" for thin pages?
Hey there Mozzers, I have a question regarding Thin pages. Unfortunately, we have Thin pages, almost empty to be honest. I have the idea to ask the dev team to do "noindex, follow" on these pages. What do you think? Has someone faced this situation before? Will appreciate your input!
Technical SEO | | Europarl_SEO_Team0 -
Duplicate Content on a Page Due to Responsive Version
What are the implications if a web designer codes the content of the site twice into the page in order to make the site responsive? I can't add the url I'm afraid but the H1 and the content appear twice in the code in order to produce both a responsive version and a desktop version. This is a Wordpress site. Is Google clever enough to distinguish between the 2 versions and treat them individually? Or will Google really think that the content has been repeated on the same page?
Technical SEO | | Wagada0 -
How to deal with duplicated content on product pages?
Hi, I have a webshop with products with different sizes and colours. For each item I have a different URL, with almost the same content (title tag, product descriptions, etc). In order to prevent duplicated content I'am wondering what is the best way to solve this problem, keeping in mind: -Impossible to create one page/URL for each product with filters on colour and size -Impossible to rewrite the product descriptions in order to be unique I'm considering the option to canonicolize the rest of de colours/size variations, but the disadvantage is that in case the product is not in stock it disappears from the website. Looking forward to your opinions and solutions. Jeroen
Technical SEO | | Digital-DMG0 -
Duplicate Content Issues on Product Pages
Hi guys Just keen to gauge your opinion on a quandary that has been bugging me for a while now. I work on an ecommerce website that sells around 20,000 products. A lot of the product SKUs are exactly the same in terms of how they work and what they offer the customer. Often it is 1 variable that changes. For example, the product may be available in 200 different sizes and 2 colours (therefore 400 SKUs available to purchase). Theese SKUs have been uploaded to the website as individual entires so that the customer can purchase them, with the only difference between the listings likely to be key signifiers such as colour, size, price, part number etc. Moz has flagged these pages up as duplicate content. Now I have worked on websites long enough now to know that duplicate content is never good from an SEO perspective, but I am struggling to work out an effective way in which I can display such a large number of almost identical products without falling foul of the duplicate content issue. If you wouldnt mind sharing any ideas or approaches that have been taken by you guys that would be great!
Technical SEO | | DHS_SH0 -
Can iFrames count as duplicate content on either page?
Hi All Basically what we are wanting to do is insert an iframe with some text on onto a lot of different pages on one website. Does google crawl the content that is in an iFrame? Thanks
Technical SEO | | cttgroup0 -
How to prevent duplicate content at a calendar page
Hi, I've a calender page which changes every day. The main url is
Technical SEO | | GeorgFranz
/calendar For every day, there is another url: /calendar/2012/09/12
/calendar/2012/09/13
/calendar/2012/09/14 So, if the 13th september arrives, the content of the page
/calendar/2012/09/13
will be shown at
/calendar So, it's duplicate content. What to do in this situation? a) Redirect from /calendar to /calendar/2012/09/13 with 301? (but the redirect changes the day after to /calendar/2012/09/14) b) Redirect from /calendar to /calendar/2012/09/13 with 302 (but I will loose the link juice of /calendar?) c) Add a canonical tag at /calendar (which leads to /calendar/2012/09/13) - but I will loose the power of /calendar (?) - and it will change every day... Any ideas or other suggestions? Best wishes, Georg.0