How to remove duplicate content issues for thin page(containing oops no resulting found)
-
In this scenarios we have multiple different URLs but the page content is rendering same (containing Oops message) due to which content duplicate's issue arises.As soon as content for these URL,s are available then those pages duplicate issue will removed. So we want to remove duplicate issue not the page & Page URLs.
-
Hi,
If you simply want to stop these pages from being indexed until they have some non-duplicate content on them, you could use the meta robots tag in your
This is how that would look:
<meta name="robots" content="<strong>noindex</strong>"></meta name="robots">
And this will immediately drop these pages from the index when googlebot crawls a page with this directive in the
If you wish to stop these pages being crawled by rogerbot (moz’s crawler) so the duplicate content errors do not show up, you could add these lines to your robots.txt:
User-agent: rogerbot
Disallow: /oopspage1.html
Disallow:/oops/page2.php
User-agent: *
Allow: /This will still allow googlebot to crawl these pages, and will keep them in the index, so when you do update the content you won’t have any problems if you forget to remove the meta robots tag on these pages.
Be careful with robots.txt and remember that Disallow:/oops/ will stop crawlers from crawling subpages of the /oops/ folder, so you need to remember your filenames. But in this case, since you’re only setting these rules for rogerbot, it wouldn’t be catastrophic for your site but it’s best to take care either way.
In reality though, duplicate content is a problem when multiple pages are competing with each other for the same terms. Sometimes the wrong version of the page will be displayed above your preferred version. Unless you’re looking to rank for your ‘Oops nothing here’ phrase, these duplicate content problems are nothing to worry about and the notices will soon disappear from your crawl errors when you create some fresh content for them.
Hope this helps,
Tom
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Newbie SEO ?: Does my About page URL have to contain the word About?
New to WordPress and SEO. Built and launched my website last week. The URL was originally domain/about. However, I installed Yoast plugin and it told me "about" was a stop word. So, without too much thought (my first problem), I changed the url (before Google crawled me) to clearwingcommunications.com/storytelling. Since then, I've noticed that sites I know are optimized have their URL with the word "about." So, is this considered a bad practice? My site HAS been crawled at this point. If I change it back to About and do a 301 redirect, does that hurt reporting? Thanks for your help! Christy
On-Page Optimization | | christyr0 -
Duplicate content issue, across site domains (blogging)
Hi all, I've just come to learn that a client has been cross-posting their blog posts to other blogs (on higher quality domains, in some cases). For example - this is the same post on 3 different blogs. http://thebioethicsprogram.wordpress.com/2014/06/30/how-an-irb-could-have-legitimately-approved-the-facebook-experiment-and-why-that-may-be-a-good-thing/
On-Page Optimization | | ketanmv
http://blogs.law.harvard.edu/billofhealth/2014/06/29/how-an-irb-could-have-legitimately-approved-the-facebook-experiment-and-why-that-may-be-a-good-thing/
http://www.thefacultylounge.org/2014/06/how-an-irb-could-have-legitimately-approved-the-facebook-experimentand-why-that-may-be-a-good-thing.html
And, sometimes a 4th time, on an NPR website. I'm assuming this is doing no one any favors and Harvard or NPR is going to earn the rank most every time. I'm going to encourage them to publish only fresh content on their real blog, would you agree? Can this actually harm the ranking of their blog and website - should we delete the old entries when migrating the blog? They are going to move their Wordpress Blog to hosting on their real domain soon:
http://www.bioethics.uniongraduatecollege.edu/news/ The current set up is not adding any value to their domain. Thank you for any advice! Ketan0 -
How to solve duplicate content issue???
I have 5 websites with different domain names, every website have same content, same pages, same website design. Kindly let me know how to solve this issue.
On-Page Optimization | | ross254sidney0 -
Is there anything wrong with having duplicate description tags if they are relevant to their pages?
I have duplicate description tags, but they make sense for the pages they're on. Is there anything wrong with this? Thanks for reading!
On-Page Optimization | | DA20130 -
Duplicate Page Content Should we 301 - Best Practices?
What would be the best way to avoid a Duplicate Page Content for these type of pages. Our website generates user friendly urls, for each page..
On-Page Optimization | | 365ToursSafaris
So it is the same exact page, just both versions of the url work.. Example: http://www.safari365.com/about-africa/wildebeest-migration http://www.safari365.com/wildebeest-migration I don't think adding code to the page will work because its the same page for the incorrect and correct versions of the page. I don't think i can use the URL parameter setting because the version with /about-africa/ is the correct (correct as it it follows the site navigation) I was thinking of using the htaccess to redirect to the correct version.. Will that work ? and does it follow best Practices ? any other suggestions that would work better ?0 -
Is this duplicate content okay?
We have a client who wants to rank locally, nationally and internationally for their products. I wrote a line that goes, "We can ship our products to you whether you’re here in Illinois, nationwide, or international." I added that line after a paragraph or two of unique product description on each of their 30-odd product pages. Will this damage their ranking? I tried researching this but only found full page duplicate content topics. Any advice would be great.
On-Page Optimization | | optimalwebinc0 -
Creating Duplicate Content on Shopping Sites
I have a client with an eCommerce site that is interested in adding their products to shopping sites. If we use the same information that is on the site currently, will we run into duplicate content issues when those same products & descriptions are published on shopping sites? Is it best practice to rewrite the product title and descriptions for shopping sites to avoid duplicate content issues?
On-Page Optimization | | mj7750 -
Duplicate page titles
We have search results pages on our site. They share the same page title, there is no real differentiator between the result pages, other than page 1, page 2 etc. How do we de-dup the titles? just add page 1/2/3 etc to the end of them?
On-Page Optimization | | lilibooz0