Google Bot Noindex
-
If a site has the tag, can it still be flagged for duplicate content?
-
You can prevent Google from crawling the site through the robots.txt rather than the .htaccess Will.
-Andy
-
Cheers for the response Andy.
I do mean from a penalty point of view.
Can I block Google from crawling the site using the htaccess or robots.txt files?
I want the page to be indexed in all search engines but Google
I'm just experimenting and I don't mind whether the site with the duplicated content gets a penalty, but I would very much mind if the site whether the original content came from was given a penalty.
Will
-
Do you mean from a penalty point of view?
Google can still crawl and see what is on a page with a noindex flag, but it just won't index it. It doesn't guarantee that duplicate content won't still be an issue for you. This is one of those areas when it comes to Panda that people try. Some just noindex pages and have success with them, others need to remove them altogether.
-Andy
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Is the Google results serp broken?
Hi everyone! We've been trying to decipher how many of our pages are indexed by google at the moment. If we do the usual "site:https://www.hobbydb.com" search term, the serp says that we have more than 740,000 pages indexed. However, when I do a deep dive and click through to the last page of results, I can only get to page 54, and then there are no more results. This would mean that I only have 540 pages indexed, not 740,000. We have also done other queries for other sub-sections of our website, and the results also truncate at 50 pages. Has anyone run into this problem? Any suggestions are appreciated! Best, Alex
Technical SEO | | mpchobbydb0 -
Google Ecommerce Alerts
I recently started getting email notifications from Google re: new products on our websites. I am subscribed to Google alerts. Can anyone shed some light on this?
Technical SEO | | AMHC0 -
Google Publisher status
Hi all, I just wondered what the general opinion was with regard getting Google publisher status for medium to large organisations. Lots of our clients write a lot of articles & publications and it would be interesting to get some thoughts on how others view Authorship & in particular Publisher credentials. Thanks!
Technical SEO | | davidmaxwell0 -
"INDEX,FOLLOW" then later in the code "NOINDEX,NOFOLLOW" which does google follow?
background info: we have an established closed E-commerce system which the company has been using for years. I have only just started and reviewing the system, I don't have direct access to the code, but can request changes, but it could take months before the changes are in effect (or done at all), and we won't can't change to a new E-commerce system for the short to mid term. While reviewing the site (with help of seomoz crawl diagnostics) I noticed that some of the existing "landing pages" have in the code: <meta name="<a class="attribute-value">robots</a>" content="<a class="attribute-value">INDEX,FOLLOW</a>" /> then a few lines later <meta name="<a class="attribute-value">robots</a>" content="<a class="attribute-value">NOINDEX,NOFOLLOW</a>" /> Which the crawl diagnostics flagged up, but in the webmaster tools says
Technical SEO | | PaddyDisplays
"We didn't detect any issues with non-indexable content on your site." so the question is which instructions does google follow? the first or 2nd? note: clearly this is need fixed, but I have a big list of changes for the system so I need to know how important this is tthanks0 -
Google is indexing my directories
I'm sure this has been asked before, but I was looking at all of Google's results for my site and I found dozens of results for directories such as: Index of /scouting/blog/wp-includes/js/swfupload/plugins Obviously I don't want those indexed. How do I prevent Google from indexing those? Also, it only seems to be doing it with Wordpress, not any of the directories on my main site. (We have a wordpress blog, which is only a portion of the site)
Technical SEO | | UnderRugSwept0 -
Google News URL Format
Hi, We are currently redesigning our gaming website (www.totallygn.com) and one of our main goals is to get listed by Google News in future. Looking at the Google News URL requirements "The URL for each article must contain a unique number consisting of at least three digits." How does the above affect SEO structure? I was planning on using a format such as www.totallygn.com/xbox-360/360-reviews/fifa-12-review how would this compare to something like? www.totallygn.com/xbox-360/360-reviews/fifa-12-review234 Thanks in advance for your help
Technical SEO | | WalesDragon0 -
My site cannot be found by google at all
I don't know why but our company site can not be found by google at all. I have submitted to google webmaster, have social media point to, etc, Is there any reason for this? url for our website is www.bistosamerica.com Thank you
Technical SEO | | BistosAmerica0
