Google Bot Noindex
-
If a site has the tag, can it still be flagged for duplicate content?
-
You can prevent Google from crawling the site through the robots.txt rather than the .htaccess Will.
-Andy
-
Cheers for the response Andy.
I do mean from a penalty point of view.
Can I block Google from crawling the site using the htaccess or robots.txt files?
I want the page to be indexed in all search engines but Google
I'm just experimenting and I don't mind whether the site with the duplicated content gets a penalty, but I would very much mind if the site whether the original content came from was given a penalty.
Will
-
Do you mean from a penalty point of view?
Google can still crawl and see what is on a page with a noindex flag, but it just won't index it. It doesn't guarantee that duplicate content won't still be an issue for you. This is one of those areas when it comes to Panda that people try. Some just noindex pages and have success with them, others need to remove them altogether.
-Andy
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Is the Google results serp broken?
Hi everyone! We've been trying to decipher how many of our pages are indexed by google at the moment. If we do the usual "site:https://www.hobbydb.com" search term, the serp says that we have more than 740,000 pages indexed. However, when I do a deep dive and click through to the last page of results, I can only get to page 54, and then there are no more results. This would mean that I only have 540 pages indexed, not 740,000. We have also done other queries for other sub-sections of our website, and the results also truncate at 50 pages. Has anyone run into this problem? Any suggestions are appreciated! Best, Alex
Technical SEO | | mpchobbydb0 -
Google not crawling the website from 22nd October
Hi, This is Suresh. I made changes to my website and I see that google is unable to crawl my website from 22nd October. Even it is not showing any content when I use Cache:www.vonexpy.com. Can any body help me in knowing why Google is unable to crawl my website. Is there any technical issue with the website? Website is www.vonexpy.com Thanks in advance.
Technical SEO | | sureshchowdary1 -
Google is not respecting the meta title
We're experiencing a peculiar situation with Google not respecting our meta <title>.</p> <p>As you can see in the first image (search result), the title <a href="http://open.iebschool.com/profesores/startups/">for the page</a> is a part of the content. This is relatevely normal for the description, but we never heard of Google doing this before.</p> <p>In the code, the <title> and meta description are correctly implemented.</p> <blockquote style="background-color: #f7f7f7; padding-top: 5px; margin-left: 0px; padding-left: 2px; padding-bottom: 5px; white-space: nowrap; overflow-y: auto; font-family: monospace; background-position: initial initial; background-repeat: initial initial;"> <p><meta name="description" content="Profesores, tutores, autores y docentes 2.0 de Open IEBS. Conoce su Biografía, experiencia, reputación, conexiones sociales y las valoraciones de alumnos."/><br /><title>Conoce los profesores, tutores, autores y docentes de Open IEBS.</title> In a further research, we discovered that the title which is using is an in anwith the following code (cleaned and simplified for the question): <hgroup> Pilar Soro
Technical SEO | | ofuente
0 Seguidor
Para poder seguir al Profesor, debes de registrarte aquí. Profesora y experta en redes sociales. Formadora de docentes, [...]
</hgroup> Note: we're correcting the code since this is quite messy, but it's the one we have now The point is that google has considered that this particular is more important than the title itself. This would make sense if we were looking for that name, but the search was simply "site:domain.com". Two things for which this is even more strange are the following: while all the /profesor/%category%/ has the same code, this only happens in some search results and not in all of them; why is it appearing in some pages, but respecting my title in others? the previous code is not the only one in the page, there are about 10 others and some are placed before and some are placed after; so, why this one and not the first or the last? What is more strange is why this article in particular and not any other of the 10 on the page since some of them are placed before and some of them are placed after. Provided this situation, we would like to know: is this a common situation? Is it happening to more people? why is it happening? Is it somehow related to , <hgroup>and ? why that piece of code and not any other article? and why is it only happening in some pages? more important, can it be corrected or can we take advantage of it somehow? Thank you in advance. Any light you can shed on this will be well received! AJ2CUSe.png?1?8232 </hgroup>0 -
Google does not show my website anymore
Hi All, We developed a new website for the domain: www.instral.com. Before we build the webste, the domain was indexed by google and was showed as first result on a search "Instral". Without a website! Only a super simple hostingprovider webpage. Now, with the new website, all the website pages are found in google from page 7... home page is not even in the results. When i search on "instral.com" in google, it will show my website on first result including homepage. Is there something wrong with the website or DNS settings? Or mabe some other webhosting setting... am i on a blacklist or something? Bing and Yahoo are showing better results (first page). I hope someone can help me out here...
Technical SEO | | extrememedia0 -
Google indexing tags help
Hey everyone, So yesterday someone pointed out to me that Google is indexing tags and that will likely hurt search engine results. I just did a "site:thetechblock.com" and I notice that tags are still being pulled. http://d.pr/i/WmE6 Today, I went into my Yoast settings and checked "noindex,follow" tags in the Taxomomies settings. I just want to make sure what I'm doing is right. http://d.pr/i/zmbd Thanks guys
Technical SEO | | ttb0 -
Site being indexed by Google before it has launched
We are currently coming towards the end of a site migration, and are at the final stage of testing redirects etc. However, to our horror we've just discovered Google has started indexing the new site. Any ideas on how this could have happened? I have most recently asked for robots.txt to exclude anything with a certain parameter in URL. Is there a chance this, wrongly implemented, could have caused this?
Technical SEO | | Sayers0 -
Google Webmaster tools error?
So I am trying to set the URL preference in google webmaster tools for my site. However when I try to save it it tells me to verify that I own the site. I have already done this so where can I go to verify I own the site exactly? Maybe I am wrong and I have not done this already but even on the homepage of webmaster tools I don't see an option to "verify".
Technical SEO | | ENSO0 -
Google Quality Algorithm Update
I'm curious what correlations or impacting variables SEO professionals have found that have increased or decreased ranking with the most recent algorithm change. It appears that many innocent sites have fallen victim, especially larger sites. It also appears that Google is maintaining that specific sites were not targeted... Meaning there must be proven characteristics.
Technical SEO | | douglaskarr0