Google Bot Noindex
-
If a site has the tag, can it still be flagged for duplicate content?
-
You can prevent Google from crawling the site through the robots.txt rather than the .htaccess Will.
-Andy
-
Cheers for the response Andy.
I do mean from a penalty point of view.
Can I block Google from crawling the site using the htaccess or robots.txt files?
I want the page to be indexed in all search engines but Google
I'm just experimenting and I don't mind whether the site with the duplicated content gets a penalty, but I would very much mind if the site whether the original content came from was given a penalty.
Will
-
Do you mean from a penalty point of view?
Google can still crawl and see what is on a page with a noindex flag, but it just won't index it. It doesn't guarantee that duplicate content won't still be an issue for you. This is one of those areas when it comes to Panda that people try. Some just noindex pages and have success with them, others need to remove them altogether.
-Andy
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Ranking Drop and Google Disavow Requests
My website, www.nile-cruises-4u.co.uk has fallen dramatically for the top industry search terms (nile cruise, nile cruises) over the last 12 months from previous page one rankings to page three which has very badly affected us financially. I found, using Linkdetox, that we had thousands of back-links for non-related anchor-text, mainly porn terms, viagra, etc. I have submitted a Disavow file and request about a week ago and wondered firstly if the enormous amount of these links would have helped cause the drop to page three and secondly if the Disavow request will eventually help the website return to better rankings? Thanks,Colin
Technical SEO | | GratefulFred0 -
Dev Site Was Indexed By Google
Two of our dev sites(subdomains) were indexed by Google. They have since been made private once we found the problem. Should we take another step to remove the subdomain through robots.txt or just let it ride out? From what I understand, to remove the subdomain from Google we would verify the subdomain on GWT, then give the subdomain it's own robots.txt and disallow everything. Any advice is welcome, I just wanted to discuss this before making a decision.
Technical SEO | | ntsupply0 -
Why my site is not indexing in google
In google webmaster i have updated my sitemap in Mar 6th..There is around 22000 links..But google fetched only 5300 links for long time...
Technical SEO | | Rajesh.Chandran
I waited for 1 month till no improvement in google index..So apr6th we have uploaded new sitemap (1200 links totally)..,But only 4 links indexed in google ..
why google not indexing my urls? Is this affect our ranking in SERP? How many links are advisable to submit in sitemap for a website?0 -
Why has Google stopped indexing my content?
Mystery of the day! Back on December 28th, there was a 404 on the sitemap for my website. This lasted 2 days before I noticed and fixed. Since then, Google has not indexed my content. However, the majority of content prior to that date still shows up in the index. The website is http://www.indieshuffle.com/. Clues: Google reports no current issues in Webmaster tools Two reconsideration requests have returned "no manual action taken" When new posts are detected as "submitted" in the sitemap, they take 2-3 days to "index" Once "indexed," they cannot be found in search results unless I include url:indieshuffle.com The sitelinks that used to pop up under a basic search for "Indie Shuffle" are now gone I am using Yoast's SEO tool for Wordpress (and have been for years) Before December 28th, I was doing 90k impressions / 4.5k clicks After December 28th, I'm now doing 8k impressions / 1.3k clicks Ultimately, I'm at a loss for a possible explanation. Running an SEOMoz audit comes up with warnings about rel=canonical and a few broken links (which I've fixed in reaction to the report). I know these things often correct themselves, but two months have passed now, and it continues to get progressively worse. Thanks, Jason
Technical SEO | | indieshuffle0 -
Google changing title tags to branded
Hi guys and gals, i noticed in the results today Google has been changing my title tags so that brand: is in the front of it all. This is not in the code but my title tag structure site-wide is keywords and then always | brand on the end. Why has this happened does anyone know? does it affect my first keyword in the title tag, i guess Google might do it as it recognises the brand and wants users to know it?
Technical SEO | | pauledwards0 -
Why are Google search results different if you are log'd into Google or not?
I get different results when I'm log'd into my Google account associated with my website than if I'm not. The same country is occurring. So how can I rely on the google results I'm seeing? For instance my site is page 1 with the improvements I made based on SEOMOZ if I'm log'd in. Yet I'm not on the first 25 pages if I'm not logged in.
Technical SEO | | Romana0 -
Noindex Pages indexed
I'm having problem that gogole is index my search results pages even though i have added the "noindex" metatag. Is the best thing to block the robot from crawling that file using robots.txt?
Technical SEO | | Tedred0 -
Google Reconsideration Request (Penguin) - Will Google give links to remove?
When Penguin v1 hit, our site took a hit for a single phrase (i.e. "widgets") due to the techniques our SEO company was using (network). We've since had those links cleaned up, and our rankings have not recovered. Our SEO company said they submitted a reconsideration request on our behalf, and that Google denied it and didn't provide which links we needed removed. Does Google list links that need removing if they are still not happy with your link profile?
Technical SEO | | crucialx0