Google Bot Noindex
-
If a site has the tag, can it still be flagged for duplicate content?
-
You can prevent Google from crawling the site through the robots.txt rather than the .htaccess Will.
-Andy
-
Cheers for the response Andy.
I do mean from a penalty point of view.
Can I block Google from crawling the site using the htaccess or robots.txt files?
I want the page to be indexed in all search engines but Google
I'm just experimenting and I don't mind whether the site with the duplicated content gets a penalty, but I would very much mind if the site whether the original content came from was given a penalty.
Will
-
Do you mean from a penalty point of view?
Google can still crawl and see what is on a page with a noindex flag, but it just won't index it. It doesn't guarantee that duplicate content won't still be an issue for you. This is one of those areas when it comes to Panda that people try. Some just noindex pages and have success with them, others need to remove them altogether.
-Andy
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google Cache issue
Hi, We’ve got a really specific issue – we have an SEO team in-house, and have had numerous agencies look at this – but no one can get to the bottom of this. We’re a UK travel company with a number of great positions on the search engines – our brand is www.jet2holidays.com. If you try ‘Majorca holidays’, ‘tenerife holidays’, ‘gran canaria holidays’ etc you’ll see us in the top few positions on Google when searching from the UK. However, none of our destination pages (and it’s only the destination pages), show a ‘cached’ option next to them. Example: https://www.google.com/search?q=majorca+holidays&oq=majorca+holidays&aqs=chrome..69i57j69i60l3.2151j0j9&sourceid=chrome&ie=UTF-8 This isn’t affecting our rankings, but we’re fairly certain it is affecting our ability to be included in the Featured Snippets. Checked and there aren’t any noarchive tags on the pages, example: https://www.jet2holidays.com/destinations/balearics/majorca Anyone have any ideas?
Technical SEO | | fredgray0 -
Google PageSpeed Insights Error
Google PageSpeed Insights showing this "An error occurred while fetching or analyzing the page.Dismiss" URL: http://westpalmbailbonds.com/ What should I do to fix this? Thanks in Advance
Technical SEO | | Beachflower0 -
Why does my Google Web Cache Redirects to My Homepage?
Why does my Google Webcache appears in a short period of time and then automatically redirects to my homepage? Is there something wrong with my robots.txt? The only files that I have blocked is below: User-agent: * Disallow: /bin/ Disallow: /common/ Disallow: /css/ Disallow: /download/ Disallow: /images/ Disallow: /medias/ Disallow: /ClientInfo.aspx Disallow: /*affiliateId* Disallow: /*referral*
Technical SEO | | Francis.Magos0 -
Google not returning an international version of the page
I run a website that duplicates some content across international editions. These are differentiated by the country codes e.g. /uk/folder/article1/ /au/folder/article1/ The UK version is considered the origin of the content. We currently use hreflang to differentiate content, however there is no actual regional or language variation between the content on these pages. Recently the UK version of a specific article is being indexed by Google as I am able to access via keyword search, however when I try to search for it via: site:domain.com/uk/folder/article1/then it is not displaying, however the AU version is. Identical articles in the same folder are not having this issue. There are no errors within webmaster tools and I have recently refetched the specific URL. Additionally when checking for internal links to the UK and AU edition of the article, I am getting internal links for the AU edition of the article however no internal links for the UK edition of the article. The main reason why this is problematic is because the article is now no longer appearing on the UK edition of the site for internal site search. How can I find out why Google is not getting a result when the URL is entered but it is coming up when doing a specific search?
Technical SEO | | AndDa0 -
From page 1th to page 18th @ Google
Hello Mozzers! I have a question, you may help.. How may it be possible that a page ranking well (1th result) goes from 1th result to the 18th page just in 1 day? It doesnt seem to be any kind of penalization.. I now had all suspicious outgoing links to be nofollow (they were not before), this may be a cause .. (?) Do you have any other suggestion? Thanks
Technical SEO | | socialengaged0 -
Noindex user profile
I have a social networking site with user- and company profiles. Some profiles have little to no content. One of the users here at moz suggested noindex-ing these profiles. I am still investigating this issue and have some follow up questions: What is the possible gain of no-indexing uninteresting profiles? Especially interested in this since these profiles do bring in long-tail traffic atm. How "irreversable" is introducing a noindex directive? Would everything "return to normal" if I remove te noindex directive? When determining the treshold for having profiles indexed, how should the following items be weighed Sum of number of words on the page (comprised of one or more of the following: full name, city, 0 to N company names, bio, activity) (unique) Profile picture (Nofollowed) Links to user's profiles on social networks or user's own site. Embedded Google Map Thanks!
Technical SEO | | thomasvanderkleij0 -
Having to type Google CAPTCHA all the time
Hi guys, Our office has about 15 computers all on the same IP address and about 10 actively search on Google. Recently we have been asked to type in CAPTCHA almost every single time searching on Google and would like to know if you have any suggestions of resolving this. We do use Firefox Rank Checker to check ranking once per week (around 400 keywords) but we use Hide My Ass to hide the IP. No malware or virus detected on computers in the network. Many thanks for your help in advance David
Technical SEO | | sssrpm0 -
Google Analtyics Changes?
Did anyone else's google analtyics numbers jump up this month? Our unique visitors increased some, but our pageviews tripled and our bounces dropped significantly. We have been working to optimize the sites and did have a huge drop in errors in our campaign reports.
Technical SEO | | Stevej240