How do you de-index and prevent indexation of a whole domain?
-
I have parts of an online portal displaying in SERPs which it definitely shouldn't be. It's due to thoughtless developers but I need to have the whole portal's domain de-indexed and prevented from future indexing.
I'm not too tech savvy but how is this achieved? No index? Robots?
thanks
-
Fastest way to do it would be through a combination of Google Webmaster Tools and Robots.txt / noindex,nofollow meta tags.
Firstly if your domain hasn't been verified within GWT, go through that process. Then proceed with the remove URL request within GWT. (Image Attached)
You will need to upload a robots.txt and/or implement the noindex meta tag within the header of the pages you no longer want indexed.
Example Robots.txt:
User-agent: *
Disallow: /offendingportal/Example Robots Meta tag:
That should be enough to get rid of the offending content from within the SERPS and prevent it from being re-indexed.
-
Hi TCE,
There are two options in my opinion for this which are the best (as their are several other options including META robots etc):
- Robots.txt: Exclude your portal via your robots.txt file.
- HTTP Auth: Let your developers add HTTP Authentication so you have to log in to see the portal. Google won't be able to see the pages itself and so will de index them.
Hope this helps.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Have You 301 Redirected Domain A to Domain B ?
I only have two questions.... Approximately when did you do it (year is close enough)? Did the rankings of Domain B go up? Any other information that you care to share will be appreciated. Thank you!
Intermediate & Advanced SEO | | EGOL0 -
Domain switch planned - new domain accessible - until the switch: redirect from new to old domain with 307?
Hi there, We are going to switch our local domain oldsite.at to newsite.com in November. As our IT department wants to use the newsite.com already for email traffic till then, the domain newsite.com has to be accessible for public and currently shows the default Apache page without useful content. The old domain has quite some trust, the new domain is a first time registered domain (not known by search engines yet and no published anyhow). The domain was parked till now. I am aware of the steps to take for the switch itself, but: **what to do with the newsite.com domain until everything is prepared for the switch? **I suppose users or search engines find the domain and as there is no useful information available it harms us already. My idea was to 307 redirect newsite.com to the oldsite.at but the concern is that this causes problems as soon as we switch the domain and redirecting with 301 from oldsite.at to newsite.com? Do you have any objections or other recommendations? Thank you a lot in advance.
Intermediate & Advanced SEO | | comicron0 -
Why my site it's not being indexed?
Hello.... I got to tell that I feel like a newbie (I am, but know I feel like it)... We were working with a client until january this year, they kept going on their own until september that they contacted us again... Someone on the team that handled things while we were gone, updated it´s robots.txt file to Disallow everything... for maybe 3 weeks before we were back in.... Additionally they were working on a different subdomain, the new version of the site and of course the didn't block the robots on that one. So now the whole site it's been duplicated, even it´s content, the exact same pages exist on the suddomain that was public the same time the other one was blocked. We came in changes the robots.txt file on both server, resend all the sitemaps, sent our URL on google+... everything the book says... but the site it´s not getting indexed. It's been 5 weeks now and no response what so ever. We were highly positioned on several important keywords and now it's gone. I now you guys can help, any advice will be highly appreciated. thanks Dan
Intermediate & Advanced SEO | | daniel.alvarez0 -
Google Indexing Feedburner Links???
I just noticed that for lots of the articles on my website, there are two results in Google's index. For instance: http://www.thewebhostinghero.com/articles/tools-for-creating-wordpress-plugins.html and http://www.thewebhostinghero.com/articles/tools-for-creating-wordpress-plugins.html?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+thewebhostinghero+(TheWebHostingHero.com) Now my Feedburner feed is set to "noindex" and it's always been that way. The canonical tag on the webpage is set to: rel='canonical' href='http://www.thewebhostinghero.com/articles/tools-for-creating-wordpress-plugins.html' /> The robots tag is set to: name="robots" content="index,follow,noodp" /> I found out that there are scrapper sites that are linking to my content using the Feedburner link. So should the robots tag be set to "noindex" when the requested URL is different from the canonical URL? If so, is there an easy way to do this in Wordpress?
Intermediate & Advanced SEO | | sbrault740 -
More Indexed Pages than URLs on site.
According to webmaster tools, the number of pages indexed by Google on my site doubled yesterday (gone from 150K to 450K). Usually I would be jumping for joy but now I have more indexed pages than actual pages on my site. I have checked for duplicate URLs pointing to the same product page but can't see any, pagination in category pages doesn't seem to be indexed nor does parameterisation in URLs from advanced filtration. Using the site: operator we get a different result on google.com (450K) to google.co.uk (150K). Anyone got any ideas?
Intermediate & Advanced SEO | | DavidLenehan0 -
301 redirect from one domain to other domain, How To?
Hi, I need to redirect 150 products pages from http://www.filtrationmontreal.com/ to http://www.furnacefilterscanada.com/ How can I do this? Is there a tool or anything I can do to do 301 from one domain to another one? Can I use Google Webmaster Tool? Thank you, BigBlaze
Intermediate & Advanced SEO | | BigBlaze2050 -
Member request pages, indexed or no indexed?
We run a service website and basically users of the site post their request to get certain items fixed/serviced. Through Google Analytics we have found that we got lots of traffic to these request pages from people searching for those particular items. E.g. A member's request page: "Cost to fix large Victorian oven" has got many visits from searchers searching for "large Victorian oven". The traffic to these pages is about 40% of our Google organic traffic but didn't covert to more users/requests well and has roughly 67% bounce rate. So my question is: should we keep these pages indexed and if yes what can we do to improve the conversion rate/reduce bounce rate? Many thanks guys. David
Intermediate & Advanced SEO | | sssrpm0 -
Redirecting One Page of Content on Domain A to Domain B
Let's say I have a nice page of content on Domain A, which is a strong domain. That page has a nice number of links from other websites and ranks on the first page of the SERPs for some good keywords. However, I would like to move that single page of content to Domain B using a 301 redirect. Domain B is a slightly weaker domain, however, it has better assets to monetize the traffic that visits this page of content. I expect that the rankings might slip down a few places but I am hoping that I will at least keep some of the credit for the inbound links from other websites. Has anyone ever done this? Did it work as you expected? Did the content hold its rankings after being moved? Any advice or philosophical opinions on this? Thank you!
Intermediate & Advanced SEO | | EGOL2