How do you de-index and prevent indexation of a whole domain?
-
I have parts of an online portal displaying in SERPs which it definitely shouldn't be. It's due to thoughtless developers but I need to have the whole portal's domain de-indexed and prevented from future indexing.
I'm not too tech savvy but how is this achieved? No index? Robots?
thanks
-
Fastest way to do it would be through a combination of Google Webmaster Tools and Robots.txt / noindex,nofollow meta tags.
Firstly if your domain hasn't been verified within GWT, go through that process. Then proceed with the remove URL request within GWT. (Image Attached)
You will need to upload a robots.txt and/or implement the noindex meta tag within the header of the pages you no longer want indexed.
Example Robots.txt:
User-agent: *
Disallow: /offendingportal/Example Robots Meta tag:
That should be enough to get rid of the offending content from within the SERPS and prevent it from being re-indexed.
-
Hi TCE,
There are two options in my opinion for this which are the best (as their are several other options including META robots etc):
- Robots.txt: Exclude your portal via your robots.txt file.
- HTTP Auth: Let your developers add HTTP Authentication so you have to log in to see the portal. Google won't be able to see the pages itself and so will de index them.
Hope this helps.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Why is a canonicalized URL still in index?
Hi Mozers, We recently canonicalized a few thousand URLs but when I search for these pages using the site: operator I can see that they are all still in Google's index. Why is that? Is it reasonable to expect that they would be taken out of the index? Or should we only expect that they won't rank as high as the canonical URLs? Thanks!
Intermediate & Advanced SEO | | yaelslater0 -
Google Indexing Stopped
Hello Team, A month ago, Google was indexing more than 2,35,000 pages, now has reduced to 11K. I have cross-checked almost everything including content, backlinks and schemas. Everything is looking fine, except the server response time, being a heavy website, or may be due to server issues, the website has an average loading time of 4 secs. Also, I would like to mention that I have been using same server since I have started working on the website, and as said above a month ago the indexing rate was more than 2.3 M, now reduced to 11K. nothing changed. As I have tried my level best on doing research for the same, so please if you had any such experiences, do share your valuable solutions to this problem.
Intermediate & Advanced SEO | | jeffreyjohnson0 -
Need help in de-indexing URL parameters in my website.
Hi, Need some help.
Intermediate & Advanced SEO | | ImranZafar
So this is my website _https://www.memeraki.com/ _
If you hover over any of the products, there's a quick view option..that opens up a popup window of that product
That popup is triggered by this URL. _https://www.memeraki.com/products/never-alone?view=quick _
In the URL you can see the parameters "view=quick" which is infact responsible for the pop-up. The problem is that the google and even your Moz crawler is picking up this URL as a separate webpage, hence, resulting in crawl issues, like missing tags.
I've already used the webmaster tools to block the "view" parameter URLs in my website from indexing but it's not fixing the issue
Can someone please provide some insights as to how I can fix this?0 -
High Quality Domains and what to do with them
Hi, I rus a travel photography business. The primary function of the website is to sell prints, though I blog about my travels on the same domain name as well as a few pieces of content that are helpful to users interested in some of the places I travel to. I do okay with it, but obviously, I am always looking for a way to increase visibility and sales of prints. I own a couple of high quality keyword domain names, that I've been trying to figure out what to do with. One of which is for a city that my prints of my photography are probably best known for. The domains I'm really trying to decide what to do with are basically a www.citystatephotography.com and www.citystatephotos.com, where the city and state are the ones I'm targeting. The question is, what do I do with it? I've seen various ideas from other photographers that have various levels of success. Here are the options I'm considering: Just redirect it to the photo gallery of photos that I'm trying to rank highly for. From what I read on various blogs, this doesn't really do much of anything, but maybe I've read wrong? Create a website or microsite with some quality content related to the city that also links back to my photography website on various places and possibly once in the navigation. I do have quality content I could put up that would be helpful to people from the city besides just trying to get sales. But there's always a chance this will cannibalize my original domain without helping sales, I assume? Spam my photo galleries across two domains. Most of my photography galleries would stay on my main domain that I already run, but the photo galleries that are key to that city would be hosted on that citystatephotography.com domain name. I've seen a photographer from Colorado do quite well with this method. (www.imagesofrmnp.com and www.morninglight.us) He's heavily known for his images of Rocky Mountain National Park and that seems to be his main brand, but all of his non-RMNP travel photography goes on the other site. The two sites look almost identical, though they link back and forth fairly extensively. There doesn't seem to be much in the way of duplicate content either. I've considered this method, but I'm nervous I'll kill what I've already built up if this were to fail. Do nothing with the domains. Seems wasteful, as these domains, particularly the citystatephotography.com domain seems useful in some way. Any thoughts? Thanks in advance!
Intermediate & Advanced SEO | | shannmg10 -
Merging domains into sudomains
I know that questions about this topic have been asked before, but I didn't really find an answer that I could apply to our situation. We have several websites that now exist on separate domains, even though their topics are closely related. We are moving each of these sites into a new CMS and are considering collapsing all of the domains into a sub-domain structure around the strongest domain. Important to note: All of the current domains have existed for many years and have strong site authority, and regardless of the domain decision, in this restructuring we will be bringing them all under a global header. I know that there are SEO risks to moving a site from an established domain to a new one, even with 301 redirects in place, but the team in charge of this move wants to know how much of a hit we would take and how quickly natural search traffic might recover. Maybe and mights aren't really satisfying their questions... Does anyone have experience with collapsing domains into a sub-domain structure and feel like sharing your results? Most importantly, was it worth it???? Thanks.
Intermediate & Advanced SEO | | JuliaG0 -
Please help :) Troubles getting 3 types of content de-indexed
Hi there,
Intermediate & Advanced SEO | | Ltsmz
I know that it takes time and I have already submitted a URL removal request 3-4 months ago.
But I would really appreciate some kind advice on this topic. Thank you in advance to everyone who contributes! 1) De-indexing archives Google had indexed all my:
/tag/
/authorname/
archives. I have set them as no-index a few months ago but they still appear in search engine.
Is there anything I can do to speed up this de-indexing? 2) De-index /plugins/ folder in wordpress site They have also indexed all my /plugins/ folder. So I have added a disallow /plugin/ in my robots.txt 3-4 months ago, but /plugins/ still appear in search engine. What can I do to get the /plugins/ folder de-indexed?
Is my disallow /plugins/ in robots.txt making it worse because google has already indexed it and not it can't access the folder? How do you solve this? 3) De-index a subdomain I had created a subdomain containing adult content, and have it completely deleted it from my cpanel 3months ago, but it still appears in search engines. Anything else I can do to get it de-indexed? Thank you in advance for your help!0 -
Should I 301 a penalized domain to another domains subfolder?
I have a niche domain seems to have been hit by Penguin. It had very good rankings before the update, and I think at least a good part of the penalty might be due to overoptimized anchor text. So here is the question; If I decide to take this site down, should I 301 the entire domain to a relevant sub-folder of another site? i.e comtemporaryfurniture.com to domain.com/category/modern-furniture.html Will the penalty get passed onto the new domain? If the penalty is partly due to anchor text, then pointing it to another site's subfolder would mean the tartget URL has more varied anchor text and could boost rankings.
Intermediate & Advanced SEO | | inhouseseo0 -
Keyword weight in Domain Name
Hi All, I'm looking for some advice on URL structure. Our domain name includes 1/2 of many keyword strategies that we're considering. For example our domain is like, dive.com Keyword strategies that we're looking at targeting would include things like, "dive shop", "dive equipment" etc etc Are we competitive to have a structure like this? dive.com/shop/ dive.com/equipment/ Or will this structure yield stronger results? dive.com/dive-shop/ dive.com/dive-equipment/ Your thoughts are appreciated. Thanks! Malcolm
Intermediate & Advanced SEO | | MAGNUMCreative0