Removing secure subdomain from google index
-
we've noticed over the last few months that Google is not honoring our main website's robots.txt file. We have added rules to disallow secure pages such as:
Disallow: /login.cgis Disallow: /logout.cgis Disallow: /password.cgis Disallow: /customer/* We have noticed that google is crawling these secure pages and then duplicating our complete ecommerce website across our secure subdomain in the google index (duplicate content) https://secure.domain.com/etc. Our webmaster recently implemented a specific robots.txt file for the secure subdomain disallow all however, these duplicated secure pages remain in the index.
User-agent: *
Disallow: /My question is should i request Google to remove these secure urls through Google Webmaster Tools? If so, is there any potential risk to my main ecommerce website? We have 8,700 pages currently indexed into google and would not want to risk any ill effects to our website. How would I submit this request in the URL Removal tools specifically? would inputting https://secure.domain.com/ cover all of the urls? We do not want any secure pages being indexed to the index and all secure pages are served on the secure.domain example. Please private message me for specific details if you'd like to see an example. Thank you,
-
I think you're saying you have
mainwebsitethatsellsstuff.com
securesubdomainof.mainwebsitethatsellsstuff.comand that you want to keep the main domain, and remove the subdomain, and that it's not a case of http vs https with the URL otherwise being the same, right?
You can verify a subdomain in Google Webmaster Tools and remove the entire subdomain. I've had to do this for a dev subdomain that accidentally got indexed. I was able to keep the main domain, and remove the subdomain. The key is to verify that subdomain, and leave the main domain alone, provided I'm understanding your question correctly.
-
Do you need 8700 pages served on https? Protocol should transition when a page is ok to serve unsecured. Generally you would only serve pages on https that contain confidential information and have general content on http. If you look at the site and ask how many of those pages can a no logged in user see? If they are not protected by authorization then they do not need https as the content is publically viewable.
-
URL Removal would not be a good action in this case. According to Google, when they remove the https version, they will also remove the http version along with it.
How long ago did you implement the robots.txt exclusion for the https pages? It will take Google some time to pull this from their index. To help you can add the following on your https pages which will keep the pages from continuing to be cached:
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Should I move my forum to a subdomain?
My forum causes a lot of 403, 404, soft 404 and 522 errors. I worry about this dragging down the value of my domain and wonder if I should move it to a sub directory. forum.domain.com. I was forced to do this with a very similar site and seems to have not suffered any google penalty (I implemented a 301 redirect to each page to its corresponding page on the subdomain.
Algorithm Updates | | 321Chat0 -
Google Open Graph
Hi I wanted to find out what makes Google select a site to show the answer to a question you type in search? For example, typing What is COSHH, brings up this site http://rospaworkplacesafety.com/2013/01/08/what-is-coshh-about-coshh/ and this answer top of Google SERPs. COSHH stands for 'Control of Substances Hazardous to Health' and under the Control of Substances Hazardous to Health Regulations 2002, employers need to either prevent or reduce their workers' exposure to substances that are hazardous to their health.8 Jan 2013 Is it their open graph mark up only? Becky
Algorithm Updates | | BeckyKey0 -
Google Mobile Algorithm update
Hi there, On April the 21st Google seems to going to update their Mobile algorithm. I have a few questions about this one. Our current mobile website is very mobile friendly. We block all mobile pages with a noindex, so the desktop pages have been indexed on mobile devices. We use a redirect from desktop page to mobile page when someone hits a result on a mobile device. My gut tells me this is not April 21st-proof so I'm thinking about an update to make this whole thing adaptive. By making the thing adaptive, our mobile pages will be indexed instead of the desktop pages. Two questions: Will Google treat the mobile page as a 100% different page than the desktop page? Or will it match those two because everything will tell Google those belong together. In other words: will the mobile page start with a zero authority and will pages lose good organic positions because of authority or not? Which ranking factor will be stronger after April 21st for mobile pages: page authority or mobile friendliness? In other words: is it worth ignoring the 21 April update because the authority of the desktop pages is more important than making every page super mobile friendly? Hope to get some good advice! Marcel
Algorithm Updates | | MarcelMoz0 -
How Does Google Treat External Links to URLs with # Anchors?
Here are two URLs to explain this example: **Original URL: **example.com/1/ **URL that points to anchor within the webpage above: **example.com/1/#anchor Does Google treat these two URLs as separate entities or the same? For example, does an external link to the anchor URL pass full PageRank value to the original URL? How does Google handle this? Is there anything negative about this situation? Are there any risks associated with links to the anchor URL? Finally, is it more valuable for an external link to point to the URL without an anchor?
Algorithm Updates | | SAMarketing0 -
Google places - are you still registering your companies?
Is Google plus taking over from Google Places? I have our company and website registered on Google places and Google Plus. Now we have a new website ( same company different product) Should I register a new listing on Google places? And what about Google Plus?
Algorithm Updates | | Realtor1010 -
Would Google Remove Pages for Inactivity?
Hi, I've been watching the Total Indexed number for 4 domains that I work with for the last few months. In Google Webmaster Tools three of them were holding steady up until August-September, when suddenly they started declining by hundreds of thousands of URLs a week. I've asked my IT department and they say they haven't done anything technically different in the last few months that would affect indexation. I've also searched on google and on search marketing blogs to see if anyone else has experience this to no avail. As you can see in the image, the "Not Selected" pages have not increased so it appears this is not due to duplicate content (of which we have a lot). However, the "Ever Crawled" number is increasing. The only reasonable answer that I can conclude is that Google is now de-indexing inactive URLs? Anyone have a better answer? yIYDm.jpg
Algorithm Updates | | OfficeFurn0 -
Related Searches in Google
Hello, We're helping a client remove/minimize some negative information about their brand in Google's search results. Just curious about your take on if the related searches that appear at the bottom of Google search results can in any way be influenced or if it is more a combination of so many factors that any one person or organization wouldn't be able to change very easily? I've heard the related results could be influenced if enough queries generated overtake the "negative" queries done initially but I feel like that is venturing into black hat land a bit. thanks -Mike
Algorithm Updates | | mattmainpath0 -
Working in the world of Google Farmer Update
So I know have seen how my websites have taken a nose dive from the google farmer update most likely with traffic significantly hit. Example site is callcatalog.com. What recommendations are there to deal with the new world order? How can we look at optimizing, changing, modifying our process to improve rankings and traffic?
Algorithm Updates | | seo_ploom0