Removing secure subdomain from google index
-
we've noticed over the last few months that Google is not honoring our main website's robots.txt file. We have added rules to disallow secure pages such as:
Disallow: /login.cgis Disallow: /logout.cgis Disallow: /password.cgis Disallow: /customer/* We have noticed that google is crawling these secure pages and then duplicating our complete ecommerce website across our secure subdomain in the google index (duplicate content) https://secure.domain.com/etc. Our webmaster recently implemented a specific robots.txt file for the secure subdomain disallow all however, these duplicated secure pages remain in the index.
User-agent: *
Disallow: /My question is should i request Google to remove these secure urls through Google Webmaster Tools? If so, is there any potential risk to my main ecommerce website? We have 8,700 pages currently indexed into google and would not want to risk any ill effects to our website. How would I submit this request in the URL Removal tools specifically? would inputting https://secure.domain.com/ cover all of the urls? We do not want any secure pages being indexed to the index and all secure pages are served on the secure.domain example. Please private message me for specific details if you'd like to see an example. Thank you,
-
I think you're saying you have
mainwebsitethatsellsstuff.com
securesubdomainof.mainwebsitethatsellsstuff.comand that you want to keep the main domain, and remove the subdomain, and that it's not a case of http vs https with the URL otherwise being the same, right?
You can verify a subdomain in Google Webmaster Tools and remove the entire subdomain. I've had to do this for a dev subdomain that accidentally got indexed. I was able to keep the main domain, and remove the subdomain. The key is to verify that subdomain, and leave the main domain alone, provided I'm understanding your question correctly.
-
Do you need 8700 pages served on https? Protocol should transition when a page is ok to serve unsecured. Generally you would only serve pages on https that contain confidential information and have general content on http. If you look at the site and ask how many of those pages can a no logged in user see? If they are not protected by authorization then they do not need https as the content is publically viewable.
-
URL Removal would not be a good action in this case. According to Google, when they remove the https version, they will also remove the http version along with it.
How long ago did you implement the robots.txt exclusion for the https pages? It will take Google some time to pull this from their index. To help you can add the following on your https pages which will keep the pages from continuing to be cached:
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Tens of duplicate homepages indexed and blocked later: How to remove from Google cache?
Hi community, Due to some WP plugin issue, many homepages indexed in Google with anonymous URLs. We blocked them later. Still they are in SERP. I wonder whether these are causing some trouble to our website, especially as our exact homepages indexed. How to remove these pages from Google cache? Is that the right approach? Thanks
Algorithm Updates | | vtmoz0 -
Google Direct Answers Box - available in other languages aside from English?
Hi guys, Last year I wrote a post for the YouMoz blog (https://moz.com/ugc/google-s-direct-answers-how-to-keep-visitors-coming-to-your-site) about Google's Direct Answers box. Needless to say, I focused purely on English language queries but I'm curious to know if anyone has seen the box appear for queries in other languages. If you've seen this happening and could provide some examples, then please let me know! Thanks in advance! Daniel
Algorithm Updates | | Daniel_Morgan0 -
Google News Results
This is more of an observation than anything else. Has anyone noticed any strange results in Google News, in terms of very old content hitting page 1? My example is football, I support Newcastle so keep checking for the latest transfer failure or humiliation. First page for couple of days is showing old articles (April, May) from the same source rather than the usual spread of tabloid and broadsheet news.
Algorithm Updates | | MickEdwards0 -
Wrong Google pin locations
Did something happen recently that would affect pin locations on Google Maps? I've been updating Google Places pages, but not touching the address or pins - but I received a phone call from one of my locations that their pin location changed in the past month and now it is wrong. Meanwhile, another department recently had MomentFeed update the pins for accuracy. Thoughts?
Algorithm Updates | | SSFCU0 -
URL is starting to appear capitalized in Google Search Results. How come?
Our domain (www.absoluteautomation.com) has just today started appearing in search results as www.AbsoluteAutomation.com. Any ideas why?
Algorithm Updates | | absoauto0 -
Is Google now ignoring title tags?
Hey guys, I noticed alot of titles of webpages in Google now vary from search to search. They also differ from the Title tag that has been set by the webmaster. Anyone else notice this? (My results are depersonalized)
Algorithm Updates | | benjaminspak0 -
Do you think Google is destroying search?
I've seen garbage in google results for some time now, but it seems to be getting worse. I was just searching for a line of text that was in one of our stories from 2009. I just wanted to check that story and I didn't have a direct link. So I did the search and I found one copy of the story, but it wasn't on our site. I knew that it was on the other site as well as ours, because the writer writes for both publications. What I expected to see was the two results, one above the other, depending on which one had more links or better on-page for the query. What I got didn't really surprise me, but I was annoyed. In #1 position was the other site, That was OK by me, but ours wasn't there at all. I'm almost used to that now (not happy about it and trying to change it, but not doing well at all, even after 18 months of trying) What really made me angry was the garbage results that followed. One site, a wordpress blog, has tag pages and category pages being indexed. I didn't count them all but my guess is about 200 results from this blog, one after the other, most of them tag pages, with the same content on every one of them. Then the tag pages stopped and it started with dated archive pages, dozens of them. There were other sites, some with just one entry, some with dozens of tag pages. After that, porn sites, hundreds of them. I got right to the very end - 100 pages of 10 results per page. That blog seems to have done everything wrong, yet it has interesting stats. It is a PR6, yet Alexa ranks it 25,680,321. It has the same text in every headline. Most of the headlines are very short. It has all of the category and tag and archive pages indexed. There is a link to the designer's website on every page. There is a blogroll on every page, with links out to 50 sites. None of the pages appear to have a description. there are dozens of empty H2 tags and the H1 tag is 80% through the document. Yet google lists all of this stuff in the results. I don't remember the last time I saw 100 pages of results, it hasn't happened in a very long time. Is this something new that google is doing? What about the multiple tag and category pages in results - Is this just a special thing google is doing to upset me or are you seeing it too? I did eventually find my page, but not in that list. I found it by using site:mysite.com in the search box.
Algorithm Updates | | loopyal0 -
Did Google Just Punish Organic For Showing Up In Local?
Google pushed the local search results in a prominent spot on a ton of search results. That means (for me) that all first page rankings (including a lot of 1st place rankings) have completely disappeared. Subfolders still rank and homepage still ranks where local search is not included. This is weird. Is this a glitch or is there something we can do about this?
Algorithm Updates | | Discountvc0