Removing secure subdomain from google index
-
we've noticed over the last few months that Google is not honoring our main website's robots.txt file. We have added rules to disallow secure pages such as:
Disallow: /login.cgis Disallow: /logout.cgis Disallow: /password.cgis Disallow: /customer/* We have noticed that google is crawling these secure pages and then duplicating our complete ecommerce website across our secure subdomain in the google index (duplicate content) https://secure.domain.com/etc. Our webmaster recently implemented a specific robots.txt file for the secure subdomain disallow all however, these duplicated secure pages remain in the index.User-agent: *
Disallow: /My question is should i request Google to remove these secure urls through Google Webmaster Tools? If so, is there any potential risk to my main ecommerce website? We have 8,700 pages currently indexed into google and would not want to risk any ill effects to our website. How would I submit this request in the URL Removal tools specifically? would inputting https://secure.domain.com/ cover all of the urls? We do not want any secure pages being indexed to the index and all secure pages are served on the secure.domain example. Please private message me for specific details if you'd like to see an example. Thank you, -
I think you're saying you have
mainwebsitethatsellsstuff.com
securesubdomainof.mainwebsitethatsellsstuff.comand that you want to keep the main domain, and remove the subdomain, and that it's not a case of http vs https with the URL otherwise being the same, right?
You can verify a subdomain in Google Webmaster Tools and remove the entire subdomain. I've had to do this for a dev subdomain that accidentally got indexed. I was able to keep the main domain, and remove the subdomain. The key is to verify that subdomain, and leave the main domain alone, provided I'm understanding your question correctly.
-
Do you need 8700 pages served on https? Protocol should transition when a page is ok to serve unsecured. Generally you would only serve pages on https that contain confidential information and have general content on http. If you look at the site and ask how many of those pages can a no logged in user see? If they are not protected by authorization then they do not need https as the content is publically viewable.
-
URL Removal would not be a good action in this case. According to Google, when they remove the https version, they will also remove the http version along with it.
How long ago did you implement the robots.txt exclusion for the https pages? It will take Google some time to pull this from their index. To help you can add the following on your https pages which will keep the pages from continuing to be cached:
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Only half of the sitemap is indexed
I have a website with high domain authority and high quality content and blog. I've resubmitted the sitemap half a dozen times. Search console getr half way through and then stops. Does anyone know any reason for this? I've seen the usual responses of 'google is not obligated to crawl you' but this site has been fully crawled in the past. It's very odd Does anyone have any ideas why it might stop half way - or does anyone know a testing tool that might illuminate the situation?
Algorithm Updates | | Andrew-SEO0 -
How do you get a url to show as a tagline in google mobile search?
When searching in google via mobile, I am seeing urls changed to taglines. I have attached pictures that show the url in a web search, but a tag line from the mobile search. Does anyone know how to get a tagline to show in place of a url in a mobile search? Any advice would be appreciated! uLkYWRx.png wljXRI3.png
Algorithm Updates | | David-Kley0 -
Help for a webstore with Google Warnings for Watermark Images and Panda
I have not had too much experience with helping websites that have been hit by Panda - any tried and tested formulas I can pass to website owner would be great. He does not want to reveal domain name - its in the area of children/baby products 'Web site featured on page 1 of Google search results for many years (website 5 years old- Australian domain) . In April/May 2014, Google suspended our Google Shopping account because we used watermarks on all our images. We were advised that the suspension would remain in place indefinitely or until such time the watermarks were removed. We wrote back to Google to explain that these watermarks were put in place by our store back 2005 with the sole purpose of protecting our intellectual property. Needless to say, their attitude was unwavering. And as a result, revenue plummeted. However, the perfect storm was about to hit our store without warning. In the same month, Panda 4.0 was unleashed and our store was hit once again. This update alone reduced visitor numbers by around 50% overnight. The Panda 4.0 algorithm update was designed to target poor quality, duplicate content and unfortunately we had some of it. We have now begun creating original content with many of the new products we're uploading onto our web site. It's slow and tedious. We have modified our web site to now include a tag on a the home page (this was missing). We have removed many duplicate links from our footer (it was too big and contained hundreds of links that were also repeated from the header). We introduced a blog and we have engaged the services of a local seo company to disavow any bad backlinks and add missing or improve existing content to category and brand pages. No improvement in our situation is yet visible and with Christmas just 3 months away, poor sales during our 'bread and butter' period will mean even tougher times for our store in 2015. ANY PANDA EXPERTS who can help please email me felicity@gardenbeet.com - looking for independent freelancers rather than agencies
Algorithm Updates | | GardenBeet0 -
What's better for seo, NOINDEX, or INDEX
Hello Mozers; I am having an issue, my client has 10K pages on their site; in WP, and they have a classified section. Question #1: I am asking, what's better for seo, NOINDEX, or INDEX, for their Classified section. They currently have no SEO plug ins, that fix their errors, and warnings. Question #2: My question is also, do I want the Categories crawled, or INDEXED or NOINDEX? Check out their Campaign results by Moz: Title Element Too Long (> 70 Characters) 32 Too Many On-Page Links 9,032 Missing Meta Description Tag 6,234
Algorithm Updates | | smstv0 -
Is it normal to receive 2 mails from Google?
I filed a reconsideration request that was answered in less than a week. Subsequently I was told that no manual penalty was in place but various algorithm factors might be causing my heavy drops in ranking. Then I got a second email which was even more specific. This was great, really heartening stuff and a total surprise as it was very helpful. Is it normal to receive 2 emails from Google with such clear information? I have been very pleased by the comments they have made as it has shown me that they're more customer focused than I had been led to believe by all the research I had done pre reconsideration request. Has anyone else had a clear outline of what they needed to fix and has their site subsequently rebounded post fixing?
Algorithm Updates | | swimwithfishes0 -
How Do I Make My Google SERP "SiteLinks" more relevant?
I have a shopping website with thousands of products, and the sitelinks that google has chosen for me (for a long time) are random product pages, which makes no sense to me. I do not emphasize those products on the home page, and I have a sitemap that clearly lists the directory of all the categories. I also added a "nofollow" attribute to almost every link on the home page that is not important. These products in the site links seem completely random and there isnt even a sitelink for "about" or any of the footer content! What gives? Also, my sitelinks never updated to the new, better version. Any suggestions?
Algorithm Updates | | cDNAInteractive0 -
How much does Google take Social into account in serps
I have been reading and trying to learn more about how google takes social media into account when ranking sites, but I cant seam to find a definitive answer to this, does it make a big difference or does it not really matter?
Algorithm Updates | | MiracleCreative0 -
What do you think of Google SERP encryption?
Really interesting post by Search Engine Land about this "issue" for tracking conversion, especially for long tail keyword research. I suppose this change will be also applied on all google search pages (.ca, .fr etc.). I Really don't think Webmaster tools is a serious compensation in Analytics for this.
Algorithm Updates | | Olivier_Lambert0
