Removing secure subdomain from google index
-
we've noticed over the last few months that Google is not honoring our main website's robots.txt file. We have added rules to disallow secure pages such as:
Disallow: /login.cgis Disallow: /logout.cgis Disallow: /password.cgis Disallow: /customer/* We have noticed that google is crawling these secure pages and then duplicating our complete ecommerce website across our secure subdomain in the google index (duplicate content) https://secure.domain.com/etc. Our webmaster recently implemented a specific robots.txt file for the secure subdomain disallow all however, these duplicated secure pages remain in the index.
User-agent: *
Disallow: /My question is should i request Google to remove these secure urls through Google Webmaster Tools? If so, is there any potential risk to my main ecommerce website? We have 8,700 pages currently indexed into google and would not want to risk any ill effects to our website. How would I submit this request in the URL Removal tools specifically? would inputting https://secure.domain.com/ cover all of the urls? We do not want any secure pages being indexed to the index and all secure pages are served on the secure.domain example. Please private message me for specific details if you'd like to see an example. Thank you,
-
I think you're saying you have
mainwebsitethatsellsstuff.com
securesubdomainof.mainwebsitethatsellsstuff.comand that you want to keep the main domain, and remove the subdomain, and that it's not a case of http vs https with the URL otherwise being the same, right?
You can verify a subdomain in Google Webmaster Tools and remove the entire subdomain. I've had to do this for a dev subdomain that accidentally got indexed. I was able to keep the main domain, and remove the subdomain. The key is to verify that subdomain, and leave the main domain alone, provided I'm understanding your question correctly.
-
Do you need 8700 pages served on https? Protocol should transition when a page is ok to serve unsecured. Generally you would only serve pages on https that contain confidential information and have general content on http. If you look at the site and ask how many of those pages can a no logged in user see? If they are not protected by authorization then they do not need https as the content is publically viewable.
-
URL Removal would not be a good action in this case. According to Google, when they remove the https version, they will also remove the http version along with it.
How long ago did you implement the robots.txt exclusion for the https pages? It will take Google some time to pull this from their index. To help you can add the following on your https pages which will keep the pages from continuing to be cached:
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Site:www Issue - Homepage of the website is not showing in Google
Hello everyone, When I have manually search site:www.blinds4uk.co.uk in google.co.uk to know about webpages status, home page of the website is not showing in google search engine result pages. Please let me know, what is the reason behind this? because website crawling and indexing is good. Many Thanks.
Algorithm Updates | | Kuldeep-Sharma0 -
Help - Losing Ranking on Plural Keywords in Google
Help! I'm losing keyword rankings for certain keywords - but still ranking well on others. The trend I can see is that we're losing rankings on plurals, so this page https://www.key.co.uk/en/key/platform-trolleys-trucks Used to rank well for: platform trolleys
Algorithm Updates | | BeckyKey
platform trucks
flat bed trolleys But now only ranks well for platform trolley flat bed trolleys platform truck I have seen a large number of keywords drop out - they're not hitting traffic massively yet, but conversions are dropping - help! I;m struggling to find the cause!0 -
Very strange, inconsistent and unpredictable Google ranking
I have been searching through these forums and haven't come across someone that faces the same issue I am. The folks on the Google forums are certain this is an algorithm issue, but I just can't see the logic in that because this appears to be an issue fairly unique to me. I'll take you through what I've gone through. Sorry for it being long. Website URL: https://fenixbazaar.com 1. In early February, I made the switch to https with some small hiccups. Overall however the move was smooth, had redirects all in place, sitemap, indexing was all fine. 2. One night, my organic traffic dropped by almost 100%. All of my top-ranking articles completely disappeared from rank. Top keyword searches were no longer yielding my best performing articles on the front page of results, nor on the last page of results. My pages were still being indexed, but keyword searches weren't delivering my pages in results. I went from 70-100 active users to 0. 3. The next morning, everything was fine. Traffic back up. Top keywords yielding results for my site on the front page. All was back to normal. Traffic shot up. Only problem was the same issue happened that night, and again for the next three nights. Up and down. 4. I had a developer and SEO guy look into my backend to make sure everything was okay. He said there were some redirection issues but nothing that would cause such a significant drop. No errors in Search Console. No warnings. 5. Eventually, the issue stopped and my traffic improved back to where it was. Then everything went great: the site was accepted into Google News, I installed AMP pages perfectly and my traffic boomed for almost 2 weeks. 6. At this point numerous issues with my host provider, price increases, and incredibly outdated cpanel forced me to change hosts. I did without any issues, although I lost a number of articles albeit low-traffic ones in the move. These now deliver 404s and are no longer indexed in the sitemap. 7. After the move there were a number of AMP errors, which I resolved and now I sit at 0 errors. Perfect...or so it seems. 8. Last week I applied for hsts preload and am awaiting submission. My site was in working order and appeared set to get submitted. I applied after I changed hosts. 9. The past 5 days or so has seen good traffic, fantastic traffic to my AMP pages, great Google News tracking, linking from high-authority sites. Good performance all round. 10. I wake up this morning to find 0 active people on my site. I do a Google search and notice my site isn't even the first result whenever I do an actual search for my name. The site doesn't even rank for its own name! My site is still indexed but search results do not yield results for my actual sites. Check Search Console and realised the sitemap had been "processed" yesterday with most pages indexed, which is weird because it was submitted and processed about a week earlier. I resubmitted the sitemap and it appears to have been processed and approved immediately. No changes to search results. 11. All top-ranking content that previously placed in carousal or "Top Stories" in Google News have gone. Top-ranking keywords no longer bring back results with my site: I went through the top 10 ranking keywords for my site, my pages don't appear anywhere in the results, going as far back as page 20 (last page). The pages are still indexed when I check, but simply don't appear in search results. It's happening all over again! Is this an issue any of you have heard of before? Where a site is still being indexed, but has been completely removed from search results, only to return within a few hours? Up and down? I suspect it may be a technical issue, first with the move to https, and now with changing hosts. The fact the sitemap says processed yesterday, suggests maybe it updated and removed the 404s (there were maybe 10), and now Google is attempting to reindexed? Could this be viable? The reason I am skeptical of it being an algorithm issue is because within a matter of hours my articles are ranking again for certain keywords. And this issue has only happened after a change to the site has been applied. Any feedback would be greatly appreciated 🙂
Algorithm Updates | | fenixbazaar0 -
Google update January 2015
Hello, In January 2015, google changed its European Algorithm. The change decreased the ranking of some of our keywords but not all. See article for more evidence in google changing its algorithm. https://www.seroundtable.com/google-update-maybe-19760.html The biggest change was the keyword phrase ‘Wholesale Silver Jewellery’ which we ranked 1 in SERP, but now we’re nowhere to be seen. However, the change didn’t affect our keyword phrase ‘Wholesale Jewellery Silver’, ’Wholesale Silver’ and ‘Wholesale Jewellery. We’ve been through our data and see that all of our ’Silver Jewellery’ keyword phrases are no longer showing in the SERP. Further research has shown that our competitors were also dropped down the rankings for the same keyword phrase. Our question is: Why has this update affected certain keyword phrases, such as ‘silver jewellery’ but not ‘jewellery silver’ and how should we over come this? Additional Information
Algorithm Updates | | SilverStar1
If you type in our company name ‘Mainly Silver’ or ‘mainlysilver’ were still showing in SERP, however if you type ‘mainlysilver jewellery’ we’re no where to be found. We’ve even checked ‘site:mainlysilver.co.uk silver jewellery’ in google search and it returns with ‘no results found’. If you switch the keyword phrase, all our web pages are showing up Our website is - www.mainlysilver.co.uk0 -
Google & Tabbed Content
Hi I wondered if anyone had a case study or more info on how Google treats content under tabs? We have an ecommerce site & I know it is common to put product content under tabs, but will Google ignore this? Becky
Algorithm Updates | | BeckyKey1 -
New Domain, Subdomain or Subfolder
Hi All, I am working with a bank that would like to rank as many parts of the company site as possible for the company name. This includes the home page, a page on careers and a page on company reviews. The question is, it is better to structure the careers and reviews content on a subfolder, subdomain or new domain. Using subfolders to retain equity of the root domain site.americanbank.comamernicanbank.com/careersamericanbank.com/reviews or (use subdomains - you lose some of the main domain equity and it is counter to the Moz research) americanbank.comcareers.americanbank.comreviews.americanbank.com or set up new domains to overcome Google bias not to rank the same root domain in the top 7 to 10 results multiple times when displaying results for a company name. americanbank.comhttp://americanbankcareers.comhttp://americanbankreviews.com Thanks for your perspective.
Algorithm Updates | | BetterAnalytics0 -
Google Rankings Dropped in Past Few Weeks
Hi All, I work for an online appliance retailer and over the past weeks, we've seen a drop in our google SERPs. This time last year we were ranking in the top 3 for our top converting key terms, but now we are ranking towards the bottom of the first page or even on the top of the second page with the big box stores now dominating for our key terms. Needless to say traffic for these pages has dropped off considerably. We still have quite a bit of traffic coming in for other key terms, but they don't convert as well. Is anyone else seeing the same thing? If so what are you doing to combat this? Do you have any suggestions? Thank you!
Algorithm Updates | | airnwater0 -
Google above the fold update
Hi everyone, Ever since the Jan 19th Google 'above the fold update' I have noticed some strange ranking changes in some of my sites. 1. rankings increased dramatically (not in top 50 to page 2) on Jan 19th for about 5 days then dropped out completely from the top 50. 2. our rankings then did the same thing again around Feb 2nd for about 5 -6 days then has bottomed out ever since. We do not have any ads on the site but our pages are dominated by images for most of the 'above the fold' section then followed by the content down the page. Any insight into this would be much appreciated. Cheers, Andrew
Algorithm Updates | | jay.raman0