How is Google finding our preview subdomains?
-
I've noticed that Google is able to find, crawl and index preview subdomains we set up for new client sites (e.g. clientpreview.example.com). I know now to use "meta name="robots" and robots.txt) to block the search engines from crawling these subdomains. My question though, is how is Google finding these subdomains? We don't link to these preview domains from anywhere else, so I can't figure out how Google is even getting there.
Does anybody have any insight on this?
-
Thanks for your response Irving. We put some of our preview sites on subdomains of our main domain, but then remove them after the site goes live, so their shouldn't be any duplicate content issues. The main question is just how Google is finding these subdomains.
-
Thanks for the insight guys.
-
I don't specifically use the Google Toolbar, but others in the office may (although I don't think so). It sounds like Chrome could be a potential source as well?
-
I think that this is a good idea. But you gotta be careful.
Our competitor (who ranked #1 and we ranked at #2) had their site redesigned and the design company included the noindex on every page. They forgot to take it off when the new design went live. It took them quite a while to figure it out and we enjoyed all of their sales for about a month.
We are #1 now and they are #2. Must have been a bad design job.
-
If the subdomains are added to WMT google will know about it. if you are designing sites for clients and putting them on your site as subdomains it behooves you to make sure 100% that their dev sites are not being seen by Google. It's duplicate content and your subdomain is the original source of this content. Looks unprofessional too
a) verify any subdomain you are creating for a client in WMT
b) block it in robots.txt and noindex nofollow all pages globally
c) for the ones that are already indexed, go into google WMT and go into that subdomain account and request removal of the site in Googles index. This will remove the indexing for that subdomain only don't worry it won't remove your main site from the index.
-
I would also consider adding a noindex tag if you want the urls removed.
-
I agree with Mat. You never know, but yes Chrome could be another major source. It also depends what you set as your privacy when you setup Chrome (Send anonymous usage data to Google, Yes/No ?) and so on.
-
We usually put them behind an .htaccess login now. We've had situations where the development site have been outranking the live site. Great demo of the power of on-site optimisation, but still a bit annoying for the client.
People used to always blame google toolbar for this. Likewise using chrome could potentially add something to the "to crawl" list. I wonder what the respective privacy policies say about that. I've also seen staging sites pick up links. When an external link on the staging site has been clicked it has alerted someone else, appeared as a link back/trackback etc.
-
The discovery can be from multiple mediums. Do you or the client have Google Toolbar installed ?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google PageSpeed Insights Error
Google PageSpeed Insights showing this "An error occurred while fetching or analyzing the page.Dismiss" URL: http://westpalmbailbonds.com/ What should I do to fix this? Thanks in Advance
Technical SEO | | Beachflower0 -
Subdomain as News Section instead of Source in Google News?
Hi, trying to dig into Google News for a large site, mostly containing news.
Technical SEO | | m.m
The structure of the site network is subdomain.domain.se, and each subdomain has it's own brand with it's own news: x.domain.se
y.domain.se
z.domain.se
etc... Each brand/subdomain is more or less to equate with its own subjectfield/section. In Google News every subdomain is configured with it's own Site Source url, but also having the set up with one section with the same url. It seems like they're getting conflicts in Google News, Google can't always figure out which news article to which brand. Example: an article owned by brand A, but it is sometimes happens that articles getting labeled as brand B in the news SERP, though the link takes you correctly to brand A. I am thinking that this config in News Publisher Center may be a problem? Anyone having any thoughts if that would be better if we delete all source urls except for domain.se-brand and then put all the other subdomains as sections? www.domain.se x.domain.se y.doamin.se z.domain.se Any smart thoughts on this one? Or anything else that could make this wrong labeling (all content included images are hosted in same domain for example). Regards,
Magnus0 -
I have consolidated my subdomains into subfolders; should i remove the subdomains also?
Hi, I have consolidated my website's many sub-domains into sub-folders. We had ~20 sub-domains. The sub-domains and the root domain shared the same code-base and sub-domain specific features were controlled using Apache's SetEnv directive in the httpd.conf includes. Now they are consolidated. I have redirected all of the sub-domains to the relevant sub-folder; so http://sub1.mysite.com now 301 redirects to http://www.mysite.com/sub1.The redirects happen in the htaccess file and all sub-domains and the root are still pointing to the same code-base. However, the Moz campaign tracker still occasionally tells me that i have optimisation opportunities at http://sub1.mysite.com and in both Bing and Google webmaster tools traffic to those sub-domains is mentioned. Should i delete the subdomains? Cheers
Technical SEO | | McCaldin0 -
Paid Links - How does Google classify them?
Greetings All, I have a question regarding "Paid Links." My company creates custom websites for other small businesses across the country. We always have backlinks to our primary website from our "Dealer Sites." Would Google and other search engines consider links from our "dealer sites" to be "paid links?" Example:
Technical SEO | | CFSSEO
http://www.atlanticautoinc.com/ is the "dealer site." Would Google consider the links from Atlantic Auto to be a "paid link," and therefor have less of an impact for page rankings, due to it not being organic? Any insight on this matter would be greatly appreciated. Thank you!!!0 -
Google showing wrong title
Hi, Can anyone assist a newbie please? My keyword 'Security Systems' is giving me position 1 on page 1, but the title it is using is not the page title. I am assuming for some reason it has made it up, please see below. The actual title tag says - <colgroup><col width="420"></colgroup>
Technical SEO | | DaddySmurf
| Security systems | wireless | battery powered | Police Approved | CSS | Google is showing - Compound Security Systems: Wireless Security Systems | Battery ... <cite>www.compoundsecurity.co.uk/</cite>Manufacturers & suppliers of The Mosquito Device & Professional industry compliant and Police recommended battery powered wireless security systems.Contact us - Mosquito Anti-Loitering Devices - Security Equipment - Installers<iframe class="SEOmoz-iframe" src="chrome-extension://eakacpaijcpapndcfffdgphdiccmpknp/html/serpbar.html#{" settings":{"dmr":false,"dmt":false,"mr":true,"mt":true,"serp-overlay":true,"toolbar-enabled":true,"subdomain-metrics":false,"serp-panel-open":true,"toolbar-position":"bottom"},"lsdata":{"feid":1245,"fipl":315,"fmrp":4.717881718426137,"fmrr":3.169235978744307e-9,"ftrp":5.245402699173709,"ftrr":8.283399271183101e-8,"fuid":15756,"pda":47.20670031521297,"peid":1266,"pid":315,"pmrp":4.634793943915049,"pmrr":1.137485303730264e-7,"ptrp":4.949510322693934,"ptrr":1.6938187788875737e-7,"puid":15777,"ueid":510,"uemrp":4.693787066897207,"uemrr":2.520503065617982e-10,"uid":1060,"uipl":191,"ujid":1038,"umrp":4.96103855611017,"umrr":5.477715129904515e-10,"upa":55.929113684261466,"utrp":6.066855084050327,"utrr":2.1655670926371268e-10},"user":{"level":"pro","options":{},"proenabled":true,"aclurl":"http:="" moz.com="" users="" level?src="mozbar","loginURL":"https://moz.com/login","logoutURL":"https://moz.com/logout","apiRequest":"url-metrics","cols":"133146078688","expired":false,"accessId":"pro-RGFkZHlTbXVyZg%3D%3D","expires":1374249469,"signature":"M%2Bh%2FXh4OS%2BAMtT2j6kuntWSTvNI%3D","displayName":"DaddySmurf"},"serp":1,"host":"www.compoundsecurity.co.uk"}"" scrolling="no"></iframe> If anyone can tell me how to correct this, i would very much appreciate it. Regards, Si0 -
Will Google Continue to Index the Page with NoIndex Tag Upon Google +1 Button Impression or Click?
The FAQs for Google +1 button suggests as follows: "+1 is a public action, so you should add the button only to public, crawlable pages on your site. Once you add the button, Google may crawl or recrawl the page, and store the page title and other content, in response to a +1 button impression or click." If my page has NoIndex tag, while at the same time inserted with Google +1 button on the page, will Google recognise the NoIndex Tag on the page (and will not index the page) despite the +1 button's impression or clicks send signals to Google spiders?
Technical SEO | | globalsources.com0 -
Is this against google rules
Hi i am wanting to know if this is against google rules. I am building a website which will have lots of different sections and i wanted to know if you were allowed to have a new domain name pointing to a section of the site. so for example if i had a site with a domain name of manchester and then i wanted a section of the site to be called www.manchester.com/complimentary health I want to know if to help with traffic to the site and to have a better domain name, if it was allowed to have a new domain name pointing to that section of the site which could be called www.complimentaryhealth.com and have that pointing to the section. would love to hear your thoughts on this
Technical SEO | | ClaireH-1848860 -
Google Dmoz description in SERPS
My dmoz description is not as KW rich as my sites normal description. IS there an advantage or disadvantage to either? If so, How do I prevent google from doing this?
Technical SEO | | DavidS-2820610