How was cdn.seomoz.org configured?
-
The SEOmoz CDN appears to have a "pull zone" that is set to the root of the domain, such that any static file can be addressed from either subdomain:
http://www.seomoz.org/q/moz_nav_assets/images/logo.png
http://cdn.seomoz.org/q/moz_nav_assets/images/logo.png
The risk of this configuration is that web pages (not just images/CSS/JS) also get cached and served by the CDN. I won't put the URL here for fear of Google indexing it, but if you replace the 'www' in the URL below with 'cdn', you'll see a cached copy of the original:
http://www.seomoz.org/ugc/the-greatest-attribution-ever-graphed
The worst-case scenario is that the homepage gets indexed. But this doesn't happen here:
That URL issues a 301 redirect back to the canonical www subdomain. As it should.
Here's my question: how was that done?
Because maxcdn.com can't do it. If you set a "pull zone" to your entire domain, they'll cache your homepage and everything else. googlebot has a field day with that; it will reindex your entire site off the CDN.
Maybe the SEOmoz CDN provider (CloudFront) allows specific URLs to be blocked? Or do you detect the CloudFront IPs and serve them a 301 (which they'd proxy out to anyone requesting cdn.seomoz.org)?
One solution is to create a pull zone that points to a folder, like example.com/images... but this doesn't help a complex site that has cacheable content in multiple places (do you Wordpress users really store ALL your static content under /wp-content/ ?).
Or, as suggested above, dynamically detect requests from the CDN's proxy servers, and give them a 301 for any HTML-page request. This gets complex quickly, and is both prone to breakage and very difficult to regression-test.
Properly retrofitting a complex site to use a CDN, without creating a half-dozen new CDN subdomains, does not appear to be easy.
-
its a SEOmoz secret...
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
If my website uses CDN does thousands of 301 redirect can harm the website performance?
Hi, If my website uses CDN does thousands of 301 redirect can harm the website performance? Thanks Roy
Intermediate & Advanced SEO | | kadut1 -
Post migration issues - #11 + configuration issue
Hello Moz community. I'm keen to find out your experiences on the following: Have you ever experienced a migration whereby a large % of keywords are stuck in position #11 - post migration? The keywords do not move up or down (whilst competitors jump from 13 to 9 and vice versa) over a three month period. Please see the % difference in the attached e-mail. (sample 1,000 keyword terms) Question: Has anyone ever experienced this type of phenomenon before? If so - what was the root cause of this and did this happen post migration? What solution did you use to rectify this? Have you ever seen a cross-indexing issue between two domains (each domain serves a different purpose) post migration, which impacts the performance of the main brand domain? I will explain a little further - say you have www.example.com (brand site) and www.example-help.com (customer service site) and the day the brand website is migrated (same domain - just different file structure), www.example-help.com points to the same server that www.example.com is on (with a different file structure) and starts to inherit the legacy file structure. For example, the following is implemented on migration day: I will explain a little further - say you have www.example.com (brand site) and www.example-help.com (customer service site) and the day the brand website is migrated (same domain - just different file structure), www.example-help.com points to the same server that www.example.com is on (with a different file structure) and starts to inherit the legacy file structure. For example, the following is implemented on migration day: For example, the following is implemented on migration day: www.example.com/fr/widgets-purple => 301s to www.example.com/fr/widgets/purple But www.example-help.com now points to the same server where the customer service content is now hosted. So although the following is rendered: So although the following is rendered correctly: www.example-help.com/how-can-we-help We also have the following indexed in Google.fr - competing for the same keyword terms and the main brand website has dropped in rankings: www.example-help.com/fr/widgets-purple [legacy content from main brand website] Even when legacy content is 301 redirected from www.example-help.com to www.example.com, the authority isn't passed across and we now have www.example.com (as per Q1) a lot lower in Google than pre-migration. Question: Have you ever experienced a cross-indexing issue like above whereby Google potentially isn't passing authority across from legacy to the new setup? I'm very keen to hear your experiences on these two subjects and whether you have had similar problems on some of your domains. E0hbb
Intermediate & Advanced SEO | | SMVSEO0 -
Google not Indexing images on CDN.
My URL is: http://bit.ly/1H2TArH We have set up a CDN on our own domain: http://bit.ly/292GkZC We have an image sitemap: http://bit.ly/29ca5s3 The image sitemap uses the CDN URLs. We verified the CDN subdomain in GWT. The robots.txt does not restrict any of the photos: http://bit.ly/29eNSXv. We used to have a disallow to /thumb/ which had a 301 redirect to our CDN but we removed both the disallow in the robots.txt as well as the 301. Yet, GWT still reports none of our images on the CDN are indexed. The above screenshot is from the GWT of our main domain.The GWT from the CDN subdomain just shows 0. We did not submit a sitemap to the verified subdomain property because we already have a sitemap submitted to the property on the main domain name. While making a search of images indexed from our CDN, nothing comes up: http://bit.ly/293ZbC1While checking the GWT of the CDN subdomain, I have been getting crawling errors, mainly 500 level errors. Not that many in comparison to the number of images and traffic that we get on our website. Google is crawling, but it seems like it just doesn't index the pictures!? Can anyone help? I have followed all the information that I was able to find on the web but yet, our images on the CDN still can't seem to get indexed.
Intermediate & Advanced SEO | | alphonseha0 -
Using a US CDN (Cloudflare) for a UK Site. Should I use a UK Based CDN as it says my server is based in USA
Hi All, We are a UK Company with Uk customers only and use CloudFlare CND. Our Site is hosted by a UK company with servers here but from looking online and checking where my site is hosted etc etc , some sites are telling me the name of our UK Hosted company and other sites are telling me my site is hosted in San Fran (USA) , where I presume the Cloudflare is based. I know Cloudflare has a couple of servers in the UK it uses but given all my customers are UK based ,I don't want this is affect rankings etc , as I thought it was a ranking benefit to be hosted in the country you are based. Is there any issue with this and should I change or is google clever enough to know so i shouldn't worry. thanks Pet
Intermediate & Advanced SEO | | PeteC120 -
Is CDN Good For International Website?
Hello - Which solution is better For International Website: 1) using a CDN, or 2) using some VPS's in each location?
Intermediate & Advanced SEO | | Vahid-af0 -
Schema.org Markup for Currencies in Multiple Countries.
Hello - Just getting in to a bit of Markup for rich snippets etc. This site sells from New Zealand , but our target market is Australia and most of our sales are there and we locate it there in Webmaster tools. Our Site changes currency / location automatically detecting IP Addresses. So -primarily I have a product with multiple variations on one page that I want to show offers in Google.com.au - in Australian Dollars Syntol Probiotic
Intermediate & Advanced SEO | | s_EOgi_Bear
90 caps $25AUD 180 Caps $50AUD 360 Caps 75AUD Here is the page http://www.return2health.net/syntol.html Ideas around that? Ideally I would like to add some country specific data to it I guess..?0 -
Amazon CloudFront CDN
Hi, I'd like to increase website's speed with Amazon CloudFront CDN. I created some CNAMEs and i've something like this: www.mydomain.com (my website) cdn1.mydomain.com cdn2..mydomain.com cdn3.mydomain.com But i've a lot duplicate content now ! One per subdomain and one per content (gif, css, html, and so one). Have you any feedback in order to not have SEO penalty ? Does Google detects CDN ? Can I help him to understand my CDNs ? Thanks, Best regards, Maxime
Intermediate & Advanced SEO | | Max840