Images on sub domain fed from CDN
-
I have a client that uses a CDN to fill images, from a sub domain ( images.domain.com). We've made sure that the sub domain itself is not blocked. We've added a robots.txt file, we're creating an image sitemap file & we've verified ownership of the domain within GWT.
Yet, any crawler that I use only see's the first page of the sub domain (which is .html) but none of the subsequent URL's which are all .jpeg.
Is there something simple I'm missing here?
-
Alphonse it sounded like they were just waiting for the sitemap to launch. Other than that, I couldn't think of anything else to add because the sitemap should solve their issue. However, I have marked this as "Discussion" again.
-
I am a little confused. The question was marked answered, but which one is the answer?
-
We have the same issue however we have image XML sitemaps on each country subdomain's XML Index which point to the image files on images.domain.com.
Example:
https://uk.domain.com/image-sitemap1.xml
https://us.domain.com/image-sitemap1.xml
These 2 files are the same.
We also don't have a homepage on images.domain.com and it currently responds with a 404.
Do you think we need to create a landing page on the homepage and host the image XML sitemap at https://images.domain.com/images-sitemap1.xml rather than in each sub-domain?
Thanks.
-
Yes, we are doing everything correctly, aside from waiting for IT department to create a sitemap.
-
Are you using your own subdomain or one somewhere else (e.g. akamai.com)? You should use your own subdomain, if possible.
Was this a change from a previous version that didn't use a CDN? If those images were/are hosted on your primary domain be sure to match the filenames and paths as closely as possible to what they were before.
If you're doing that you shouldn't have a problem once the sitemap is submitted.
For more information please check out this post:
http://www.goinflow.com/four-seo-best-practices-for-using-a-content-delivery-network-cdn/How do you know that Google only attempts to crawl the primary domain URL (i.e. the .html page)? Are you checking log files?
Is the crawler you're using set to crawl external URLs? If not, that could be the issue. Technically a subdomain is a totally separate website so most tools don't crawl them by default.
-
We've correctly applied the CNAME directive from the CDN to reflect the subdomain. Yet, when Google or any other tool attempts to crawl it only shows ONE URL. Not the images that are residing on their own independent URL's.
-
In order to put those image URLs for the crawler to be able to access them you should either:
- Link to the URLs of the images (does that .html page in the subdomain contain these URLs?)
or
- Use the images URLs as resources in the pages already been crawled. Unfortunately this could be tricky when dealing with CDNs since those resources are dynamic.
In either case, the sitemap will solve your problem.
-
The sitemap is not completed yet. Server logs show Googlebot only indexing one page the .html page, not other pages.
-
Did you reference the sitemap in the robots.txt file or did you set up it in GWT?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
<sub>& <sup>tags, any SEO issues?</sup></sub>
Hi - the content on our corporate website is pretty technical, and we include chemical element codes in the text that users would search on (like S02, C02, etc.) A lot of times our engineers request that we list the codes correctly, with a <sub>on the last number. Question - does adding this code into the keyword affect SEO? The code would look like SO<sub>2</sub>.</sub> Thanks.
Technical SEO | | Jenny10 -
Forwarding kw rich domains to main domain
Hi My client has a clutch of kw rich domains that want to point to main domain, apart from being good for promotional reasons is there any seo benefit for doing so (i know there used to be years ago but under impression hasn't been any benefit for a long while) Most importantly though can any bad come from doing this ? Best Rgds Dan
Technical SEO | | Dan-Lawrence0 -
CDN Being Crawled and Indexed by Google
I'm doing a SEO site audit, and I've discovered that the site uses a Content Delivery Network (CDN) that's being crawled and indexed by Google. There are two sub-domains from the CDN that are being crawled and indexed. A small number of organic search visitors have come through these two sub domains. So the CDN based content is out-ranking the root domain, in a small number of cases. It's a huge duplicate content issue (tens of thousands of URLs being crawled) - what's the best way to prevent the crawling and indexing of a CDN like this? Exclude via robots.txt? Additionally, the use of relative canonical tags (instead of absolute) appear to be contributing to this problem as well. As I understand it, these canonical tags are telling the SEs that each sub domain is the "home" of the content/URL. Thanks! Scott
Technical SEO | | Scott-Thomas0 -
Showing My domain twice
I have a question, I am showing my domain twic, how can i have page rank 32 and page rank 44 for the same page? Regards | The UK's Leading Probate Brokers - Lowest Probate Quote guarantee! Probate experts. http://finalduties.co.uk/ 83 (over by 13) 32 1 The UK's Leading Probate Brokers - Lowest Probate Quote guarantee! Probate experts. http://www.finalduties.co.uk/ | 83 (over by 13) | 44 | 112 |
Technical SEO | | Chris__Chris0 -
Pointing a sub-domain to a sub-folder in htaccess
I have a client who currently uses shopify for there blog. I want to set them up with a separate blog away from the shopify system and host it in Australia. I know the best option is using a subfolder but as the shopify system is an unmoveable CMS can I somehow point my subdomain to a subfolder and get the benefits of the domain name? I could do this by using the rewrite rule in the htaccess file. If I was to do this would it end up cloaking the URL's of the articles?
Technical SEO | | acs1110 -
Should this site start again on a new domain
Hi We have not done SEO on this site they have used another company who looks like they outsourced and the links have been built by a third party all blog networks and this company have said they cannot get the links removed. Google flagged artificial links on this web site in February and in April it lost over 10000 visitors in a month and its just free falled ever since. The categories have been recreated and no redirects created due to the amount of backlinks from the blog sites to the original category pages but the site is not recovering its down to 1500 visitors a month and used to get 14000 a month. So should my customer ditch the domain and move this site to fresh domain? http://www.kids-beds-online.com Any answers would really be appreciated. thanks Tracy
Technical SEO | | dashesndots0 -
How do I fix a multiple domain mess?
I just picked up an account that is a franchisee and they have 6 exact match domains plus their main domain (all exact duplicates, not 301s). GA shows the main domain as getting the lion's share of hits, but for some important local keywords, the exact match domains rank higher. Some pages may have the exact match domain, primary domain and the franchise domain all ranking on the same SERP. Yuk! My strategy is to work on the main domain and as the work progresses, the main site will surpass the exact match domains and main franchise domain for the important searches. For the exact match domains I plan on just leaving them alone. Is this a sound strategy? I could pull the exact match domains down, but since they rank well for their keywords, it seems most sensible to leave them up. What do you think?
Technical SEO | | KristinnD0 -
Lots of Domains Going Nowhere - Point to a Real Domain?
I have hundreds of domains that I have purchased over the years that arent going anywhere except GoDaddy's Cash Parking system, which returns very little revenue, if at all. I wonder if it would make more sense to just point these domains to actually e-commerce sites that I own. If so, how best to take these domains and point them so that SEO credit is given properly. Most of these available domains dont have anything to do with the e-commerce stores. So not sure it would help. Furthermore, if I were to purchase new domains that were more relevant to the keywords to our e-commerce sites, how best to set them up so we can generate traffic on them and point them over to the actual domains? Many thanks.
Technical SEO | | findachristianjob0