Images on sub domain fed from CDN
-
I have a client that uses a CDN to fill images, from a sub domain ( images.domain.com). We've made sure that the sub domain itself is not blocked. We've added a robots.txt file, we're creating an image sitemap file & we've verified ownership of the domain within GWT.
Yet, any crawler that I use only see's the first page of the sub domain (which is .html) but none of the subsequent URL's which are all .jpeg.
Is there something simple I'm missing here?
-
Alphonse it sounded like they were just waiting for the sitemap to launch. Other than that, I couldn't think of anything else to add because the sitemap should solve their issue. However, I have marked this as "Discussion" again.
-
I am a little confused. The question was marked answered, but which one is the answer?
-
We have the same issue however we have image XML sitemaps on each country subdomain's XML Index which point to the image files on images.domain.com.
Example:
https://uk.domain.com/image-sitemap1.xml
https://us.domain.com/image-sitemap1.xml
These 2 files are the same.
We also don't have a homepage on images.domain.com and it currently responds with a 404.
Do you think we need to create a landing page on the homepage and host the image XML sitemap at https://images.domain.com/images-sitemap1.xml rather than in each sub-domain?
Thanks.
-
Yes, we are doing everything correctly, aside from waiting for IT department to create a sitemap.
-
Are you using your own subdomain or one somewhere else (e.g. akamai.com)? You should use your own subdomain, if possible.
Was this a change from a previous version that didn't use a CDN? If those images were/are hosted on your primary domain be sure to match the filenames and paths as closely as possible to what they were before.
If you're doing that you shouldn't have a problem once the sitemap is submitted.
For more information please check out this post:
http://www.goinflow.com/four-seo-best-practices-for-using-a-content-delivery-network-cdn/How do you know that Google only attempts to crawl the primary domain URL (i.e. the .html page)? Are you checking log files?
Is the crawler you're using set to crawl external URLs? If not, that could be the issue. Technically a subdomain is a totally separate website so most tools don't crawl them by default.
-
We've correctly applied the CNAME directive from the CDN to reflect the subdomain. Yet, when Google or any other tool attempts to crawl it only shows ONE URL. Not the images that are residing on their own independent URL's.
-
In order to put those image URLs for the crawler to be able to access them you should either:
- Link to the URLs of the images (does that .html page in the subdomain contain these URLs?)
or
- Use the images URLs as resources in the pages already been crawled. Unfortunately this could be tricky when dealing with CDNs since those resources are dynamic.
In either case, the sitemap will solve your problem.
-
The sitemap is not completed yet. Server logs show Googlebot only indexing one page the .html page, not other pages.
-
Did you reference the sitemap in the robots.txt file or did you set up it in GWT?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Acquired domains for SEO
hi there, For one of our insurance websites we acquired a domain, this domain is going to be redirected to our domain. After some research we discovered the domain we've bought also includes other domains which 301 redirect to specific 'insurance products'. These domains are also included in the deal. But what is the best technical solution for redirecting these specific redirected product domains? They already redirect to the product pages of the domain we've bought, so after redirect this domain, the sub (product) redirected domains will also link to us. It would be like this: A) www.sub-carinsurancesite.nl (301) -> www.domain-we-bought.com/car-insurance -> www.ourdomain.com/car-insurance
Technical SEO | | remkoallertz
B) www.sub-carinsurancesite.nl (301) -> www.ourdomain.com/car-insurance & www.domain-we-bought.com/car-insurance -> www.ourdomain.com/car-insurance etc0 -
Sub Domain Redirect
Hey Everyone, Here is the situation : Currently, a website's sub domain is being redirected to the main website home page. We're having issues getting the sub domain pages indexed. Just want to confirm that it is because of the redirect on the sub domain URL. Should we kill the sub domain redirect and set it up as it's own page? Will that solve the indexing issue for the sub domain pages. More explanation below: subdomain.domain.com currently redirects to domain.com We're having issues indexing pages belonging to the sub domain ( subdomain.url.com/page1 or subdomain.url.com/page2) Appreciate your input in advance. Cheers,
Technical SEO | | SEO5Team0 -
Sub Domains and Robot.txt files...
This is going to seem like a stupid question, and perhaps it is but I am pulling out what little hair I have left. I have a sub level domain on which a website sits. The Main domain has a robots.txt file that disallows all robots. It has been two weeks, I submitted the sitemap through webmaster tools and still, Google has not indexed the sub domain website. My question is, could the robots.txt file on the main domain be affecting the crawlability of the website on the sub domain? I wouldn't have thought so but I can find nothing else. Thanks in advance.
Technical SEO | | Vizergy0 -
Blog.domain.co.uk or domain.co.uk/blog
Hi Guys, I'm just wondering which offers more SEO value and which is easier to set up out of: blog.domain.co.uk domain.co.uk/blog Thanks, Dan
Technical SEO | | Sparkstone0 -
Image map crawlability
Hey Mozzer's, I'm reviewing a moving site which utilizes an image map for their service areas. As I'm new to image mapping how should I diagnose them for crawlability, effectiveness, code (which appears correct using HTML & alt text tag), what should I be looking for, etc? I'm trying to determine the effectiveness of the image map links compared to including some regular lateral links on the page. Page: http://bit.ly/KOOtgN Thanks in advance for all advice and insights.
Technical SEO | | Bragg0 -
Redirecting Parked Domain
I just recently switched hosting. We have 3 domains and 1 being the main domain. Right now they are set up as parked domains. Can I redirect the parked domains under my main websites .htaccess file ? I am just worried that good we will get hit for duplicate content, even though technically the content is only on our main domain. right now if I type in my parked domains in Google, they appear in search. The site is all hand coded and not using a CMS.
Technical SEO | | hfranz0 -
301 from old domain to new domain
Hi, I need to create a 301 redirect for all internal pages located on organic7thheaven.com to the homepage of our new site at http://www.7thheavennaturals.com/ Currently internal pages of the old site such as the following are returning a page not found www.organic7thheaven.com/products/deepcleansing/miraclemud.asp Can anyone help me in setting up a .htaccess file for this problem please? Thanks
Technical SEO | | MJMarketing0 -
Domain.com and domain.com/ redirect(error)
When I view my campaign report I'm seeing duplicate content/ meta for mydomain.com and mydomain.com/ (with a slash) I already applied a 301 redirect as follows: redirect 301 /index.php/ /index.php Where am I messing up here?
Technical SEO | | cgman0