Hey thanks for the response. There are about 14,000 url's in the sitemap. It shouldn't freeze up - please would you try again.

http://www.bedsite.co.uk/sitemaps/sitemap.xml

I know what you mean about the allow all

Hi,

Here is my sitemap:

User-agent: *

Sitemap: http://www.bedsite.co.uk/sitemaps/sitemap.xml

Directories

Disallow: /sendfriend/
Disallow: /catalog/product_compare/
Disallow: /media/catalog/product/cache/
Disallow: /checkout/
Disallow: /categories/
Disallow: /blog/index.php/
Disallow: /catalogsearch/result/index/
Disallow: /links.html

I'm using Magento and want to make sure I have constructed my robots.txt file correctly with the sitemap autodiscovery?

thanks,