Welcome to the Q&A Forum

Browse the forum for helpful insights and fresh discussions about all things SEO.

Your browser does not seem to support JavaScript. As a result, your viewing experience will be diminished, and you have been placed in read-only mode.

Please download a browser that supports JavaScript, or enable it if it's disabled (i.e. NoScript).

Should I add my html sitemap to Robots?

Technical SEO

656

Trazo last edited by

I have already added the .xml to Robots. But should I also add the html version?
1 Reply Last reply
Reply Quote 0
Libra_Photographic last edited by

As said above, great question though.
1 Reply Last reply
Reply Quote 1
Martijn_Scheijbeler last edited by

No, it won't help you at all as it's not a valid extension that they will use. What you can do is add a link to the HTML sitemap from multiple pages on your site so you provide an efficient way for Google to access it and use it to crawl the other pages on your site.
1 Reply Last reply
Reply Quote 4

Got a burning SEO question?

Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.

Start my free trial

Browse Questions

View

From

Sorted by

With category

Explore more categories

Related Questions

Utilizing one robots.txt for two sites

I have two sites that are facilitated hosting in similar CMS. Maybe than having two separate robots.txt records (one for every space), my web office has made one which records the sitemaps for the two sites, similar to this:
Technical SEO | | eulabrant

0
X-robots tag causing no index issues

I have an interesting problem with a site which has an x-robot tag blocking the site from being indexed, the site is in Wordpress, there are no issues with the robots.txt or at the page level, I cant find the noindex anywhere. I removed the SEO plug-in which was there and installed Yoast but it made no difference. this is the url: https://www.cotswoldflatroofing.com/ Its coming up with a HTTP error: x-robots tag noindex, nofollow, noarchive
Technical SEO | | Donsimong

0
Robots.txt Disallow: / in Search Console

Two days ago I found out through search console that my website's Robots.txt has changed to User-agent: *
Disallow: / When I check the robots.txt in the website it looks fine - I see its blocked just in search console( in the robots.txt tester). when I try to do fetch as google to the homepage I see its blocked. Any ideas why would robots.txt block my website? it was fine until the weekend. before that, in the last 3 months I saw I had blocked resources in the website and I brought back pages with fetch as google. Any ideas?
Technical SEO | | RAN_SEO

0
How to add 301 for many urls

Hi I need to redirect many urls in a website and I was wondering if instead of doing it one by one there is a way to get it the other way round.... Redirect all pages but a few. I get a feeling this is not possible, but prefer asking just in case. Thanks for any feedback
Technical SEO | | turismodevino1

0
Using a single sitemap for multiple domains

We have a possible duplicate content issue based on the fact that we have a number of websites run from the same code base across .com / .co.uk / .nl / .fr / .de and so on. We want to update our sitemaps alongside using the href lang tags to ensure Google knows we've got different versions of essentially the same page to serve different markets. Google has written an article on tackling this:https://support.google.com/webmasters/answer/75712?hl=en but my question remains whether having a single sitemap accessible from all the international domains is the best approach here or whether we should have individual sitemaps for each domain.
Technical SEO | | jon_marine

0
How could i create sitemap with 1000 page and should i update sitemap frequently?

My website have over 1000 pages but the sitemap creator tools i knew only create maximum 500 pages, how could i create sitemap with full of my webpage?
Technical SEO | | magician

0
Basic SEO HTML

Hello Everyone, One place I am weak is coding for SEO. I need to get better. One question I do have is can anyone explain why it's important to place css and java script files in an external file? How do you do this and how do you know if it's already being done? If it has not been done on a site is it hard to go back and do? I understand this is important from a site load time issue Thanks, Bill P.S. Can anyone recommend a resource where I can learn proper html coding for SEO? Thank you!
Technical SEO | | wparlaman

0
Robots.txt and 301

Hi Mozzers, Can you answer something for me please. I have a client and they have 301 re-directed the homepage '/' to '/home.aspx'. Therefore all or most of the linkjuice is being passed which is great. They have also marked the '/' as nofollow / noindex in the Robots.txt file so its not being crawled. My question is if the '/' is being denied access to the robots is it still passing on the authority for the links that go into this page? It is a 301 and not 302 so it would work under normal circumstances but as the page is not being crawled do I need to change the Robots.txt to crawl the '/'? Thanks Bush
Technical SEO | | Bush_JSM

0