Protecting sitemaps - Good idea or humbug?
-
Is there a way to protect your sitemap.xml so that only Google can read it and would it make sense to do this?
-
From a hacker's perspective, the first order of business is going to be gathering information on the target. does a hacker or someone with malicious intent gain something in obtaining access to your sitemap?
Yes, they do, and that is more information on the layout of your site. How common would there actually be something on the sitemap that could critically expose you to compromise on your VPS/Shared hosting? Um, probably super ultra rare.
But yes there was one time that I was doing an audit for a company and the sitemap did point to a directory that was vulnerable to directory browsing. Fishing around in the directory, I was able to obtain a picture of a PayPal MasterCard front and back because some idiot snapped pictures of it and uploaded it onto the site.
So there are benefits to hiding it, it's relatively easy to do, but if your lazy and don't want to, chances are your good.
-
Hi Herb,
Thank you for your feedback. I think you are right. We are dealing with very short lived up-to-date information so it is vital that as few sites as possible have the information we have. For this reason I was considering to "hide" our sitemaps. Some of our competitors do that but probably we need to find some other measures to achieve our goal.
Cheers
Thomas -
Hi Thomas;
You have not specified your web server platform, but assuming it is Apache it would be easy to do with a regular expression in your .htaccess
However, I do not see any valid reason for doing so. Your sitemap should be a refection of your public menu and internal public links. So other than making it easier for search and other spiders to crawl your site, it does not expose any information that is not available by other methods. So, best practices say that you should have an accurate site map, and unless you have a reson for hiding it that you did not mention I would not hide it.
I will tell you those that you should not bother putting areas you do not want crawled in your robots.txt file and any of the bad folks will not respect the request.
Take care,
Herb
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Xml sitemaps giving 404 errors
We have recently made updates to our xml sitemap and have split them into child sitemaps. Once these were submitted to search console, we received notification that the all of the child sitemaps except 1 produced 404 errors. However, when we view the xml sitemaps in a browser, there are no errors. I have also attempted crawling the child sitemaps with Screaming Frog and received 404 responses there as well. My developer cannot figure out what is causing the errors and I'm hoping someone here can assist. Here is one of the child sitemaps: http://www.sermonspice.com/sitemap-countdowns_paged_1.xml
Technical SEO | | ang0 -
Should Sitemaps be placed in the sub folder they reference?
I have a sitemap-index.xml file in the root. I then have several sitemaps linked to from the index in example.com/sitemaps/sitemap1.xml, example.com/sitemaps/sitemap2.xml, etc. I have seen on other sites that for example a sitemap containing blogs where the blogs are located at example.com/blog/blog1/ would be located at example.com/blog/sitemap.xml. Is it necessary to have the sitemap located in the same folder like this? I would like to have all sitemaps in a single sitemap folder for convenience but not if it will confuse search engines. My index count for URLs in some sitemaps has dropped dramatically in Google Webmaster Tools over the past month or so and I'm not sure if this is having an effect. If it matters, I have all sitemap files, including the index, listed in the robots.txt file.
Technical SEO | | Giovatto0 -
Sitemap issue
How can I create XML as well as HTML sitemaps for my website (both eCommerce and non - eCommerce )Is there any script or tool that helps me making perfect sitemapPlease suggest
Technical SEO | | Obbserv0 -
What may be the reason a sitemap is not indexed in Webmaster Tools?
Hi,
Technical SEO | | SorinaDascalu
I have a problem with a client's website. I searched many related questions here about the same problem but couldn't figure out a solution. Their website is in 2 languages and they submitted 2 sitemaps to Webmaster Tools. One got 100% indexed. From the second one, from over 800 URLs only 32 are indexed. I checked the following hypothesis why the second sitemap may not get indexed: sitemap is wrongly formatted - False sitemap contains URLs that don't return 200 status - False, there are no URLs that return 404, 301 or 302 status codes sitemap contains URLs that are blocked by robots.txt - False internal duplicate content problems - False issues with meta canonical tags - False For clarification, URLs from the sitemap that is not indexed completely also don't show up in Google index. Can someone tell me what can I also check to fix this issue?0 -
Google ranks my sitemap.xml instead of blog post
Hello, For some reason Google shows sitemap results when i search for my blog url website.com/blog/postwhy is Google ranking my sitemap but not a post, especially when i search for full URL? Thanks
Technical SEO | | KentR0 -
Does Bing support a news sitemap yet?
With Bing's new app that will integrate their news feed into Facebook, I'd like to optimize for inclusion in Bing news pickup. Does Bing accept news sitemaps yet?
Technical SEO | | Aggie0 -
XML Sitemap without PHP
Is it possible to generate an XML sitemap for a site without PHP? If so, how?
Technical SEO | | jeffreytrull11