XML Sitemap Question!
-
Hi All,
I know that the sitemaps.xml URL must be findable but what about the sitemaps/pageinstructions.xml URL?
Can we safely noindex the sitemaps/pageinstructions.xml URL?
Thanks!
Yael
-
In regards to an "XML site map"- it's essential to have a site-map on your company website, that's because it makes it much easier for a Google Bot, to find the pages/ blog posts to crawl and index these if you have a XML site map.
We added a Yoast SEO site map to our Bristol Garden Room company website, in Bristol to ensure that all the blog posts / main pages were getting indexed.
-
Yes, you can noindex practically anything using the robots.txt file
-
Hey
Yes, you can safely do that. In the end, you mostly want to make sure that the right pages are being Crawled > Indexed. If that requires certain (XML) files to be no-indexed, then that's the way you want to go.
Martijn.
-
Hey Yael,
You should be able to. If it's being indexed, you can request removal via Bing Webmaster Tools and Google Search Console. You can also update your robots.txt file to disallow the page "http://www.robotstxt.org/robotstxt.html"
I hope that helps!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Disavow File and SSL Conversion Question
Moz Community, So we have a website that we are moving to SSL. It has been 4 years since we submitted our disavow file to google via GWT. We decided to go through our backlinks and realized that many domains we are disavowing currently (under Since we are moving to SSL I understand Google looks at this as a new site. Therefore, we decided to go through our backlinks and realized that many domains we are disavowing currently are no longer active (after 4 years this is expected). Therefore, is it ok to create a new disavow file with the new profile on GW (ssl version of our site)? Also, is it ok the new GW disavow file doesn't include urls we previously disavowed with the non https version? Some links from the old disavow we found were disavowed but they shouldn't have been. Moreover, we found new links we wanted to disavow as well. Thanks QL
Intermediate & Advanced SEO | | QuickLearner0 -
Hreflang Sitemap
Hi all, Have you ever created an hreflang sitemap using in-house resources or a third-party company for a group of over 70 sites, each with hundreds of pages and all with localised URLs? If so, would you mind sharing how you did it, or the contact details of the company you used? In this specific case there is nothing in the URL or code that I can use to group the alternatives automatically. Thanks, Carlos
Intermediate & Advanced SEO | | Carlos-R0 -
We have a static sitemap and a blog...Uh oh!
Hi We have a static sitemap and a blog...Uh oh! We are are just realizing that the 2 are not very compatible!! We need a dynamic sitemap. Is there such a thing as a "set and forget" sitemap that updates itself without us maintaining a database of sitemap data? Thanks Andrew
Intermediate & Advanced SEO | | Studio330 -
Should canonical links be included or excluded in a sitemap?
Our company is in the process of updating our sitemap. Should we include or exclude canonical links.
Intermediate & Advanced SEO | | WebRiverGroup0 -
2013 Panda Update Question
Hi everyone, I'm new here 🙂 So far I've had wonderful success seo wise and none of the updates (Penguin nor Panda) affected any sites, until this one. For example, one site has 7 keywords I'm optimizing for. Out of those 7, all but 2 (and variations of the 2 - one word vs long-tail) completely tanked. These keywords were all on page 2/3. One of the two survivors never budged from page 2 (it's a brand keyword so I was sooo happy to finally get it to page 2) Now when I check rankings, the other terms show up in the 200-400 spots, but NOT for the URL I was optimizing for (category page) but instead for random products in the category. The only thing I've done differently with the 2 keywords that are still doing well, was focus - we did more link-building for those, but not an extreme amount. Never over-optimize. My question is, how did 2 survive and 5 are still floating up and down. Last night I saw one go up 122 spots, now today down 14. I'm really struggling with this. Thank you
Intermediate & Advanced SEO | | Freelancer130 -
How to generate xml sitemape for an ecommerce site with more than 50000 pages?
Hi, I am new to the forum and struggling hard to work on xml sitemap for an ecommerce site. Site is dynamic and more that 50,000 pages (including product pages). Challenges I am facing should I opt for category wise xml sitemap? how to include new product pages (dynamically) I was wondering if there is any tool that can generate xml site map online (I mean as soon as a new page is added to the site it will pick up automatically). thanks
Intermediate & Advanced SEO | | posy0 -
"Iffy" Question
Hi Guys and Girls, I have been studying SEO for a few years now. I have learned quite a bit along the way. One thing that I have had BEAT in my head is "Create Quality Content". When I used to ask "What Should I do to get more visits?" I was told, "Create QUALITY CONTENT". That was great advice, and I have done that. I have created over 600 pages since then. I went from 200 visits per month to now just over 5,000. Even more importantly, I have increased my lead conversions. I say all of this because, about two years ago I told a competitor basically how you rise up in the search engines. He turned around and bought a domain that was 3 years older than mine and had a main keyword in the domain. He then just started building links with a bunch of blog comments and forum posts (The blog comments have like 2,000 comments on them). In other words, he did what I would NOT do yet he is #5 for this keyword and I am #9. (Although, I turned around and built another site and now I am at #13 for that and it has been up for a year now. I say all this, not to bore you but to tell ask you, DOES IT WORK IF YOU DO BLOG COMMENTING, FORUM POSTING, LINK WHEELS, ETC. Do you obtain higher rankings? Should I be doing a bit of video marketing? I know a lot of people will say, "Why would you spend your time on things that may slightly impact your rankings, but obviously it did something for this guy. Any help you can give would be greatly appreciated.
Intermediate & Advanced SEO | | blake-766241 -
What is the best method for segmenting HTML sitemaps?
Sitemaps create a Table of Contents for web crawlers and users alike. Understanding how PageRank is passed, HTML sitemaps play a critical role in how Googlebot and other crawlers spider and catalog content. I get asked this question a lot and, in most cases, it's easy to categorize sitemaps and create 2-3 category-based maps that can be linked to from the global footer. However, what do you do when a client has 40 categories with 200+ pages of content under each category? How do you segment your HTML sitemap in a case like this?
Intermediate & Advanced SEO | | stevewiideman0