Moz Q&A is closed.
After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.
Sitemap_index.xml = noindex,follow
-
I was running a rapport with Sreaming Frog SEO Spider and i saw:
(Tab) Directives > NOindex :
https://compleetverkleed.nl/sitemap_index.xml/ is set on X-Robots-Tag 1 > noindex,follow
Does this mean my sitemap isn't indexed?
If anyone has some more tips for our website, feel free to give some suggestions
(Website is far from complete) -
Top, thanks!
-
Hi There
I don't think you need to worry about the sitemap being indexed or not - it's an XML sitemap, not an HTML page users will need to find. It's accessible to Google, and they will use it to crawl the site. Have you submitted the XML sitemap to webmaster tools? If so, make sure it's free of errors and you should be all set!
-
Hi Patrick,
Thanks for the support. I'm only wondering how to fix this problem with a Wordpress website:
https://www.compleetverkleed.nl/sitemap_index.xml/ There should not be a trailing at the end of this URL. This needs to be fixed as soon as possible.
It was set in our footer and linked to the sitemap, and i removed the "/". But in Screaming frog i still see:
https://compleetverkleed.nl/sitemap_index.xml is set on X-Robots-Tag 1 > noindex,followI would also make sure that your non www. sitemap redirects to https://www.compleetverkleed.nl/sitemap_index.xml. Where can i fix this?
This should clear up your problem. Make sure this URL reflects in your Google and Bing Webmaster Tools. Like this? (Google Webmaster tools copy)
|
# 
SitemapType Verwerkt Problemen Items Verzonden Geïndexeerd --- --- --- --- --- --- --- --- --- 1 /sitemap_index.xml Sitemapindex 19 mei 2015 - In behandeling In behandeling -1 van 1 -
Hi there
This doesn't appear to be your sitemap. Your sitemap lives at:
https://www.compleetverkleed.nl/sitemap_index.xml
However, this works as well:
https://www.compleetverkleed.nl/sitemap_index.xml/
There should not be a trailing at the end of this URL. This needs to be fixed as soon as possible.
I would also make sure that your non www. sitemap redirects to https://www.compleetverkleed.nl/sitemap_index.xml.
This should clear up your problem. Make sure this URL reflects in your Google and Bing Webmaster Tools.
Let me know if this helps - good luck!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How do I add "noindex" or "nofollow" to a link in Wordpress
It's been a while since I've SEOed a Wordpress site. How do I add "nofollow" or "noindex" to specific links? I highlight the anchor text in the text editor, I click the "link" button. I could have sworn that there used to be an option in the dialogue box that pops up.
Technical SEO | | CsmBill0 -
Will an XML sitemap override a robots.txt
I have a client that has a robots.txt file that is blocking an entire subdomain, entirely by accident. Their original solution, not realizing the robots.txt error, was to submit an xml sitemap to get their pages indexed. I did not think this tactic would work, as the robots.txt would take precedent over the xmls sitemap. But it worked... I have no explanation as to how or why. Does anyone have an answer to this? or any experience with a website that has had a clear Disallow: / for months , that somehow has pages in the index?
Technical SEO | | KCBackofen0 -
Should i Noindex my privacy policy page?:
Hi, We have a privacy policy page but it can be found at Copyscape and might affect Google Panda content farming. My questions is, should i Noindex my private policy page?:
Technical SEO | | chanel270 -
Can you have a /sitemap.xml and /sitemap.html on the same site?
Thanks in advance for any responses; we really appreciate the expertise of the SEOmoz community! My question: Since the file extensions are different, can a site have both a /sitemap.xml and /sitemap.html both siting at the root domain? For example, we've already put the html sitemap in place here: https://www.pioneermilitaryloans.com/sitemap Now, we're considering adding an XML sitemap. I know standard practice is to load it at the root (www.example.com/sitemap.xml), but am wondering if this will cause conflicts. I've been unable to find this topic addressed anywhere, or any real-life examples of sites currently doing this. What do you think?
Technical SEO | | PioneerServices0 -
Should each new blog post be added to Sitemap.xml
Hello everyone, I have a website that has only static content. I have recently added a Blog to my website and I am wondering if I need to add each new Blog post to my Sitemap.xml file? Or is there another way/better way to get the Blog posting index? Any advice is greatly appreciated!
Technical SEO | | threebiz0 -
Exclude Child URLs from XML Sitemap Generator (Wordpress)
Hi all, I was recommended the XML Sitemap Generator for Wordpress by the very helpful Keith Bloemendaal and John Pring - however I can't seem to exclude child URLs. There is a section Exclude items and a subsection Exclude posts. I have tried inputting the URLs for the pages I don't want in the sitemap, however that didn't work. So I read that you have to include a list of "IDs" - not sure where on earth to find that info, tried the page name and the post= number from the URL, however neither worked. I hope somebody can point me in the right direction - and apologies, I am a Wordpress novice, and I got no answers from the Wordpress forums so turned right back to SEOmoz! Cheers.
Technical SEO | | markadoi840 -
NoIndex/NoFollow pages showing up when doing a Google search using "Site:" parameter
We recently launched a beta version of our new website in a subdomain of our existing site. The existing site is www.fonts.com with the beta living at new.fonts.com. We do not want Google to crawl the new site until it's out of beta so we have added the following on all pages: However, one of our team members noticed that google is displaying results from new.fonts.com when doing an "site:new.fonts.com" search (see attached screenshot). Is it possible that Google is indexing the content despite the noindex, nofollow tags? We have double checked the syntax and it seems correct except the trailing "/". I know Google still crawls noindexed pages, however, the fact that they're showing up in search results using the site search syntax is unsettling. Any thoughts would be appreciated! DyWRP.png
Technical SEO | | ChrisRoberts-MTI0 -
Thoughts about stub pages - 200 & noindex ok, or 404?
With large database/template driven websites it is often possible to get a lot of pages with no content on them. What are the current thoughts regarding these pages with no content, options; Return a 200 header code with noindex meta tag Return a 404 page & header code Something else? Thanks
Technical SEO | | slingshot0