Sitemap Warnings
-
Due to an issue with our CMS, I had a bunch of URL aliases that were being indexed and causing duplicate content issues.
I disallowed indexing of the bad URLs (they all had a similar URL structure so that was easy). I did this until I could clean up the bad URLs
I then recieved a bunch of sitemap warnings that the URLs that I blocked URLs with robots.txt that were in the sitemap.
Isn't this the point of robots.txt? Why am I getting warnings and how can I get rid of them?
-
Irving -
Ok, so we took the restriction out of robots.txt while IT tries to fix the issue of URLs showing up on the sitemap that shouldn't.
Warnings haven't fallen off and now our sitemap is a day behind now as it's stuck in pending for almost a full day.
Any thoughts on what might be causing? I'm assuming this is impacting what's indexed and hurting our site.
-
Ok, so we took the restriction out of robots.txt while IT tries to fix the issue of URLs showing up on the sitemap that shouldn't.
Warnings haven't fallen off and now our sitemap is a day behind now as it's stuck in pending for almost a full day.
Any thoughts on what might be causing? I'm assuming this is impacting what's indexed and hurting our site.
-
Irving,
Totally get that and we're working to ensure they are no longer included in the sitemap.
Thanks,
Lisa
-
The purpose of your sitemap is to tell Google to go out and index the pages you specify. The purpose of the robots.txt is to tell Google not to index the page. The warning is likely just a precaution to let you know that you may have by accident requested them to block something in robots.txt. If you remove the URL's from your submitted sitemap the warnings should disappear. If you leave them, you will have warnings but Google should not index the content since your blocked it in robots.txt.
-
you are not supposed to include blocked URLs in the sitemap.xml files, or Google considers it wasting their crawl time. Are these automated sitemap.xml files?
You're basically saying "come index these pages i've listed, but don't index them!"
Remove the URLs that are blocked content (or rerun/regenerate them) and resubmit the sitemaps and the warnings will go away.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Inbound Links Warning
I got the following error about our domain name in Link Explorer. "You entered the URL freexy.net which redirects to youlovelife.com/?domain=freexy.net. Click here to analyze youlovelife.com/?domain=freexy.net instead." Can you give me an advice about this problem?
Moz Pro | | ligia.tatucu0 -
What Should I Do About Duplicate Title Warning From Category Pages Of Store?
I know a lot of the MOZ warnings can be ignored, however, I'm trying to figure out of this one should be added to that list: my store has urls setup like this for categories: https://www.mysite.com/sweaters https://www.mysite.com/sweaters/page/2 The meta title is "Sweaters" for both pages. Is that bad practice? I don't think I can automatically change the meta title to to Sweaters Page 2 or even want to. or should I do that? Or just ignore these type of warnings?
Moz Pro | | IcarusSEO0 -
How to fix the Crawl Diagnostics error and warnings
hi im new to the seo world and i dont know a lot about it , so after my site get crawled i found 1 error and 151 warning and 96 notices , it that bad ?? and plz cam someone explain to me how to fix thos problem , a will be very thankful
Moz Pro | | medlife0 -
I have had ro resubmit my sitemap to google, Bing & yahoo. Does SEOmoz automatically pic that up?
Hi there I am monitoring this website for a client: www.smsquality.com Someone on their side had gone and blocked the sitemap from being crawled and also in some form or another removed it as well. (Confusing I know) However I have gone and recreated the sitemat for these guys allowing robots to crawl the site, resubmitted it to all major search engines. My question is; Will SEOmoz be ableto crawl the site like it usually does and give me proper results for my Keywords placed into the Keywords Capmaign as well as give me Onsite page crawls using these keywords with proper results? Thanks in Advance Ray
Moz Pro | | RayHay0 -
Does anyone have suggestions for a good XML Sitemap Generator?
Does anyone have any suggestions on a good XML Sitemap Generator? Also interested in best practices and tips for updating the XML Sitemap. I typically have relied on my web developers to do this however it seems that they have not been setting this up with SEO in mind.
Moz Pro | | webestate0 -
How does SEOMoz crawl sites? Does it follow the sitemap?
I only ask because in my report it returned a few "no follow" links. Was wondering how SEOMoz operates.
Moz Pro | | Pikimal0 -
Campaign warnings and recommendations
How crucial is it to follow / fix the campaign warnings. For example, even though our website is ranking well the campaign report is stating that we've too many links on the majority of pages and the title element is too long. My question is if I make changes based on these recommendations, do you think that we will see a dramatic improvement in our rankings ?
Moz Pro | | NeilTompkins0 -
SEOMoz's Crawl Diagnostics showing an error where the Title is missing on our Sitemap.xml file?
Hi Everyone, I'm working on our website Sky Candle and I've been running it as a campaign in SEOmoz. I've corrected a few errors we had with the site previously, but today it's recrawled and found a new error which is a missing Title tag on the sitemap.xml file. Is this a little glitch in the SEOmoz system? Or do I need to add a page title and meta description to my XML file. http://www.skycandle.co.uk/sitemap.xml Any help would be greatly appreciated. I didn't think I'd need to add this. Kind Regards Lewis
Moz Pro | | LewisSellers0