Google Webmaster tools: Sitemap.xml not processed everyday
-
Hi,
We have multiple sites under our google webmaster tools account with each having a sitemap.xml submitted
Each site's sitemap.xml status ( attached below ) shows it is processed everyday except for one
_Sitemap: /sitemap.xml__This Sitemap was submitted Jan 10, 2012, and processed Oct 14, 2013._But except for one site ( coed.com ) for which the sitemap.xml was processed only on the day it is submitted and we have to manually resubmit every day to get it processed.Any idea on why it might?thank you
-
My initial reaction was that this is more likely technical than something Google is doing - checking the load-time is a good idea. Make sure the sitemap validates and there's nothing odd about it. If you manually re-submit it, does it seem to take?
-
Just so I am clear, you have been waiting and finally Google processed it, or was it sitting there and someone took an action which caused Google to process it?
I am surprised that nothing happened for nearly two years. Has the site had traffic, etc.? Any warnings, manual actions, etc.?
Thanks
-
Thanks Paul,
On custom crawl setting, I just verified and they remain same same across all our sites.
Yes, sitemap is dynamic but rendered via cache which refreshed when new content get published, will check on the load time.
thank you
-
Thanks Robert,
All our sites are getting indexed for sure, but one site ( coed.com ) sitemap.xml on GWT says it was processed only on the day it submitted, while other sites ( collegecandy.com and bustedcoverage.com ) sitemap.xml GWT was getting processed everyday
Our sitemap.xml on all sites updates automatically when new content get published so I believe it has to be processed every day
-
Any chance there's been a custom crawl setting accidentally added to your Google Webmaster Tools, Robert? Some devs do this during development or it can happen accidentally.
Also, your sitmemap takes a ridiculously long time to load - well over 15 seconds for me and over 18 seconds using webpagetest.org. It could be that Google simply isn't waiting for the page to load when it tries to visit. If the sitemap's being generated dynamically, you may have a rendering problem. Otherwise there's something borked when a 50kb file takes that long.
Might also want to try submitting it through Bing Webmaster Tools and see if they are better able to index it consistently for comparison?
Bit of a head-scratcher. Hope that gives you a starting point.
Paul
-
Robert,
There is no screenshot attached, but I am unaware of sitemaps being processed daily by search engines. What are you trying to achieve by continuously resubmitting the sitemap?
The site is indexed, correct? And when you look at crawl stats it is showing the site being crawled on some semi regular basis? Google does not process your sitemap every day.
Hope that helps,
Robert
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
.xml sitemap showing in SERP
Our sitemap is showing in Google's SERP. While it's only for very specific queries that don't seem to have much value (it's a healthcare website and when a doctor who isn't with us is search with the brand name so 'John Smith Brand,' it shows if there's a first or last name that matches the query), is there a way to not make the sitemap indexed so it's not showing in the SERP. I've seen the "x-robots-tag: noindex" as a possible option, but before taking any action wanted to see if this was still true and if it would work.
Technical SEO | | Kyleroe950 -
Bing Webmaster Tools Incompatibility Issues with new Microsoft Edge Browser
Our client received an email from Bing WMTs saying "We have identified 4 known issues with your website in Microsoft Edge – the new default browser for Windows 10 and Bing – Of the four problems mentioned, only two seem to be relevant (maybe) We’ve found that this webpage may include HTML markup that treats Microsoft Edge differently from other modern browsers. The new EdgeHTML rendering engine for Microsoft Edge is document-mode agnostic and designed for fast, modern rendering. We recommend that you implement one code base for all modern browsers and include Microsoft Edge as part of your modern browser test matrix. **We've found that this webpage may have missing vendor-specific prefixes **or may have implemented vendor-specific prefixes when they are not required in common CSS properties. This may cause compatibility problems with how this webpage renders across different browsers. Last month the client received 20K visitors from all IE browsers and this is significant enough to be concerned about. **Are other folks making changes to their code to adapt to MS Edge? **
Technical SEO | | RosemaryB0 -
Webmaster Tools Search Queries Data Drop
Hi I'm seeing a significant drop in search queries being reported for a client in GWT starting on the 7th Feb. I have seen a few articles on SERound Table etc saying that many are reporting probs like delays etc with GWT updating its data, such as these ones: https://www.seroundtable.com/google-webmaster-tools-data-stalled-19854.html https://www.seroundtable.com/google-webmaster-tools-analytics-data-19870.html However these seem to suggest the problem is simply a delay with displayed data being updated, in the case im looking at the data is up to date but showing an increasing decline. When i look at Analytics data though the data is completely different. For exmaple GWT says on the 21st Feb there were 23 impressions with zero clicks but Analytics says there were 6 clicks/sessions from organic search. I take it this means that there is a likely problem with GWT data and I shouldn't worry ? All Best Dan
Technical SEO | | Dan-Lawrence0 -
I disappeared from Google, but not Bing?
A few weeks ago the company I work for created a website called Nabceptraining.com in hopes to find clientele who are looking to become NABCEP Certified. So we installed WordPress and a nice looking theme that we edited, and we saw that without any real content on the page we were already rank 12 on Google! We never even submitted our site. So we decided to Search Engine Optimize our website to gain even better rankings. So on the first day we added an SEO Tool that would allow us to change the page titles and descriptions on our site, a Sitemap generation tool so we could submit the site to Google, Bing, and Yahoo. At the end of the day we submitted the site and was beginning to create our content. That next morning our ranking was gone and we were not to be found on Google, but we were rank 3 on Bing!? What happened? Why did we disappear? We didn't receive any messages from Google on the webmaster tools saying we were blacklisted. Does anyone have an idea?
Technical SEO | | edlinkim0 -
XML Sitemap Issue or not?
Hi Everyone, I submitted a sitemap within the google webmaster tools and I had a warning message of 38 issues. Issue: Url blocked by robots.txt. Description: Sitemap contains urls which are blocked by robots.txt. Example: the ones that were given were urls that we don't want them to be indexed: Sitemap: www.example.org/author.xml Value: http://www.example.org/author/admin/ My issue here is that the number of URL indexed is pretty low and I know for a fact that Robot.txt aren't good especially if they block URL that needs to be indexed. Apparently the URLs that are blocked seem to be URLs that we don't to be indexed but it doesn't display all URLs that are blocked. Do you think i m having a major problem or everything is fine?What should I do? How can I fix it? FYI: Wordpress is what we use for our website Thanks
Technical SEO | | Tay19860 -
Do Seomozers recommend sitemaps.xml or not. I'm thoroughly confused now. The more I read, the more conflicted I get
I realize I'm probably opening a can of worms, but here we go. Do you or do you not add a sitemap.xml to a clients site?
Technical SEO | | catherine-2793880 -
Unnatural links in webmaster tools
Google Webmaster Tools notice of detected unnatural links to my site.I download latest link from google webmaster tool and decided to remove links from last four months.I also got several links from directory sites and i want to remove the links, how to delete links from directory sites ? I f there is no way to delete directory link , please let me know other option to get rid of this issue.
Technical SEO | | Alick3000 -
Remove Site from Google
How can I get my website out of google? I want all pages completely gone. Thanks!
Technical SEO | | tylerfraser0