Webmaster tools crawl stats
-
Hi
I have a clients site that was having aprox 30 - 50 pages crawled regularly since site launch up until end of Jan.
On the 21st Jan the crawled pages dropped significantly from this average to about 11 - 20 pages per day.
This also coincided with a massive rankings drop on the 22nd which i thought was something to do with panda although it later turned out the hosts had changed the DNS and exactly a week after fixing it the rankings returned so i think that was the cause not panda.
However i note that the crawl rate still hasn't returned to what it was/previous average and is still following the new average of 10-20 pages per day rather than the 30-50 pages per day.
Does anyone have any ideas why this is ?
I have since added a site map but hasnt increased crawl rate since
A bit of further info if it helps in any way is that In the indexed status section says 48 pages ever crawled with 37 pages indexed. There are 48 pages on the site. The site map section says 37 submitted with 35 indexed.
I would have thought that since dynamic site map would submit all urls
Any clarity re the above much appreciated ?
Cheers
Dan
-
Hi EE3
Sorry but im not otherwise would have done so already. If it was one of my own projects then no problem but its a client
Cheers
Dan
-
Hi Dan,
If you're comfortable with it, post your client's website and sitemap link here for people to take a look at.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Some bots excluded from crawling client's domain
Hi all! My client is in healthcare in the US and for HIPAA reasons, blocks traffic from most international sources. a. I don't think this is good for SEO b. The site won't allow Moz bot or Screaming Frog bot to crawl it. It's so frustrating. We can't figure out what mechanism they are utilizing to execute this. Any help as we start down the rabbit hole to remedy is much appreciated. thank you!
Technical SEO | | SimpleSearch0 -
Should I resubmit a 301 redirected domain in Webmaster Tools
We recently switched over a .com site to a new server. The .com site had a .co.uk domain redirecting to it previously, but when the switchover happened, the .co.uk was forgotten about. We have now realised what has happened, but not before taking a hit with our rankings. The .co.uk is still indexed in Google and now that we have sorted the redirects they are pointing to the right places. My question now; is there anything further I need to do? I know that the .co.uk will soon be removed from the SERPs, but I just want to make sure I haven't forgotten anything.
Technical SEO | | Ben_Malkin_Develo0 -
Increase in pages crawled per day
What does it mean when GWT abruptly jump from 15k to 30k pages crawled per day? I am used to see spikes, like 10k average and a couple of time per month 50k pages crawled. But in this case 10 days ago moved from 15k to 30k per day and it's staying there. I know it's a good sign, the crawler is crawling more pages per day, so it's picking up changes more often, but I have no idea of why is doing it, what good signals usually drive google crawler to choose to increase the number of pages crawled per day? Anyone knows?
Technical SEO | | max.favilli1 -
Change of address form in Webmaster Tools
I have changed my domain from .co.uk to .com and tried to submit a change of address form in Google Webmaster Tools. However it seems because I redirect my home page onto www.domain.com/index.php I cannot submit the form as it is not a domain name. Is there a way round this? It is not currently an option to move away from www.domain.com/index.php Thanks in advance
Technical SEO | | TheHutGroup0 -
Google Webmasters News Errors ressolution
Hello to the community, i had a sudden increase from just a couple to 50 someting Google Webmaster News Errors. The two areas affected are Content of article and date of article.I found a very good article in SEOMoz about Google Webmasters, but it was published before the changes early last year were done in Google Webmasters. http://www.seomoz.org/blog/how-to-fix-crawl-errors-in-google-webmaster-tools The people that have been asking the same question in the internet have not yet received replies from Google and the Google support replies dont make it really clear. http://support.google.com/webmasters/bin/answer.py?hl=en&answer=93994 Any views experiences with this. My site is in Google News, but we do not have a Google News Sitemap. Thanks, Polar
Technical SEO | | Polarstar0 -
Received Google Webmaster Tools notice of detected unnatural links , but no negative impact on ranking and traffic, What should i do next?
Hello, On May 19 , 2012 Google webmaster sent notification "Google Webmaster Tools notice of detected unnatural links to " both sites haven't lost any ranking or traffic as yet. I am worried, Should i panic,what should i be doing , will it be going down anytime soon? how to naturally build links?
Technical SEO | | conversiontactics0 -
My report only says it crawled 1 page of my site.
My report used to crawl my entire site which is around 90 pages. Any idea of why this would happen? www.treelifedesigns.com
Technical SEO | | nathan.marcarelli0 -
Webmaster Tools finding phantom 404s?
We recently (three months now!) switched over a site from .co.uk to .com and all old urls are re-directing to the new site. However, Google Webmaster tools is flagging up hundreds of 404s from the old site and yet doesn't report where the links were found, i.e. in the 'Linked From' tab there is no data and the old links are not in the sitemap. SEOmoz crawls do not report any 404s. Any ideas?
Technical SEO | | Switch_Digital0