Massive drop off in Google crawl stats
-
Hi
Could i get a second opinion on the following please. ON a client site we seem to have had a massive drop off in google crawling in the past few weeks, this is linked with a drop in search impressions and a slight reduction in penalty. There are no warning messages in WMT to say the site is in trouble, and it shouldn't be, however cannot get to the bottom of what is going on.
In Feb the Kilobytes downloaded per day was between 2200 and about 3800, all good there. However in the past couple of weeks it has peaked at 62 and most days are not even over 3! Something odd has taken place.
For the same period, the Pages crawled per day has gone from 50 - 100 down to under 3.
At the same time the site speed hasn't changed - it is slow and has always been slow (have advised the client to change this but you know how it is....) Unfortunately I am unable to give the site url out so i understand that may impact on any advice people could offer.
Ive attached some screen shots from WMT below.
Many thanks for any assistance.
-
I use it on daily basis, because it gives me quite a good image of what we might miss with manual checking. I hope it will do the trick for you too.
BTW: good to know article for Screaming Frog: http://www.seerinteractive.com/blog/screaming-frog-guide/
That article might speed up things for you and Screaming Frog (for future use )
-
Thanks, will give screaming frog a go. I own the tool but rarely seem to use it. Hopefully that finds out what is amiss.
-
Hi Carl,
What I'd try to do, is to make some crawls via Screaming Frog to the website, try to analyze how your content is crawled, what could have gone wrong.
Also try to keep a timeline of implementations on the website. (I work at a quite big website, and keeping everything organized for myself helps me analyze issues as yours. I will try to make a copy of my excel timeline and post it here in the following days, if you'd like to )
Another thing that you could do, is to fetch and render the pages. Check for anything out of the ordinary.
These are a few ideas of where I would start digging.
I hope it helps. Keszi
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Does google look at H3 tags?
I've had someone tell me that google doesn't pay attention to H3 tags -- only H1 and H2. I haven't found much online to back this up or discredit it; thought I'd ask the Moz community!
Technical SEO | | LivDetrick5 -
Sudden Drop in Rankings
I manage a site and this past month have noticed some pretty significant drops in rankings for some of our main keywords. I'm trying to understand why and what I can do to correct it. We did do some 301 redirects a little over a month ago so I've been wondering if that has anything to do with it as Google indexes the new URLs. One thing I did notice in Moz is that it is showing that Google is indexing the new URLs but associating them with the wrong keywords in the wrong location. The company services three locations and each location has a service page on the site. But Moz is showing that in Atlanta, the indexed URL is the Nashville one, etc. Is there any way to affect that? Another thing to note is that organic search traffic is actually up so it doesn't appear to be affecting them which I find a bit strange. Just looking for any insight as to things I should be looking into to explain the sudden drops. Thanks!
Technical SEO | | maghanlinchpinsales0 -
Massive drop in sales - how do I find out why?
I am working with my client managing/maintaining their online store.We have been making big improvements to onsite SEO, improved URL structures (complete with re-directs) etc, we use Yoast Premium to optimise pages for search and Yoast Local. We have a bounce rate of around 20%, however, suddenly in mid August, our revenues virtually vanished overnight. Moz is reporting a search visibility drop of 51%, while visitor levels remain fairly average, certainly no drops that can be compared with the sudden loss of conversions. I am at a loss and cannot fathom why our sales have dropped. Can anyone suggest where I might look? Many thanks Bob
Technical SEO | | SushiUK0 -
Why HTML entities gets crawled as content keywords in Google search console?
My Google search console shows HTML parameters such as div, class, img, src, gif, align as content keywords, but why google crawls HTML parameters as keywords? because of this, I would be losing traffic for my on-page content keywords. Please let me know how to solve this. Thanks, Jenifer
Technical SEO | | Jenifer300 -
Google's ability to crawl AJAX rendered content
I would like to make a change to the way our main navigation is currently rendered on our e-commerce site. Currently, all of the content that appears when you click a navigation category is rendering on page load. This is currently a large portion of every page visit’s bandwidth and even the images are downloaded even if a user doesn’t choose to use the navigation. I’d like to change it so the content appears and is downloaded only IF the user clicks on it, I'm planning on using AJAX. As that is the case it wouldn’t not be automatically on the site(which may or may not mean Google would crawl it). As we already provide a sitemap.xml for Google I want to make sure this change would not adversely affect our SEO. As of October this year the Webmaster AJAX crawling doc. suggestions has been depreciated. While the new version does say that its crawlers are smart enough to render AJAX content, something I've tested, I'm not sure if that only applies to content injected on page load as opposed to in click like I'm planning to do.
Technical SEO | | znotes0 -
Google not pulling my favicon
Several sites use Google favicon to load favicons instead of loading it from the Website itself. Our favicon is not being pulled from our site correctly, instead it shows the default "world" image. https://plus.google.com/_/favicon?domain=www.example.com Is the address to pull a favicon. When I post on G+ or see other sites that use that service to pull favicons ours isn't displaying, despite it shows up in Chrome, Firefox, IE, etc and we have the correct meta in all pages of our site. Any idea why is this happening? Or how to "ping" Google to update that?
Technical SEO | | FedeEinhorn0 -
Google Index Speed Opinions
Hello Everyone, Under normal circumstances, new posts to my site are indexed almost instantly by Google. I know this because an occasional search with quotation marks surrounding the 1st paragraph of text displays my newly published page. I use this tactic from time to time to ensure contributors aren't syndicating content. My question is this: I've noticed over the last day or so that my newly published articles are not yet indexed. For example, an article that was published over 24 hours ago does not appear to be indexed yet. Is this cause for concern? Is there an average wait time for indexation? XML issue? Thanks in advance for the help/insight.
Technical SEO | | JSOC0 -
Google India Rankings Drop Suddenly
Keyword: indian wedding planner page:www.hukamievent.com/indian-wedding-planner.php It didn't drop, it just disappeared. No, I didn't do anything to be penalized at all and I am still in there index. I know this question has been asked many times here, while I apply all the suggestions given to other people with similar problem, here's a quick question I need to ask, I made following changes recently, could either of these be the reason for ranking decline? In google webmasters' tools, I had changed geographic target from - india, to- nothing, few weeks ago. could this be the reason. Around 2 weeks ago I paid someone to submit articles to 500 article directories, same article spun many times, he did the job in around 6 days and got back to me with a list of around 150 submissions with as many backlinks, all with same anchor text, "indian wedding planner", one of my targeted keywords. Before the submissions I was on page 9, yes that was nine, and just after the submission I was at page 2 position 1, within a matter few days. I saw my the targeted page improve in ranking and jump on to page one position 10, for a short while, before it disappeared. is this something they call "google dance"? My website is fluctuating on google SERP for many other keywords too, in most cases, my ranking fell. P.S.:With the same keyword,"indian wedding planner" my rankings in google US improved, I am on position 6 page 1. please enlighten fellow mozzers!
Technical SEO | | virtualhandshake0