A Serious drop in Pages crawled per day
-
On 21st April ,I spotted a sudden decrease in pages crawled per day.Previously it was about 5,000 bust after the drop it reached to 225.From the crawl rate never spiked.
Here is my website url - http://www.wpstuffs.com/
-
What strikes me is at the same time as your pages crawled fell off a cliff your page load time doubles. What changed round about then that would have caused an increase in load times?
If that was around the time you switched CDN, I'd be double and triple checking the new one was set up perfectly and is an improvement on cloudflare. Did all your pages suddenly get markedly bigger, perhaps due to another plugin being added? Pingdom tools is also great for picking apart load times.
That would be where I'd start...
-
Iam using WordPress blogging platform and checked the GWT and Analytics but there were no break in code.
Im using Google Pagespeed CDN service ? will that cause such problems.Previously I was using Cloudflare.
Current PageSpeed is 92 and load time is under 3seconds.I checked the pingdom report and it doesn't show any down time stats.
-
Yes, well there are a couple of things to consider and test.
First and foremost, check your GWT code and make sure that nothing's change. Same with Google Analytics.
Have you changed platforms?
If not, I would say it's "Hoy Crap Batman" time and you need to start analyzing everything. What's your Pagespeed? Is your site online? If so, was it offline earlier in the week? (Maybe you weren't checking every minute) If your site is hosted by someone else, get on the phone and demand some answers.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
75% Overnight Drop in Organic Search Traffic
On April 10th my organic search clicks dropped 75% overnight. I have never seen anything like it. What Google algorithm change could have caused this? I have no manual actions and my indexed page count is about the same. I have noticed that several queries that I was number one for including my brand have dropped by anywhere from two to ten spots. The brand dropping out of the first spot is what really gets me. There is nothing similar to it at all. My speed score is moderate, so I don't think that is it. My site was down most of the day on the 9th or 10th, but that has never caused a drop in search clicks and the next day they were about the same. I noticed that the CDC now occupies the number one spot for my brand. Even though the exact brand name is nowhere in the text of the CDC page. I think this might be due to Google trying to help official health organizations do better due to COVID19, but the queries I have dropped on have nothing to do with Coronavirus. Also, none of my other sites have seen this type of problem. Only the health site seems affected. I recently did a press release campaign and my link counts are up, so I don't that is it either. The brand page is https://stdcarriers.com and an example of an effected query is Celebrities with STDs.
Algorithm Updates | | STDCarriers0 -
Need only tens of pages to be indexed out of hundreds: Robots.txt is Okay for Google to proceed with?
Hi all, We 2 sub domains with hundreds of pages where we need only 50 pages to get indexed which are important. Unfortunately the CMS of these sub domains is very old and not supporting "noindex" tag to be deployed on page level. So we are planning to block the entire sites from robots.txt and allow the 50 pages needed. But we are not sure if this is the right approach as Google been suggesting to depend mostly on "noindex" than robots.txt. Please suggest whether we can proceed with robots.txt file. Thanks
Algorithm Updates | | vtmoz0 -
On-Page Markup: Still a Worthwhile Practice?
So I have a question for the community that hopefully someone can help me with. Previously, whenever I created/worked on a website, when I would create or edit the content, I would bold the keywords, italicize certain items, add internal/external links and generally mark-up the content. More recently, however, I've noticed that both my client and many of their leading competitors have abandoned this practice. Now, it appears that all the text appears as plain text, there are rarely bold or italicized items and there does not seem to be as much emphasis on inserting internal/external links. While I understand the ladder to still be an effective/holistic approach to SEO, I'm wondering why the former (the bold, italicized, text variation) has gone by the wayside. So with that, is adding bold/italicized text still a worthwhile SEO technique and is it something I should continue applying to sites I work on? Please advise.
Algorithm Updates | | maxcarnage0 -
Should my canonical tags point to the category page or the filter result page?
Hi Moz, I'm working on an ecommerce site with categories, filter options, and sort options – teacherexpress.scholastic.com. Should I have canonical tags from all filter and sort options point to the category page like gap.com and llbean.com? or have all sort options point to the filtered page URL like kohls.com? I was under the impression that to use a canonical tag, the pages have to have the same content, meaning that Gap and L.L. Bean would be using canonical tags incorrectly. Using a filter changes the content, whereas using a sort option just changes the order. What would be the best way to deal with duplicate content for this site? Thanks for reading!
Algorithm Updates | | DA20130 -
How can I check Googles Page Cache ?
Hi I use to have a handy tool in Firefox (Google Toolbar) that was very handy for checking page ranks and what date a page had been cached. For a while with the newer versions of Firefox I cannot seem to locate this useful tool, Can anybody recommend any useful tools for checking the above. Thanks Adam
Algorithm Updates | | AMG1000 -
Dropped off cliff for a partic keyword & can't find out why
At the beginning of Dec we ranked consistently in the top 3 for the keyword 'Suffolk' for the site www.suffolktouristguide.com (apge rank 4, thousands of quality inboud links, site age 5 years +). Since then we've been falling off a cliff and today aren't even in the top 50 for this search term, but most of our othr search terms are unaffected. Our SEOMoz grade remains A for 'Suffolk' and we haven't changed anything in that time that could have had such a material effect (knowingly at least). A similar issue happened to my other site www.suffolkhotelsguide.com back in April and it hasn't recovered despite grade A's on the homepage and key pages. We've checked internal broken links, page download times, external links (used the disavow tool and reconsideration request and got back 'We reviewed your site and found no manual actions by the webspam team that might affect your site's ranking in Google'); etc etc Any thoughts on what I can try next? All suggestions appreciated as I am completely stuck (& have spent a fortune on 'SEO experts' to no effect).
Algorithm Updates | | SarahinSuffolk0 -
Google +1 link on Domain or Page?
Since its release, I've seen Google +1 being used across an entire domain but only reference the root href in the code snippet. At the same time, you see other sites use +1 more naturally with the button being specific to the page you're on. What's your take on this? To clarfiy, do you add: or .. on each page.
Algorithm Updates | | noeltock0 -
Why would my product pages no longer be indexed in Google?
Our UK site has 72 pages in our sitemap. 30 of them are product pages which take a productid parameter. Prior to 1st Feb 2011, all pages were indexed in Google but since then all of our product pages seem to have dropped from the index? If I check in webmaster tools, I can see that we have submitted 72 pages and 42 are indexed. I realise we should have some better url structuring and I'm working on that but do you have any ideas on how we can get our product poages back into googles index http://www.ebacdirect.com
Algorithm Updates | | ebacltd0