Google crawl rate dropped after we activated CloudFront
-
Hello!
Previously we've been using Amazon CloudFront for our static content (js, css etc). But to be able to reduce load on our origin servers and to be able to give our international users a good user experience we decided to deliver a couple of our sites through CloudFront.
We noticed very nice drops in page load time, but when checking Google webmaster tools we noticed that all CloudFront-activated sites got a huge drop in pages crawled per day (from avg ~3500 to ~150). Also one of the sites have issues with the Google sitemaps (just marked as "Pending" in GWT) and no new pages or updated pages seems to be updated in the Google SERP. The rest of the sites gets some updates on the Google SERP, but very few compared to before CloudFront activation.
Is there anybody here who have experience in full site delivery through CloudFront (or other CDNs) and effects on SEO/Google? Would be very glad for any insights or suggestions. The risk is that we need to remove CloudFront if this just continues.
-
My very best educated guess (after talking with dev friends here at Moz) is that CloudFront is supposed to be used for long-term storage. This means the content isn't expected to change frequently. This is great for images and other static things. However, this is also a signal to Google not to crawl your site as often as it won't be changing that often.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Unable to site crawl
Hi there, our website was revamped last year and Moz is unable to crawl the site since then. Could you please check what is the issue? @siteaudits @Crawlinfo gleneagles.com.my
Technical SEO | | helensohdg380 -
Rankings Dropped to Nothing
We're kind of in crisis mode, as our ad revenue is about to take a huge hit. Hoping someone can help me figure out what to do next. Site: https://indoorgardening.com Here's what I did (below) that I think broke things somehow. I'm clearly not an SEO expert but thought I was making things better. And things did improve over the last week or so but then fell apart 2 days ago. 1. Most posts did not have a Yoast focus keyword. I added keyword phrases and used Yoast suggestions to optimize for that and for readability. 2. In some cases I changed post titles to better reflect the keyword phrase. 3. In some cases I changed the slug per Yoast's suggestion and did a 301 redirect from the old slug to the new one. 4. I used Grammarly to fix all spelling, grammar, punctuation, etc. 5. Some Yoast-suggested changes that were made: Image alt tags, subheading structures, adding keyword to subheadings, first paragraph, and meta description, changed sentence length for those over 20 words to clean up the text, added transition words where applicable, reworded passive voice sentences, added internal links when needed, eliminated consecutive sentences (first word), improved Flesch reading ease when necessary. 6. I also added or changed Amazon affiliate links where needed and swapped out images when necessary. Results:
Technical SEO | | Jbyron
I started this project about 3 weeks ago. On 11/29 we had one of our highest traffic days, with 1017 hits coming from Google. On 11/30, 257 hits came from Google, and on 12/1, 3 (three!!) hits came from Google. At this point, 82 of 89 posts have a double green "Good" score in Yoast; 6 are "OK" and 1 does not have a focus keyword designated. Thanks in advance for any help anyone can provide. -John0 -
Organic traffic drops a lot
Please check my url that is https://www.drmalpani.com , it was first build up in Joomla and then for security reason, I moved from Joomla to Laravel but in that mean time, I have facing steep fall in organic traffics. It was earlier 10,000 users per day and now I can see only 2500 users per day. I don't know what went wrong, each url are same as it was in Joomla as well I have worked out on all 404 urls too. Looking forward to your guidance on this. Thanks, Bharat
Technical SEO | | DrMalpani0 -
Google index graph duration in Google Webmaster Tools
Hello guys, I wonder, my sites are currently being indexed every 7 days, exactly. At Index Status page in GWT. However, this new site gets updated almost everyday, how can I ask google to index faster and more frequently/almost daily? Is it about SItemap.xml frequency ? I changed it today to Daily. Thanks!
Technical SEO | | mdmoz0 -
My rank suddenly drop
Hi before i subscribe for MOZ i have hired a seo expert to do an optimization for my escort website, it was really hard since im not knowledgeable and have no clue about it also my niche is kinda hard as what everyone says. The SEO did a good job ranking my site but in less than 2 weeks my ranking goes down and every time it happen what i do is call him and tell him about the situation and in just few days my rank goes up again and im in page 1 of google. My question is, is that normal? like ranking drop in less than 2 weeks and he could easily rank me up in just few days? I pay just to make sure that im on page 1, so every time he put my site on page 1 i pay for it. Gawd i still have no clue whats going on, i have a lot to learn, black hat, white hat, proper back linking, etc etc so i'd really love to know if its normal cause if it is then no point for me to learn SEO as he is doing job well done. I was just afraid that he is doing this to rip me, not that i dont trust him but you know this days everyone needs something for a living 🙂
Technical SEO | | geefex6nsy0 -
Getting Recrawled by Google
I have been updating my site a lot and some of the updates are showing up in Google and some are not. Is there a best practice in getting your site fully recrawled by Google?
Technical SEO | | ShootTokyo0 -
Traffic drop with no explaination
Hello Everyone, I made a post a few months ago about one of my sites, Poker On A Mac dropping off the face of google. I found out that it was hacked, and redirecting traffic from google to other, pharmacy websites. Doing some research, I found out it was the WordPress PharmaHack, and we ended up recovering from it. Now, since October 23rd traffic has went down again mainly organic traffic, I tried to see if it was hacked again, with the same thing and their was nothing I noticed. I was wondering if anyone could give me any insights as to why this happened? Thanks Zach CwFda.png
Technical SEO | | Zachary_Russell0 -
Remove Site from Google
How can I get my website out of google? I want all pages completely gone. Thanks!
Technical SEO | | tylerfraser0