Database Crash affecting our rankings
-
Here is one for you. We had a database crash, and was down for an hour and 15 minutes, on Dec 18th. During this time our system automatically sent out a 301 redirect to our product home page. (iboats.com/boat-parts-accessories/dm/)
On Dec 24th I noticed a huge ranking drop on a lot of our keywords, and some keywords that were serving up the redirect page from our database crash, in place of its normally ranked page.
So my assumption is that Google was crawling our site during the database crash and caching the redirect pages in place of our regularly ranked pages. So when a search query is made that would normally display our ranked page, it is now displaying the redirect page from the crawled cache.
And on other keywords where we normally would rank for, Google no longer has that page cached, so it drops it from ranking.
My question: Does this assumption sound accurate? If so, I'm assuming over time that our past ranked pages will show up once again after Google recrawls those pages and saves them in their cache.
I have several sites that were affected with this:
Thanks in advance for your input
-
Glad to hear you appear to be back on the right track
-
Update:
After unblocking Google, almost all of our rankings are back up, and in some cases a little improved. I was surprised how quickly Google recrawled and ranked the pages that dropped.
I did do a Fetch and submit on all my websites and pages. I'm not sure how Fetches are attended to by Google, but it appears they are.
I have an IT guy who doesn't feel a Fetch makes any difference. He says Google will crawl when they want to and probably does not pay attention to the Fetch request.
-
I wanted to get back to report the cause of our problem with a lot of our rankings dropping. We originally thought it was due to a database crash. And that still could be affecting some things.
We moved our servers to a new location. A couple of weeks ago, we switched over to some new IP addresses. When we did so, a program started blocking Google's IP addresses because they were hitting our servers with a lot of hits, and put those IP addresses on a blocked list. So our pages on some of the websites stopped getting crawled, and thus rankings dropped. We discovered the problem when exploring our Webmaster Tools Crawl Stats and noticed that the "kilobytes downloaded per day" had flat-lined. Doing a fetch on selected pages we could see that it was not fetching the page. So after unblocking those IP addresses, the pages got crawled, our crawl stats went back up, and our rankings started going back to where they were. We still have some rankings that have not bounced back, but assume they will as new crawls re-index those pages.
A lesson learned for me was to pay attention to the crawl stats. Any drastic variations could be a sign of a problem.
Brad
iboats.com -
I'm pretty sure that GWT is a little vague in terms of times that Googlebot visited with just the crawlrate shown on a day by day basis and there's some lag between now and when it happened. CPanel will show when spiders visited in latest visitors from memory, however I think that rather than looking back it would be better to simply ensure that this doesn't happen again with the 302 rather than the 301. Maybe refreshing of your sitemap submissions in GWT would help to get some crawling going on with a ping or two to get Googlebot back soon.
-
Also, can't we use GWT to find out the exact times or at least approximate times of when Google crawled a site? Or can we not find that information so precisely? In this case, it does seem that the 301 vs. 302 is the key problem, but also the fact that the system for getting rid of the 301 didn't work out properly.
-
It seems extreme that Google would react within an hour and 15 mins but who knows nowadays. Keep an eye on your GWT account and hopefully if you have a decent crawl rate things should remedy themselves. Stephen Salstrand made a good point. 302's rather than 301's should help. It's great to have something in place like a redirection for user experience in case of database failure, however, as Stephen said, it should be made clear to Google that it's only temporary.
-
Investigating it further, we have some permanent redirects that are still hanging. I've got my IT guys working on it as we speaking.
I've got to run, but I would like to revisit this soon. Once we have determined the problem, I will post it here.
Thanks for your opinions. They have helped.
-
Your assumptions are globally right. However, I have never seen Google react that quickly to a 301 redirect or even a crash, I always tell my customers "if you have a temporary issue with 404 and 500 erros, don't worry about it too much, Google won't react unless it is a permanent issue that lasts over three or four crawls".
This is why I would also investigate a penalty that just happened to take effect shortly after the crash. To rule that out, could you please tell us if some of your major keywords are still ranking well?
I don't think you sould worry too much about that, just wait a few days and see if some pages recover.
-
Good point on the 301's. I'll look into that.
Thanks, for your input!
-
I think your assumptions are on par. However, make sure to review that the 301 has been removed. Also, if you still want that auto redirect to happen on a crash, make it a 302 and not a 301 as a 301 will suck a lot of life out of your site (since SEs see it as permanent).
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Ranking fluctuation
According to Miles Rank tracking I have a keyword that is fluctuating by 5 or 6 positions in a 24-hour period every time it goes 10 down to 15 and back to 10 again with very little stopping point in between and keeps fluctuating like that. As far as I know the website is stable it uses Magento 1.9 and is a fixed category page. The page URL is relatively new as we migrated from an old site with a different URL structure I added 301s at server level from the old page URL to the new one. There is only two things I can think that can be the problem -
Technical SEO | | seoman10
1. Lack of links directly to the page (there is literally only one or two)
2. Loading speed problem But I don't see any of these would cause a 5 position fluctuation regularly every day. But do you have any thoughts? Thanks in advance.0 -
Domain not ranking in Google
https://www.buitenspeelgoed.nl/ is a domain acquired by our client. Previously this website was on http://www.buitenspeelgoed-keupink.nl. With the old domain they were ranking top 30 on 'buitenspeelgoed' in google.nl. Now with the new exact match domain they aren't ranking any more (for months). However, the website is indexed, as you can see on http://1l1.be/nz I don't know what to do anymore. Need some advise. What we allready have done the last months: made adjustments to the 301-redirects (this was originaly setup wrong by the webdesigner (de) optimized the homepage on 'buitenspeelgoed' (strange is the fact that the Moz robot can't access the site). Checked the robots.txt to see if the website was blocked for Google Checked the meta robots to see if the website was blocked for Google Disavowed some spammy (old) links which linked to the old domain Checked Search console > Fetch as Google if there isn't any Malware of some kind (and to see if Google can access the site) Checked Search consol to see if there manual spam actions (isn't the case) Checked for duplicate content by copy/paste some texts in Google and see if any other results are showing up (isn't the case for most of the texts) Please let me know what we can do.
Technical SEO | | InventusOnline0 -
Ranking and Indexing Issue
We have an established site www.getinspired365.com that previously wasn't SEO optimised. We are currently in the process of testing out some new pages to see if we can get them to rank in Google, however we are seeing huge fluctuations in where they rank. Within the first few days we saw our page rank on the first or second page, however it has now dropped out of the top 250 search results. We are wondering if we have made any mistakes with our optimisation ? Example Page : Keyword to target - "If you laugh, you think, and you cry, that's a full day. That's a heck of a day. You do that seven days a week, you're going to have something special." URL : http://www.getinspired365.com/if-you-laugh-you-think-and-you-cry-thats-a-full-day-thats-a-heck-of-a-day-you-do-that-seven-days-a-week-youre-going-to-have-something-special We can see it has been indexed by Google but is now not ranking in the top 250 search engine results. We have run the On Page Grader from SEOMoz and it ranks the page as an "A" so we suspect that we are doing the SEO ok on the page, but can't work out why it isn't ranking, despite ranking on the first or second page after a few days ? We have other pages that aren't SEO optimised that rank better than our newly SEO optimised pages e.g. Keyword - "THE BEST LOVE IS THE KIND THAT AWAKENS THE SOUL AND MAKES US REACH FOR MORE, THAT PLANTS A FIRE IN OUR HEARTS AND BRINGS PEACE TO OUR MINDS. AND THAT'S WHAT YOU'VE GIVEN ME. THAT'S WHAT I'D HOPED TO GIVE YOU FOREVER" URL: http://www.getinspired365.com/20130528 Any advice you could offer would be great. Thanks ! Mike
Technical SEO | | MichaelWhyley0 -
Website not ranking but the blog is!
I am hoping someone might be able to help me, I am doing some work on a website. A new version of the site was recently launched and since then rankings have plummeted and the new blog pages are ranking better! When the new version of the site went live, the domain changed to the non-www version, plus an incorrect robots.txt file and we have never really been able to fully recover (both of these things were beyond my control!). The robots.txt file was corrected and some of the external links links changed to the non-www but there is a 301 redirect in place so changing to the non-www shouldn't have been the reason to drop the site out completely. Before the launch of the new website, the site was ranking on the front page of Google for a lot of relevant keywords such as outdoor blinds, outdoor blinds Perth, cafe blinds, patio blinds, etc. The quality of the links is pretty bad and I am attempting to remove them before doing a disavow of all the really bad quality links but unless we were really unlucky I don't think it's the links right now that are causing the problem. I have ran the site through numerous crawl tests, checked the robots.txt, there are no messages in GWMT, the pages are indexed but I have a feeling there is something wrong with the site that is stopping this site from ranking well. If anyone could give me any insights I would be really grateful. I know the site could be better structured from a keyword/ structure perspective but the site was ranking fine!
Technical SEO | | Karen_Dauncey0 -
Content: How Much and What Kind for top Keyword Rankings?
Aside from well optimized meta tags, alt tags, captions I could use some help figuring out: What's the recommended number of words (original of course) on page? What should be the keyword % ? We've recently engaged an SEO company to provide an audit, and hired another for comparison and guess what ... 2 different viewpoints. I consider this to be the worst $3 grand we've ever spent! I look forward to your opinions
Technical SEO | | HMCOE0 -
Why is this site ranking better than me
Hi just used the compare tool to try and find out why a site is ranking better than me http://www.opensiteexplorer.org/comparisons?site=www.lifestylemonthly.co.uk%2F my site is www.in2town.co.uk and the site i am comparing with is http://www.lifestylemonthly.co.uk/ Can anyone explain what is going on and how i can achieve better ranking results
Technical SEO | | ClaireH-1848860 -
Dismal content rankings
Hi, I realize this is a very broad question, but I am going to ask it anyways in the hopes that someone might have some insight. I have created a great deal of unique content for the site http://www.healthchoices.ca. You can select a video category from the top dropdown, then click on a video beside the provider box to see. The articles I've written are accessible by the View Article tab under each video. I have worked hard to make the articles informative and they are all unique with quotes from expert physicians. Even for strange health conditions that don't have a lot of competition - I don't see us appearing. Our search results are quite dismal for the amount of content we have. I guess I'm checking to see if anyone is able to point me in the right direction at all? If anything jumps out... Thanks, Erin
Technical SEO | | erinhealthchoices0 -
Why Does this Site Rank for so Many Keywords?
I was doing some research today and kept coming across this site sylvane.com I checked OSE and found that it only has a DA of 37 PA of 35, and only 79 linking domains (966 links total). I got to looking and there are a few links like this http://lulala.us/twxoops/html/userinfo.php?uid=303 Anyway, accroding to SEMRush, this site ranks for over 6,000 keywords. I ran content on a couple of their high ranking pages and the content is not unique, at least not very unique on the pages I checked. What is so special about that site that google is giving it so much love? I am not trying to call them out, I just want to know what this site is doing to garner the favor of google when I have a site in the same niche with a higher DA, PA, linking domains, and social signals.
Technical SEO | | CharlesMontgomery0