Google Dropped 3,000+ Pages due to 301 Moved !! Freaking Out !!
-
We may be the only people stupid enough to accidentally prevent the google bot from indexing our site.
In our htaccess file someone recently wrote the following statement
RewriteEngine On
RewriteCond %{HTTP_HOST} ^mysite.com$ [NC]
RewriteRule ^(.*)$ http://www.mysite.com/$1 [L,R=301]Its almost funny because it was a rewrite that rewrites back to itself... We found in webmaster tools that the site was not able to be indexed by the google bot due to not detecting the robots.txt file. We didn't have one before as we didn't really have much that needed to be excluded. However we have added one now for kicks really. The robots.txt file though was never the problem with regard to the bot accessing the site.
Rather it was the rewrite statement above that was blocking it. We tested the site not knowing what the deal was so we went under webmaster tools then health and then selected "Fetch as Google" to have the website. This was our way of manually requesting the site be re-indexed so we could see what was happening. After doing so we clicked on status and it provided the following:
HTTP/1.1 301 Moved Permanently
Content-Length: 250
Content-Type: text/html
Location: http://www.mystie.com/
Server: Microsoft-IIS/7.5
MicrosoftOfficeWebServer: 5.0_Pub
MS-Author-Via: MS-FP/4.0
X-Powered-By: ASP.NET
Date: Wed, 22 Aug 2012 02:27:49 GMT
Connection: close<title>301 Moved Permanently</title>
Moved Permanently
The document has moved here.
We changed the screwed up rewrite mistake in the htaccess file that found its way in there but now our issue is that all of our pages have been severely penalized with regard to where they are now ranking compared to just before the indecent.
We are essentially freaking out because we don't know the real time consequences of this and if or how long it will take for the certain pages to regain their prior ranks. Typical pages when down anywhere between 9-40 positions on high volume search terms. So to say the least our company is already discussing the possibilities of fairly large layoffs based on what we anticipate with regard to the drop in traffic. This sucks because this is peoples lives but then again a business must make money and if you sell less you have to cut the overhead and the easiest one is payroll.
I'm on a team with three other people that I work with to keep the SEO side up to snuff as much as we can and we sell high ticket items so the potential effects if Google doesn't restore matters could be significant. My question is what would you guys do? Is there any way we can contact Google about such a matter? If you can I've never seen such a thing. I'm sure the pages that are missing from the index now might make their way back in but what will there rank look like next time and with that type of rewrite has it permanently effected every page site wide, including those that are still in the index but severely effected by the index.
Would love to see things bounce back quick but I don't know what to expect and neither do my counterparts. Thanks for any speculation, suggestions or insights of any kind!!!
-
Interested to know if the indexing and rankings of the pages came back, David?
Paul
-
If every single page was set to 301 back to your home page and subsequently Google has removed all other URLs from the index, then disable the 301's, make sure the pages are working, and resend the sitemap to Google.
Do a scan with link sleuth to confirm all your pages are working correctly.
Greg
-
Thanks for the reply. Actually we were not trying to redirect the site anywhere. That was a redirect that literally redirected mysite.com to the identical mysite.com. Therefore we had no want or need to redirect anything other than internal url's. I think someone may have intended this to be used as a temporary overall site redirect during a maintenance schedule or something so that every page went to the same page but I assuming this was messed up because someone didn't put in the correct redirect code or url for that matter. Hope that makes sense.
So the issue was the google seen thousands of pages that were looking like a 301 redirect to the homepage and even the homepage redirected to the homepage as a permanently moved.
-
Ouch, sorry to hear that. I just added the htaccess code to one of my dev sites and it seems to redirect just fine (200 when visiting www, 301 when visiting non-www). I'm guessing that Google is going to the non-www page and is being redirected to the www version. Maybe switch the code to do the opposite (assuming your goal was to canonical/consolidate the urls). Message me if you want the htaccess code.
For fixing your site rankings, they will most likely bounce back on their own in due time so the best thing to do is try to get G to recrawl all the pages ASAP. I would:
-
Make sure you add and verify both the www and non-www urls of the site in GWT, then choose which one you want to be primary.
-
Change you sitemap to have have frequency/priority crawl rates, have the correct www/non-ww links and resubmit your sitemap in GWT.
-
Share links on twitter (they are crawled very often)
-
Create an rss feed with your pages and ping it
Good luck getting it ranking again.
-Oleg
-
-
First off a 301 redirect takes a few weeks (2-12) to fully work.
Second off you can send google a letter of reconsideration if you feel you were penalized. However I would give the 301 redirect a little boit of time to transfer your link juice. Also when you do a 301 i've heard that only 80% of your link juice will transfer. I like to contact my best links and have them link directly to the new site.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google ranking penalty: Limited to specific pages or complete website?
Hi all, Let's say few pages on the website dropped in the rankings due to poor optimisation of the pages or hit by algo updates. Does Google limits the ranking drop only to these pages or the entire website will have any impact? I mean will this cause ranking drop to the homepage for primary keyword? Will Google pose the penalty to other pages in the website if few pages drop in the rankings. Thanks
Algorithm Updates | | vtmoz0 -
Very strange, inconsistent and unpredictable Google ranking
I have been searching through these forums and haven't come across someone that faces the same issue I am. The folks on the Google forums are certain this is an algorithm issue, but I just can't see the logic in that because this appears to be an issue fairly unique to me. I'll take you through what I've gone through. Sorry for it being long. Website URL: https://fenixbazaar.com 1. In early February, I made the switch to https with some small hiccups. Overall however the move was smooth, had redirects all in place, sitemap, indexing was all fine. 2. One night, my organic traffic dropped by almost 100%. All of my top-ranking articles completely disappeared from rank. Top keyword searches were no longer yielding my best performing articles on the front page of results, nor on the last page of results. My pages were still being indexed, but keyword searches weren't delivering my pages in results. I went from 70-100 active users to 0. 3. The next morning, everything was fine. Traffic back up. Top keywords yielding results for my site on the front page. All was back to normal. Traffic shot up. Only problem was the same issue happened that night, and again for the next three nights. Up and down. 4. I had a developer and SEO guy look into my backend to make sure everything was okay. He said there were some redirection issues but nothing that would cause such a significant drop. No errors in Search Console. No warnings. 5. Eventually, the issue stopped and my traffic improved back to where it was. Then everything went great: the site was accepted into Google News, I installed AMP pages perfectly and my traffic boomed for almost 2 weeks. 6. At this point numerous issues with my host provider, price increases, and incredibly outdated cpanel forced me to change hosts. I did without any issues, although I lost a number of articles albeit low-traffic ones in the move. These now deliver 404s and are no longer indexed in the sitemap. 7. After the move there were a number of AMP errors, which I resolved and now I sit at 0 errors. Perfect...or so it seems. 8. Last week I applied for hsts preload and am awaiting submission. My site was in working order and appeared set to get submitted. I applied after I changed hosts. 9. The past 5 days or so has seen good traffic, fantastic traffic to my AMP pages, great Google News tracking, linking from high-authority sites. Good performance all round. 10. I wake up this morning to find 0 active people on my site. I do a Google search and notice my site isn't even the first result whenever I do an actual search for my name. The site doesn't even rank for its own name! My site is still indexed but search results do not yield results for my actual sites. Check Search Console and realised the sitemap had been "processed" yesterday with most pages indexed, which is weird because it was submitted and processed about a week earlier. I resubmitted the sitemap and it appears to have been processed and approved immediately. No changes to search results. 11. All top-ranking content that previously placed in carousal or "Top Stories" in Google News have gone. Top-ranking keywords no longer bring back results with my site: I went through the top 10 ranking keywords for my site, my pages don't appear anywhere in the results, going as far back as page 20 (last page). The pages are still indexed when I check, but simply don't appear in search results. It's happening all over again! Is this an issue any of you have heard of before? Where a site is still being indexed, but has been completely removed from search results, only to return within a few hours? Up and down? I suspect it may be a technical issue, first with the move to https, and now with changing hosts. The fact the sitemap says processed yesterday, suggests maybe it updated and removed the 404s (there were maybe 10), and now Google is attempting to reindexed? Could this be viable? The reason I am skeptical of it being an algorithm issue is because within a matter of hours my articles are ranking again for certain keywords. And this issue has only happened after a change to the site has been applied. Any feedback would be greatly appreciated 🙂
Algorithm Updates | | fenixbazaar0 -
Best and easiest Google Depersonalization method
Hello, Moz hasn't written anything about depersonalization for years. This article has methods, but I don't know if they are valid anymore. What's an easy, effective way to depersonalize Google search these days? I would just log out of Google, but that shows different ranking results than Moz's rank tracker for one of our main keywords, so I don't know if that method is correct. Thanks
Algorithm Updates | | BobGW0 -
Lots of dublicate titles and pages on search page
I own a paiting website with a lot of searchable paintings. The "search paintings" feature creates tons of dublicate pages and titles. See here:
Algorithm Updates | | KasperGJ
http://www.maleribasen.dk/soegmaleri.asp I guess the problem is, that the URL can actually be different and still return the same content. First time you click the "Search paintings" the URL will shown as above. But as soon as users
begin to definere they search to the left and use the "Search button" the top URL changes. So, depending on how the top URL looks different results are shown. This is pretty standard in searches. But it returns tons of dublicate pages and titles. How, do you guys cope with that? Is there a clever way to use ref="cannonical" or some other smart way to avoid this? /Kasper0 -
How long does google take to re-ranking pages in results?
I mean when google dance, the pages in results go up and down frequency every minue, but finally your page will rank in any position in google, what is the time when you get another position in google
Algorithm Updates | | engtamous0 -
Which Google Snippet to use? Products or Review
A "product" page that includes "user reviews/ratings" and "comparison prices", which Google snippet do/should you use? Thanks in advance
Algorithm Updates | | righty0 -
Ranking drops over weekend
Hi all, Has anyone noticed any ranking changes over the weekend? I've noticed an almost uniform drop of a few positions across almost all of my chosen phrases. Nothing major has occurred on our site, no messages in webmaster tools. An analysis of our competitors positions (which we track the top 10) doesn't reveal any clues. Cheers Aran
Algorithm Updates | | Aran_Smithson1 -
Why Google Cache is not showing ?
Hello Everyone, I have a question for you. Today when I checked cache:www.bollywoodshaadis.com on Google.com, it is showing following message. 404. <ins>That’s an error.</ins> The requested URL /search?hl=en&source=hp&biw=1366&bih=519&q=cache%3Awww.bollywoodshaadis.com&oq=cache%3Awww.bollywoodshaadis.com&aq=f&aqi=&aql=&gs_sm=e&gs_upl=2501l16050l0l16394l30l30l0l23l0l0l353l1781l0.1.4.2l7l0 was not found on this server. I was able to see sitelinks till yesterday but they also have disappeared now. Can someone please tell me why is this happening? Is this a temporary issue? or something needs to be done.
Algorithm Updates | | SEOcandy0