Google Dropped 3,000+ Pages due to 301 Moved !! Freaking Out !!
-
We may be the only people stupid enough to accidentally prevent the google bot from indexing our site.
In our htaccess file someone recently wrote the following statement
RewriteEngine On
RewriteCond %{HTTP_HOST} ^mysite.com$ [NC]
RewriteRule ^(.*)$ http://www.mysite.com/$1 [L,R=301]Its almost funny because it was a rewrite that rewrites back to itself... We found in webmaster tools that the site was not able to be indexed by the google bot due to not detecting the robots.txt file. We didn't have one before as we didn't really have much that needed to be excluded. However we have added one now for kicks really. The robots.txt file though was never the problem with regard to the bot accessing the site.
Rather it was the rewrite statement above that was blocking it. We tested the site not knowing what the deal was so we went under webmaster tools then health and then selected "Fetch as Google" to have the website. This was our way of manually requesting the site be re-indexed so we could see what was happening. After doing so we clicked on status and it provided the following:
HTTP/1.1 301 Moved Permanently
Content-Length: 250
Content-Type: text/html
Location: http://www.mystie.com/
Server: Microsoft-IIS/7.5
MicrosoftOfficeWebServer: 5.0_Pub
MS-Author-Via: MS-FP/4.0
X-Powered-By: ASP.NET
Date: Wed, 22 Aug 2012 02:27:49 GMT
Connection: close<title>301 Moved Permanently</title>
Moved Permanently
The document has moved here.
We changed the screwed up rewrite mistake in the htaccess file that found its way in there but now our issue is that all of our pages have been severely penalized with regard to where they are now ranking compared to just before the indecent.
We are essentially freaking out because we don't know the real time consequences of this and if or how long it will take for the certain pages to regain their prior ranks. Typical pages when down anywhere between 9-40 positions on high volume search terms. So to say the least our company is already discussing the possibilities of fairly large layoffs based on what we anticipate with regard to the drop in traffic. This sucks because this is peoples lives but then again a business must make money and if you sell less you have to cut the overhead and the easiest one is payroll.
I'm on a team with three other people that I work with to keep the SEO side up to snuff as much as we can and we sell high ticket items so the potential effects if Google doesn't restore matters could be significant. My question is what would you guys do? Is there any way we can contact Google about such a matter? If you can I've never seen such a thing. I'm sure the pages that are missing from the index now might make their way back in but what will there rank look like next time and with that type of rewrite has it permanently effected every page site wide, including those that are still in the index but severely effected by the index.
Would love to see things bounce back quick but I don't know what to expect and neither do my counterparts. Thanks for any speculation, suggestions or insights of any kind!!!
-
Interested to know if the indexing and rankings of the pages came back, David?
Paul
-
If every single page was set to 301 back to your home page and subsequently Google has removed all other URLs from the index, then disable the 301's, make sure the pages are working, and resend the sitemap to Google.
Do a scan with link sleuth to confirm all your pages are working correctly.
Greg
-
Thanks for the reply. Actually we were not trying to redirect the site anywhere. That was a redirect that literally redirected mysite.com to the identical mysite.com. Therefore we had no want or need to redirect anything other than internal url's. I think someone may have intended this to be used as a temporary overall site redirect during a maintenance schedule or something so that every page went to the same page but I assuming this was messed up because someone didn't put in the correct redirect code or url for that matter. Hope that makes sense.
So the issue was the google seen thousands of pages that were looking like a 301 redirect to the homepage and even the homepage redirected to the homepage as a permanently moved.
-
Ouch, sorry to hear that. I just added the htaccess code to one of my dev sites and it seems to redirect just fine (200 when visiting www, 301 when visiting non-www). I'm guessing that Google is going to the non-www page and is being redirected to the www version. Maybe switch the code to do the opposite (assuming your goal was to canonical/consolidate the urls). Message me if you want the htaccess code.
For fixing your site rankings, they will most likely bounce back on their own in due time so the best thing to do is try to get G to recrawl all the pages ASAP. I would:
-
Make sure you add and verify both the www and non-www urls of the site in GWT, then choose which one you want to be primary.
-
Change you sitemap to have have frequency/priority crawl rates, have the correct www/non-ww links and resubmit your sitemap in GWT.
-
Share links on twitter (they are crawled very often)
-
Create an rss feed with your pages and ping it
Good luck getting it ranking again.
-Oleg
-
-
First off a 301 redirect takes a few weeks (2-12) to fully work.
Second off you can send google a letter of reconsideration if you feel you were penalized. However I would give the 301 redirect a little boit of time to transfer your link juice. Also when you do a 301 i've heard that only 80% of your link juice will transfer. I like to contact my best links and have them link directly to the new site.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How to analyze a drop in featured snippets?
Our company site is, in general, performing great in organic search, but we've lost about 10 featured snippets throughout the last 5-6 months, and I'd like to know how to best approach analyzing this.
Algorithm Updates | | vibelingo0 -
Any Google Algorithm update?
Did Google release any seo update in last couple of days? My client is seeing 33% low traffic. The site is hyper-local marketplace website in India.
Algorithm Updates | | Avin1230 -
Drastic Drop in Link Juice
Hi Back in December we shifted my web domain from a gourmetdirect.com to gourmetdirect.co.nz as part of a site-wide revamp. Everything was going along fine until recently when my Linking domains plummeted and external links fell from 6000 approx to 600. We still have the .com live for loads of disfunctional reasons. Can anyone help? I have gone from a top ranker to a no show and my contractors are all shaking their heads.
Algorithm Updates | | GourmetDirect0 -
Do we take a SEO hit for having multiple URLs on an infinite scroll page vs a site with many pages/URLs. If we do take a hit, quantify the hit we would suffer.
We are redesigning a preschool website which has over 100 pages. We are looking at 2 options and want to make sure we meet the best user experience and SEO. Option 1 is to condense the site into perhaps 10 pages and window shade the content. For instance, on the curriculum page there would be an overview and each age group program would open via window shade. Option 2 is to have an overview and then each age program links to its own page. Do we lose out on SEO if there are not unique URLS? Or is there a way using metatags or other programming to have the same effect?
Algorithm Updates | | jgodwin0 -
Google Algorithm change this month - theories ?
Hi Moz fans! If you've not had the chance to check out moz cast go check it out it seems Google has been busy and so much so it broke Moz cast. There has been some discussion on seo round table about the changes and something seems to be going on. I wanted to find out how you guys are all finding it have you had a change in rankings? Any theories on what you think Google is up to? Personally I've seen rather a few of my sites go up in the rankings last week or so. As always look forward to hearing your thoughts and feelings on it Moz.
Algorithm Updates | | GPainter3 -
Google News Results
This is more of an observation than anything else. Has anyone noticed any strange results in Google News, in terms of very old content hitting page 1? My example is football, I support Newcastle so keep checking for the latest transfer failure or humiliation. First page for couple of days is showing old articles (April, May) from the same source rather than the usual spread of tabloid and broadsheet news.
Algorithm Updates | | MickEdwards0 -
SERP Drop
Hi, I have been trading online since 2006 and over the years I have built up some impressive SERP's for keywords such as "mens underwear' which I was SERP 1 for. However, over the past 6 months I have pretty much dropped off the face of Google for a large proportion of my keywords. I suspect I have been hit by the Panda/Penguin updates and do not know how to recover this. I have a mixture of what I consider to be relevant and healthy links, but there are also a few links in there that Google would no longer like. However, I believe that the majority of my links are OK. What should I do? Thanks i97zo6W.jpg
Algorithm Updates | | UnderMe0 -
Google.co.uk vs pages from the UK - anyone noticed any changes?
We've started to notice some changes in the rankings of Google UK and Google pages from the UK. Pages from the UK have always typically ranked higher, however it seems like these are slipping, and Google UK pages (pages from the web) are climbing. We've noticed a similar thing happening in the Bing/Yahoo algorithm as well. Just wondered if anyone else has anyone else noticed this? Thanks
Algorithm Updates | | Digirank0