Long term plan for a large htaccess file with 301 redirects
-
We setup a pretty large htaccess file in February for a site that involved over 2,000 lines of 301 redirects from old product url's to new ones.
The 'old urls' still get a lot of traffic from product review sites and other pretty good sites which we can't change.
We are now trying to reduce the page load times and we're ticking all of the boxes apart from the size of the htaccess file which seems to be causing a considerable hang on load times. The file is currently 410kb big!
My question is, what should I do in terms of a long terms strategy and has anyone came across a similar problem?
At the moment I am inclined to now remove the 2,000 lines of individual redirects and put in a 'catch all' whereby anything from the old site will go to the new site homepage.
Example code:
RedirectMatch 301 /acatalog/Manbi_Womens_Ear_Muffs.html /manbi-ear-muffs.html
RedirectMatch 301 /acatalog/Manbi_Wrist_Guards.html /manbi-wrist-guards.htmlThere is no consistency between the old urls and the new ones apart from they all sit in the subfolder /acatalog/
-
When I faced a situation with several hundred pages, I decided to to only list the most important ones. I determined the important ones by there presence in Google and the import of the page content.
I first Googled "site:www.example.com" to get a good idea of what was indexed.
I used Analytics to see if any pages were entry pages. If a page gets no hits as an entry page, the 301 redirect is never needed.
I made a list of about 100 redirects, then made the 404 error page a slight variation of my homepage.
Now if you have any pages that have links in, you will need to maintain those redirects.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Is it a good idea to 301 redirect one same niche site towards another site for seo benefit
Hello friends, I have 2 android niche sites, one site is running on a technology dropped domain i catch 1 year ago it has, almost 400+ domains linking to different parts of the site, the other one i established from scratch and both are running from jan 2015. Now i want to redirect first site which already has 400 links pointing towards it to the home page of my 2nd android site. Is it a good idea to do so and does it give any boost in terms of seo?
Algorithm Updates | | RizwanAkbar0 -
Keyword Targeting - How to Properly Target Two Similar Terms?
Hi all, So I have a question about "best practices" when you have two unique, but highly similar keywords you are targeting. Let's use the examples of "raincoats for women," which gets 9,900 searches a month, and "rain jackets for women," which gets 4,400. I am in the process of selecting keywords for my client's "keyword portfolio" and need to come up with a strategy when faced with two similar keywords that use different terminology. I'm well aware that there should only be one page for "women's raincoats" but there is no doubt in my mind that Google will give preferential treatment to whichever version of the keyword (raincoats/rain jackets) I include in my title tag, meta description, content, etc. I know that the modern philosophy is that Google is sophisticated enough to understand that the two words are essentially synonymous. That said, would you A) only pick "raincoats for women" for your client's keyword portfolio and focus exclusively on that term in your optimizations? b) pick both terms and try to strike an even balance between both in your optimizations? c) pick both terms and only optimize for "raincoats for women" and hope that "rain jackets for women" gets some peripheral benefit from your optimizations via Google's understanding of synonyms? Thanks!
Algorithm Updates | | FPD_NYC0 -
Google is forcing a 301 by truncating our URLs
Just recently we noticed that google has indexed truncated urls for many of our pages that get 301'd to the correct page. For example, we have:
Algorithm Updates | | mmac
http://www.eventective.com/USA/Massachusetts/Bedford/107/Doubletree-Hotel-Boston-Bedford-Glen.html as the url linked everywhere and that's the only version of that page that we use. Google somehow figured out that it would still go to the right place via 301 if they removed the html filename from the end, so they indexed just: http://www.eventective.com/USA/Massachusetts/Bedford/107/ The 301 is not new. It used to 404, but (probably 5 years ago) we saw a few links come in with the html file missing on similar urls so we decided to 301 them instead thinking it would be helpful. We've preferred the longer version because it has the name in it and users that pay attention to the url can feel more confident they are going to the right place. We've always used the full (longer) url and google used to index them all that way, but just recently we noticed about 1/2 of our urls have been converted to the shorter version in the SERPs. These shortened urls take the user to the right page via 301, so it isn't a case of the user landing in the wrong place, but over 100,000 301s may not be so good. You can look at: site:www.eventective.com/usa/massachusetts/bedford/ and you'll noticed all of the urls to businesses at the top of the listings go to the truncated version, but toward the bottom they have the full url. Can you explain to me why google would index a page that is 301'd to the right page and has been for years? I have a lot of thoughts on why they would do this and even more ideas on how we could build our urls better, but I'd really like to hear from some people that aren't quite as close to it as I am. One small detail that shouldn't affect this, but I'll mention it anyway, is that we have a mobile site with the same url pattern. http://m.eventective.com/USA/Massachusetts/Bedford/107/Doubletree-Hotel-Boston-Bedford-Glen.html We did not have the proper 301 in place on the m. site until the end of last week. I'm pretty sure it will be asked, so I'll also mention we have the rel=alternate/canonical set up between the www and m sites. I'm also interested in any thoughts on how this may affect rankings since we seem to have been hit by something toward the end of last week. Don't hesitate to mention anything else you see that may have triggered whatever may have hit us. Thank you,
Michael0 -
How long does it take for a website to starting ranking once the website becomes live?
I am in the process of finishing the last touches on my new company's website and I am wondering about the process of getting my new website to rank on google. I've heard many different things from many different people who believe they know everything about SEO, but they can't all be right. Is there a definite timeline? Thanks
Algorithm Updates | | uofmiamiguy0 -
How long does it take for a new website to start showing in the SERP'S
I launched my website about 6 weeks ago. It was indexed fairly quickly. But it is not showing up in the Google SERP. I did do the on page SEO and followed the best practise's for my website. I have also been checking webmaster tools and it tells me that there is no errors with my site. I also ran it through the seomoz on page seo analyzer and again no real big issues. According to seomoz I had 1 duplicate content issue with my blog posts, which i corrected. I understand it takes some time, but any ideas of how much time? And f.y.i it's a Canadian website. So it should be a lot easier to rank as well. Could my site be caught in the Google 'sandbox effect' ? Any thoughts on this would be greatly appreciated.
Algorithm Updates | | CurtCarroll0 -
Google Dropped 3,000+ Pages due to 301 Moved !! Freaking Out !!
We may be the only people stupid enough to accidentally prevent the google bot from indexing our site. In our htaccess file someone recently wrote the following statement RewriteEngine On
Algorithm Updates | | David_C
RewriteCond %{HTTP_HOST} ^mysite.com$ [NC]
RewriteRule ^(.*)$ http://www.mysite.com/$1 [L,R=301] Its almost funny because it was a rewrite that rewrites back to itself... We found in webmaster tools that the site was not able to be indexed by the google bot due to not detecting the robots.txt file. We didn't have one before as we didn't really have much that needed to be excluded. However we have added one now for kicks really. The robots.txt file though was never the problem with regard to the bot accessing the site. Rather it was the rewrite statement above that was blocking it. We tested the site not knowing what the deal was so we went under webmaster tools then health and then selected "Fetch as Google" to have the website. This was our way of manually requesting the site be re-indexed so we could see what was happening. After doing so we clicked on status and it provided the following: HTTP/1.1 301 Moved Permanently
Content-Length: 250
Content-Type: text/html
Location: http://www.mystie.com/
Server: Microsoft-IIS/7.5
MicrosoftOfficeWebServer: 5.0_Pub
MS-Author-Via: MS-FP/4.0
X-Powered-By: ASP.NET
Date: Wed, 22 Aug 2012 02:27:49 GMT
Connection: close <title>301 Moved Permanently</title> Moved Permanently The document has moved here. We changed the screwed up rewrite mistake in the htaccess file that found its way in there but now our issue is that all of our pages have been severely penalized with regard to where they are now ranking compared to just before the indecent. We are essentially freaking out because we don't know the real time consequences of this and if or how long it will take for the certain pages to regain their prior ranks. Typical pages when down anywhere between 9-40 positions on high volume search terms. So to say the least our company is already discussing the possibilities of fairly large layoffs based on what we anticipate with regard to the drop in traffic. This sucks because this is peoples lives but then again a business must make money and if you sell less you have to cut the overhead and the easiest one is payroll. I'm on a team with three other people that I work with to keep the SEO side up to snuff as much as we can and we sell high ticket items so the potential effects if Google doesn't restore matters could be significant. My question is what would you guys do? Is there any way we can contact Google about such a matter? If you can I've never seen such a thing. I'm sure the pages that are missing from the index now might make their way back in but what will there rank look like next time and with that type of rewrite has it permanently effected every page site wide, including those that are still in the index but severely effected by the index. Would love to see things bounce back quick but I don't know what to expect and neither do my counterparts. Thanks for any speculation, suggestions or insights of any kind!!!0 -
Ranking well for main key terms but site traffic has dropped sharply?
Hello All, Just a quick question. Since the penguin update our site www.caravanguard.co.uk has seen some pretty fluctuating movement in Google, many of our key terms dropped over night, but over the last few weeks they have slowly started to move back up the rankings. The bizarre thing is despite the recover in rankings our unique traffic has taken a fairly large whack in numbers. Seasonality? Weather? ( it's been nice in the UK for a change) I can only assume the longer tail terms are taking more time to recover. I have tried to look into our back link profile and have noticed a little too much in terms concise keyword targeting, How do you go about changing these terms and removing the really bad links (struggling to identify the worst cases) on totally irrelevant sites or poor directories. Put in place before I started here 🐵 Any help truly appreciated. Regards Tim
Algorithm Updates | | TimHolmes0 -
Domain Deindexed because of Redirect
I think this is an interesting topic to discuss though I'm looking for answers too. One of my well performing domain deindexed by Google today. Reason: Redirect from a 9 year old Deindexed domain (Must be penalysed) I believe this is done by one of my competitor. What you people suggest me to do now? Don't you think if this is the way Google treat the redirects after Penguin anybody can use this technique to harm their competitors?
Algorithm Updates | | HeIsHere0