Hi again Chris,

OK, well that makes it interesting!

First, if the list of 301's in your .htaccess file numbers in the hundreds, then there is definitely cause for concern about the effect it might be having on your load times.

The .htaccess file is read from top to bottom until a rule is matched. The first one that is matched will be used and no rule after that will be checked. Obviously, if there are hundreds of rules to check, this can cause processing bottlenecks.

Now there are some things you might be able to do to alleviate the problem IF you have the right conditions.

1. If your rebuild involved the relocation of an entire directory or directories which still contain the same pages as before, you can write a 301 Redirect for all pages in a directory with a single line of code

2. If there is a database behind your site and you have some means of matching pages from the old URL to the new (for example, is there a unique product ID for each page?) then you can use database lookups to write the redirects on the fly. This will eliminate the processing bottlenecks, but can only work if you have a reliable means of matching the pages.

3. If you have only one competition or offer at a time, then you could use database lookups to match any page with say, "competition" in the URL and 301 it to the current competition page (and the same with "offer").

4. For unknown links from external sites you can use a "catch-all" 301 to catch any other page that returns a 404 and send it to a single page (you put this rule at the very end of your .htaccess so it is the last to be matched). You could send these all to home, a generally relevant landing or category page, or a specially designed 404 page. The most important thing is that the page you redirect them to is as relevant as possible, or provides options that may keep the visitor on your site (search, menus etc). I prefer not to send these "random" 404's to home as they are likely to bring an  increase in bounce rate. Since bounce is now acknowledged as a ranking factor, I prefer to keep any traffic with a higher probability of bouncing away from pages with the highest Page Authority (PA).

Hope that helps,

Sha

Hi D4,

Rankings are affected by a host of factors, not the least of which are personalization and localization. Given that different people search for different things in the normal course of things, it is quite likely that two different people searching on different computers will get different results.

I would be very cautious about conducting repetitive searches every few minutes as you have described, because in the end, it could be you and your helpers that are influencing what you see in the rankings!

It is important to remember that Google and Bing both utilize click through data from the SERPs to inform rankings. Every time your site is returned in the results, but not clicked-through and every time a searcher clicks through, but quickly returns to the search engine, Google and Bing are gathering information that suggests your site is not seen as relevant or helpful by searchers. Constantly running redundant searches like this could eventually have a negative impact for your site.

I would say that you might be better served by conducting an in depth analysis of the pages concerned and putting the focus on improving them. If you get this aspect right, rankings and traffic will improve naturally over time.

Hope that helps,

Sha

Hi ahw,

A 500 Server Error is generally caused by one of two things:

1. A hardware error in the server itself
2. A software error - something in the code which causes the server to barf

From your description, it sounds like the 500 errors may be appearing intermittently - ie a page is Ok, then errors, then is Ok etc.... This would tend to suggest that the problem is a hardware fault, but it is still possible that it may be caused by random code which is not encountered every time a page is loaded.

The most reliable way to determine which problem you have is to check whether you are experiencing the 500 errors with a static file - ie an image, video, pdf, xml, css etc. If you are seeing the error with a static file, then the problem is 100% Server hardware fault (or Server misconfiguration, but this should return a permanent 500 Error).

If "the sitemap also goes down on a regular basis", means that your sitemap also is intermittently returning a 500 Error and it is a static xml file, that may be your clue.

If yours is a LAMP based system (Apache/Linux) and you have root access to your Server, my boss is a Systems Administrator and able to take a look at it for you, but his gut feeling is that you are looking at a hardware problem. This means your best course of action would be to contact your Hosting provider as soon as possible and ask them to take a look at it.

Hope that helps,

Sha

Hi Serge,

The reason your redirect is creating a 302 is that it is an unspecified Redirect.

Any redirect that is unspecified will by default be seen as a 302 (Temporary)

To create a 301, you need to specify like this

[R=301,L]

Hope that helps,

Sha

Hi JU1985,

Good call from Donnie!

I would be keeping both pages live and adding a unique explanation to each page that lets them know that the product they searched for has been superseded by Widget C.

When deciding on the right solution for any issue like this, the first thing to consider is the effect your solution will have on the user. Ask yourself "If I search for Widget A and land on a page that offers Widget C, what will I think?".

The answer for me is that I will most likely assume the result is incorrect and return to the search engine looking for a better result. That is not the best user experience possible and therefore unlikely to provide the best conversion rate possible.

So for me, any solution that simply delivers the client to a different product without an explanation (301 or rel=canonical)  is least preferred.

The key to good business is good customer service - essentially being as helpful to your potential customer as possible. If a customer arrived at your offline store and said "I'm looking for Widget A", would you push them quickly across the store and say "here's Widget C"? Or would you explain that "Widget A has now been superseded by Widget C" and provide Widget C for them to look at?

The more you can emulate the offline store experience in your online store, the better the chance that the customer will feel comfortable buying from you.

Incidentally, I would make sure that the Widget C description added to the pages includes a Buy button and sufficient information that the customer can proceed to purchase without having to go to the Widget C page.

Hope that helps,

Sha