Welcome to the Q&A Forum

Lynn,

We had a few "site:" queries that we were watching as the full URLs came back replacing the truncated ones, for example: site:eventective.com/usa/Georgia/Atlanta.  When we discovered the original problem, almost every listing page in those SERPs had a truncated URL, but by the start of last week it had gradually cleared up to only 6 or 7 listings with truncated URLs while all others had the full URL.  Then suddenly we had 5 pages (50 listings) of truncated URLs and now almost 300 of them for that one query have the truncated version indexed.  It appears to be continuing.

Another detail I noticed was in Webmaster Tools.  All of our listings are in our sitemap with the full URL.  When we had this problem before only about 50% of our pages listed in our sitemap were indexed, assuming that is because the truncated ones were in the index instead of the full URLs that were in the sitemap.  As the truncated URL problem cleared up that ratio improved to the point where it was pretty steady at about 96-97% of our pages in our sitemap were indexed.  Once this problem started to reappear that number dropped down to 90% and kept going down to the point where it is at 77% now.

The only real change we made was an upgrade to our server hardware at our hosting company.

I've considered disallowing the truncated URL pattern in the robots.txt, but I really shouldn't have to do that with the 301.

I'm starting to wonder whether google is sending us a signal that they like the shorter version of the URL better.

Thanks for taking the time to take a look at it.

Michael

Lynn,

Thanks again for helping us out with this back in May.  After we made the corrections you pointed out it cleared up over the course of a few months.  There were just a few truncated urls left until suddenly this week we noticed it starting again.  I've looked at our 301s, our canonical/alternates, and made sure we are not linking to the truncated version anywhere, yet google continues to index the truncated version.  I'm tempted to disallow the truncated version in my robots.txt file, but hesitate to do that because of the possibility of some unexpected side effects.

Do you or anyone else reading this have any idea why google would index:

http://www.eventective.com/USA/Massachusetts/Bedford/107/

rather than:

http://www.eventective.com/USA/Massachusetts/Bedford/107/Doubletree-Hotel-Boston-Bedford-Glen.html

when all links point to the latter and the former is even 301'd to the latter.

Any and all help is appreciated.

Thank you,

Michael

Lynn,

You nailed it.  That's exactly what the problem was.  Since we were using the same URL pattern for m. and www., we had created the canonical by swapping the "m" out of the current url and replacing it with "www".  Since the truncated versions for mobile were in the index, they were all pointed to a truncated version for desktop.

As you pointed out, this should resolve itself over time.  Now I can focus on just the ranking issue.

Thank you both Lynn and Jesse for your help.

Michael

Just recently we noticed that google has indexed truncated urls for many of our pages that get 301'd to the correct page.

For example, we have:
http://www.eventective.com/USA/Massachusetts/Bedford/107/Doubletree-Hotel-Boston-Bedford-Glen.html

as the url linked everywhere and that's the only version of that page that we use.

Google somehow figured out that it would still go to the right place via 301 if they removed the html filename from the end, so they indexed just:

http://www.eventective.com/USA/Massachusetts/Bedford/107/

The 301 is not new. It used to 404, but (probably 5 years ago) we saw a few links come in with the html file missing on similar urls so we decided to 301 them instead thinking it would be helpful.  We've preferred the longer version because it has the name in it and users that pay attention to the url can feel more confident they are going to the right place.

We've always used the full (longer) url and google used to index them all that way, but just recently we noticed about 1/2 of our urls have been converted to the shorter version in the SERPs.  These shortened urls take the user to the right page via 301, so it isn't a case of the user landing in the wrong place, but over 100,000 301s may not be so good.

You can look at: site:www.eventective.com/usa/massachusetts/bedford/ and you'll noticed all of the urls to businesses at the top of the listings go to the truncated version, but toward the bottom they have the full url.

Can you explain to me why google would index a page that is 301'd to the right page and has been for years?

I have a lot of thoughts on why they would do this and even more ideas on how we could build our urls better, but I'd really like to hear from some people that aren't quite as close to it as I am.

One small detail that shouldn't affect this, but I'll mention it anyway, is that we have a mobile site with the same url pattern.

http://m.eventective.com/USA/Massachusetts/Bedford/107/Doubletree-Hotel-Boston-Bedford-Glen.html

We did not have the proper 301 in place on the m. site until the end of last week.  I'm pretty sure it will be asked, so I'll also mention we have the rel=alternate/canonical set up between the www and m sites.

I'm also interested in any thoughts on how this may affect rankings since we seem to have been hit by something toward the end of last week.  Don't hesitate to mention anything else you see that may have triggered whatever may have hit us.

Thank you,
Michael