Moz Q&A is closed.
After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.
Google is forcing a 301 by truncating our URLs
-
Just recently we noticed that google has indexed truncated urls for many of our pages that get 301'd to the correct page.
For example, we have:
http://www.eventective.com/USA/Massachusetts/Bedford/107/Doubletree-Hotel-Boston-Bedford-Glen.htmlas the url linked everywhere and that's the only version of that page that we use.
Google somehow figured out that it would still go to the right place via 301 if they removed the html filename from the end, so they indexed just:
http://www.eventective.com/USA/Massachusetts/Bedford/107/
The 301 is not new. It used to 404, but (probably 5 years ago) we saw a few links come in with the html file missing on similar urls so we decided to 301 them instead thinking it would be helpful. We've preferred the longer version because it has the name in it and users that pay attention to the url can feel more confident they are going to the right place.
We've always used the full (longer) url and google used to index them all that way, but just recently we noticed about 1/2 of our urls have been converted to the shorter version in the SERPs. These shortened urls take the user to the right page via 301, so it isn't a case of the user landing in the wrong place, but over 100,000 301s may not be so good.
You can look at: site:www.eventective.com/usa/massachusetts/bedford/ and you'll noticed all of the urls to businesses at the top of the listings go to the truncated version, but toward the bottom they have the full url.
Can you explain to me why google would index a page that is 301'd to the right page and has been for years?
I have a lot of thoughts on why they would do this and even more ideas on how we could build our urls better, but I'd really like to hear from some people that aren't quite as close to it as I am.
One small detail that shouldn't affect this, but I'll mention it anyway, is that we have a mobile site with the same url pattern.
http://m.eventective.com/USA/Massachusetts/Bedford/107/Doubletree-Hotel-Boston-Bedford-Glen.html
We did not have the proper 301 in place on the m. site until the end of last week. I'm pretty sure it will be asked, so I'll also mention we have the rel=alternate/canonical set up between the www and m sites.
I'm also interested in any thoughts on how this may affect rankings since we seem to have been hit by something toward the end of last week. Don't hesitate to mention anything else you see that may have triggered whatever may have hit us.
Thank you,
Michael -
Lynn,
We had a few "site:" queries that we were watching as the full URLs came back replacing the truncated ones, for example: site:eventective.com/usa/Georgia/Atlanta. When we discovered the original problem, almost every listing page in those SERPs had a truncated URL, but by the start of last week it had gradually cleared up to only 6 or 7 listings with truncated URLs while all others had the full URL. Then suddenly we had 5 pages (50 listings) of truncated URLs and now almost 300 of them for that one query have the truncated version indexed. It appears to be continuing.
Another detail I noticed was in Webmaster Tools. All of our listings are in our sitemap with the full URL. When we had this problem before only about 50% of our pages listed in our sitemap were indexed, assuming that is because the truncated ones were in the index instead of the full URLs that were in the sitemap. As the truncated URL problem cleared up that ratio improved to the point where it was pretty steady at about 96-97% of our pages in our sitemap were indexed. Once this problem started to reappear that number dropped down to 90% and kept going down to the point where it is at 77% now.
The only real change we made was an upgrade to our server hardware at our hosting company.
I've considered disallowing the truncated URL pattern in the robots.txt, but I really shouldn't have to do that with the 301.
I'm starting to wonder whether google is sending us a signal that they like the shorter version of the URL better.
Thanks for taking the time to take a look at it.
Michael
-
Hi Micheal,
When you say you started noticing it again, this is through webmaster tools or through your own monitoring? I ask because having a look at the site I can see no technical reason why those truncated urls would be getting indexed again at first glance. Maybe it is just a matter of waiting a bit more for the last of them to get removed? If all of a sudden they have started creeping up again, it suggests some variable in the mix has changed again, but I cannot see anything that stands out.
-
Lynn,
Thanks again for helping us out with this back in May. After we made the corrections you pointed out it cleared up over the course of a few months. There were just a few truncated urls left until suddenly this week we noticed it starting again. I've looked at our 301s, our canonical/alternates, and made sure we are not linking to the truncated version anywhere, yet google continues to index the truncated version. I'm tempted to disallow the truncated version in my robots.txt file, but hesitate to do that because of the possibility of some unexpected side effects.
Do you or anyone else reading this have any idea why google would index:
http://www.eventective.com/USA/Massachusetts/Bedford/107/
rather than:
http://www.eventective.com/USA/Massachusetts/Bedford/107/Doubletree-Hotel-Boston-Bedford-Glen.html
when all links point to the latter and the former is even 301'd to the latter.
Any and all help is appreciated.
Thank you,
Michael
-
Lynn,
You nailed it. That's exactly what the problem was. Since we were using the same URL pattern for m. and www., we had created the canonical by swapping the "m" out of the current url and replacing it with "www". Since the truncated versions for mobile were in the index, they were all pointed to a truncated version for desktop.
As you pointed out, this should resolve itself over time. Now I can focus on just the ranking issue.
Thank you both Lynn and Jesse for your help.
Michael
-
Hi Micheal,
I suspect the mobile site might be responsible for the indexed urls issue. Your mobile site has loads of indexed pages with the shorter urls: https://www.google.com/#output=search&sclient=psy-ab&q=site:m.eventective.com&oq=site:m.eventective.com&fp=9861fb8dc6b3e7c
Before the 301 redirects on the mobile site were created, were the rel canonical links pointing to the truncated urls on the main site? Seems to be the case on this random page I grabbed:
So a kind of odd mixture of 301s on the main site, and a well indexed mobile site saying the rel canonical on the main site is the shorter url. Seems maybe the rel canonical won! Are you sure this is a recent issue? Maybe it has been like this for a while and just not noticed much?
I would think that with the 301s and rel canonicals now properly implemented on the mobile site then the index will slowly sort itself out. I suppose you could put a rel canonical on the main site page also referencing itself, might speed up the process a bit more.
Agree with Jesse that it is not likely a major worry and wouldn't think this alone would cause a ranking issue.
-
I'm responding to this in a semi-rushed matter as something is coming up but I just want to mention that the most likely reason for Google to index this version of your URL is because of the links pointing to it. Those which caused you to put a 301 in place, those that were 404ing prior... They are clearly demonstrating to be the authoritative URL to Google.
I'm not sure why you're worried about what the customer/user sees for URL. They are most likely looking more at the Title/Description in the SERPs well before the URL string. Most people only read the domain portion of a URL string and it's more used for the search engines purposes.. (my opinion) Also, once the user clicks your title or page they are taken to the redirect and the full URL string will be visible in the address bar of their browser.
As for why your rankings are affected... I'd be surprised if it had anything to do with this, honestly. If anything redirecting should help especially if you had links pointing to a broken page. The only exception would be if those links were poison, of course.
Okay got to run hope I was helpful. Good luck!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Site appearing and disappearing from google serps.
Hi, My website is normally on page 2-3 on google consistently. Over the past month it has been appearing and then completely disappearing from the serps. One day it will be on page 2, then the next day completely missing from the serps. When i check the index it seems to be indexed correctly when doing site:mysite.com. I don't understand why this keeps happening, any experience with this issue? It doesn't seem to be a google dance as far as I can tell. When my other sites dance they typically just go up or down a few ranks for a couple weeks until they stabilize. Not completely fall off the search engine.
Algorithm Updates | | Chris_www0 -
Google Cache
So, when I gain a link I always check to see if the page that is linking is in the Google cache. I've noticed recently that more and more pages are actually not showing up in Google's cache, yet still appear in search results. I did read an article from someone whoo works at Google a few weeks back that there is sometimes an error with the cache and occasionally the cache will not display. This week, my own website isn't showing up in the cache yet I'm still ranking in SERP's. I'm not worried about it, mostly whitehat, but has there been any indication that Google are phasing out the ability to check cache's of websites?
Algorithm Updates | | ThorUK0 -
Is it possible that Google may have erroneous indexing dates?
I am consulting someone for a problem related to copied content. Both sites in question are WordPress (self hosted) sites. The "good" site publishes a post. The "bad" site copies the post (without even removing all internal links to the "good" site) a few days after. On both websites it is obvious the publishing date of the posts, and it is clear that the "bad" site publishes the posts days later. The content thief doesn't even bother to fake the publishing date. The owner of the "good" site wants to have all the proofs needed before acting against the content thief. So I suggested him to also check in Google the dates the various pages were indexed using Search Tools -> Custom Range in order to have the indexing date displayed next to the search results. For all of the copied pages the indexing dates also prove the "bad" site published the content days after the "good" site, but there are 2 exceptions for the very 2 first posts copied. First post:
Algorithm Updates | | SorinaDascalu
On the "good" website it was published on 30 January 2013
On the "bad" website it was published on 26 February 2013
In Google search both show up indexed on 30 January 2013! Second post:
On the "good" website it was published on 20 March 2013
On the "bad" website it was published on 10 May 2013
In Google search both show up indexed on 20 March 2013! Is it possible to be an error in the date shown in Google search results? I also asked for help on Google Webmaster forums but there the discussion shifted to "who copied the content" and "file a DMCA complain". So I want to be sure my question is better understood here.
It is not about who published the content first or how to take down the copied content, I am just asking if anybody else noticed this strange thing with Google indexing dates. How is it possible for Google search results to display an indexing date previous to the date the article copy was published and exactly the same date that the original article was published and indexed?0 -
Geo Target Location in your URL Structure
Hello everyone at SEOMOZ 😄 I have a question if you would be as kind as to inform me of which direction that I should take on this matter would be the more desirable approach for my seo strategy I have been using my location in my URL structure since I started doing SEO 5 years ago and I have always benefited from including my city in the URL. My question is, since the SEO landscape has change so drastically over the past 2 years and the Search Engines have become much more end user friendly and list suggestions for users as they type would it be more beneficial in 2013 to have the "Keyword" before or after the Geo Targeted Location in the URL structure? I own a computer repair business for the past 6 years now and I know that when i check to see where I am ranking for a particular keyword phrase such as "Computer Repair" GOOGLE detects my location and provides suggestions as I start typing out "Computer Repair" for the search query. One of the suggestions is "Computer Repair Wilmington NC" so I am starting to wonder if placing the Geo Targeted City after the Keyword would be the wiser choice instead of before it like a couple of years ago? Working Example: Here is a site that I am building out right now to re-brand my business. Currently I have one of the Silo Category Slugs set as seen below using the Location before the Keyword The First Example has the Geo Target Location before the Keyword and looks more natural to visitors on the site (at least to me) however I'm afraid that I may be shooting myself in the foot not placing the keyword before the Target Location? But if I do that, It does not read or flow fluently to the average looker so kinda confused and torn on how to deal with this>! FIRST EXAMPLE: Location Before Keyword Silo Parent Category = "Computer Repair" http://www.pcmedicsoncall.com/wilmington-nc-computer-repair/ Silo Child Category = "Laptop" http://www.pcmedicsoncall.com/wilmington-nc-computer-repair/laptop-repair/ Silo Grand Child Category = "LCD Replacement" http://www.pcmedicsoncall.com/wilmington-nc-computer-repair/laptop/lcd-screen-replacement/ **SECOND EXAMPLE: ** Keyword Before Location Silo Parent Category = "Computer Repair" http://www.pcmedicsoncall.com/computer-repair-wilmington-nc/ Silo Child Category = "Laptop" http://www.pcmedicsoncall.com/computer-repair-wilmington-nc/laptop-repair/ Silo Grand Child Category = "LCD Replacement" http://www.pcmedicsoncall.com/computer-repair-wilmington-nc/laptop-repair/lcd-screen-replacement/ Which would be the more favorable of the 2 examples that I have given please? Keyword before or After the Geo Targeted Location? thank you
Algorithm Updates | | MarshallThompson310 -
Proper Way To Submit A Reconsideration Request To Google
Hello, In previous posts, I was speaking about how we were penalized by Google for unnatural links. Basically 50,000 our of our 58,000 links were coming from 4-5 sites with the same exact anchor text and img alt tags. This obviously was causing our issues. Needless to say, I wen through the complete link profile to determine that all of the links besides this were of natrural origins. My question here is what is the accepted protocol of submitting a reinclusion request; For example, how long should it be? Should I disclose that I was in fact using paid links, and now that I removed (or at least nofollowed) them? I want to make sure that the request as good as it should so I can get our rankings up in a timely manner. Also, how long until the request is typically aknowledged? Thanks
Algorithm Updates | | BestOdds0 -
Google automatically adding company name to serp titles
Maybe I've been living under a rock, but I was surprised to see that Google had algorithmically modified my page titles in the search results by adding the company name to the end of the (short) title. <title>About Us</title> became About Us - Company Name Interestingly, this wasn't consistent - sometimes it was "company name Limited" and sometimes just "company name. Anyone else notice this or is this a recent change?
Algorithm Updates | | DougRoberts0 -
Lost 50% google traffic in one day - panic?
Hi girls + guys, a site of us were hit by a google update or a google penalty. We have lost 50% google traffic in one day (25th april, 2012). (Total visitors in average per day: 6k, yesterday: 3k) It's a german website, so I think google.de (germany) was updated. Our rankings in google.at (austria) are also affected, but it's not that bad as in google.de. We have not done any specific on page seo activities in the last two months. GWT doesn't have any message for us (no critical errors). After my first analyse I can say this: google has indexed 17k pages (thats fine) we are on 1st place with our domain name the last three days, the google traffic went up (+20%), but yesterday it was 50% below average (so -70%) last week we had a very good day, we had twice the traffic than normal, but this calmed down the following days we have lost number no. 1 places at two high traffic keywords. We had these no 1 rankings for years. We have been outranked by two of our competitors, but they have not done any onpage changes. We have lost a lot of positions at a lot of keywords. But there are also keywords which moved up. We have good content, useres are visiting 5 pages in average. No virus, no hacker (no hidden cloaking page) it's an old domain (2002) Lot of (good) inbound links Lot's of likes, g+. Good twitter activty. So, all in all I think it's more likely a ranking algo change than a penalty (a penalty for what reason?) My specific question(s): Is there any "check list" which could help me to find out the reason for this mess? What is the best strategy to regain the positions? New HTML code? New On page seo? (seomoz grades most of our important pages an A) Any idea would be appreciated! Best wishes,
Algorithm Updates | | GeorgFranz
Georg.1 -
Stop google indexing CDN pages
Just when I thought I'd seen it all, google hits me with another nasty surprise! I have a CDN to deliver images, js and css to visitors around the world. I have no links to static HTML pages on the site, as far as I can tell, but someone else may have - perhaps a scraper site? Google has decided the static pages they were able to access through the CDN have more value than my real pages, and they seem to be slowly replacing my pages in the index with the static pages. Anyone got an idea on how to stop that? Obviously, I have no access to the static area, because it is in the CDN, so there is no way I know of that I can have a robots file there. It could be that I have to trash the CDN and change it to only allow the image directory, and maybe set up a separate CDN subdomain for content that only contains the JS and CSS? Have you seen this problem and beat it? (Of course the next thing is Roger might look at google results and start crawling them too, LOL) P.S. The reason I am not asking this question in the google forums is that others have asked this question many times and nobody at google has bothered to answer, over the past 5 months, and nobody who did try, gave an answer that was remotely useful. So I'm not really hopeful of anyone here having a solution either, but I expect this is my best bet because you guys are always willing to try.
Algorithm Updates | | loopyal0