Google indexing wrong pages
-
We have a variety of issues at the moment, and need some advice.
First off, we have a HUGE indexing issue across our entire website.
Website in question: http://www.localsearch.com.au/
Firstly
In Google.com.au, if you search for 'plumbers gosford' (https://www.google.com.au/#q=plumbers+gosford), the wrong page appears - in this instance, the page ranking should be http://www.localsearch.com.au/Gosford,NSW/PlumbersI can see this across the board, across multiple locations.
Secondly
Recently I've seen Google reporting in 'Crawl Errors' in webmaster tools URLs such as:
http://www.localsearch.com.au/Saunders-Beach,QLD/Electronic-Equipment-Sales-Repairs&Sa=U&Ei=xs-XVJzAA9T_YQSMgIHQCw&Ved=0CIMBEBYwEg&Usg=AFQjCNHXPrZZg0JU3O4yTGjWbijon1Q8OAThis is an invalid URL, and more specifically, those query strings seem to be referrer queries from Google themselves: &Sa=U&Ei=xs-XVJzAA9T_YQSMgIHQCw&Ved=0CIMBEBYwEg&Usg=AFQjCNHXPrZZg0JU3O4yTGjWbijon1Q8OA
Here's the above example indexed in Google: https://www.google.com.au/#q="AFQjCNHXPrZZg0JU3O4yTGjWbijon1Q8OA"
Does anyone have any advice on those 2 errors?
-
Issue 1:
I think your intended ranking page is not indexed.
https://www.google.com/?gws_rd=ssl#q=site:http:%2F%2Fwww.localsearch.com.au
It's probably because, as Donna indicated, you have so many pages. This happens when you have what are essentially search pages that are indexed. Stuff happens like having a page for plumbing and plumbers in the same city, for example.
In the short term, you can make sure that non-indexed pages are linked to across the site. Long-term you're going to want to think of a way to organize your site to make sure Google and users can find the most important pages. For example, add breadcrumbs back to the city page, and have the city page linking to your most important types of pages (even if they're still searches) for the city. Right now your city pages are just more search pages, which is a big wasted opportunity to layout which pages you most want people to find. Also make sure you figure out what's going on between these two "types" of the exact same page. There should only be one for the same results where possible:
http://www.localsearch.com.au/Gosford,NSW
http://www.localsearch.com.au/Search?where=Gosford,NSW
Issue 2:
Look at the "linked from" and figure out where these bad pages are linked to on the site. Google wouldn't make up a URL if someone wasn't linking to them, and my guess is your site is causing them. With a highly-dynamic site like yours it's usually either a crawl trap or a combination of dynamic URLs through a particular path that the server wasn't expecting.
Alternatively, and maybe more likely, Google has been trying to parse Javascript lately, and doing a rather poor job of it. I've seen Google try to find links in Javascript that were never intended to be links. You can either ignore these errors and wait for Google to get better, or you can dig into the JS with a dev and see what's causing Google to interpret something as a link. There's usually another way to put the code together where Google understands.
-
Issue #1:
I think what you're doing is fine with canonicals. The problem (I think) might be all the duplicates. The page you're asking about (http://www.localsearch.com.au/Gosford,NSW/Plumbers) isn't indexed, yet ~5 million others are. Google is probably abandoning the site before all the relevant pages get indexed. You should look into removing duplicates like in the following examples:
-
http://www.localsearch.com.au/Australia
http://www.localsearch.com.au/Australia/ -
http://www.localsearch.com.au/Atherton,QLD
http://www.localsearch.com.au/Atherton,QLD/ -
http://www.localsearch.com.au/Albion-Park,NSW/Body-Ear-Piercing
http://www.localsearch.com.au/Albion-Park-Rail,NSW/Body-Ear-Piercing -
http://www.localsearch.com.au/Airlie-Beach,QLD/Breeze-Bar/profile/tSdO
http://www.localsearch.com.au/Airlie-Beach,QLD/Breeze-Bar/profile/tSdO.vcf
Issue #2:
Sounds like issue #1 and 2 are closely related. I think you're on the right path though. If it doesn't fix it, come back and ask again. You'll have eliminated some possibilities and can get a different perspective 2nd time round.
Good luck!
-
-
Issue #1
I'm not sure how else we would use them. The example given above (Gosford, NSW) is about 40KM (or around 20miles) from the page that is ranking (Wyong, NSW). In our business model, these are 2 separate markets. We wouldn't be able to canonical 1 to the other as they are completely separate.Issue #2
I believe the issue could be because we're displaying "search results" as static pages - this is something that I have my team working towards fixing by having "static" proximity based business listing pages (such as root.com/find/plumbers/state/city/suburb/) and having no-indexed search result pages (such as root.com/search?what=plumbers&where=suburb,state).The above may even fix issue #1, but I wanted to get some more information from a community as 2 minds are better than 1..
-
Issue #1
Neither of the results that Google has indexed when executing the site operator are duplicated pages - we also have canonical URLs setup on all pages to avoid duplicated URLs.You might not be using canonical tags to your advantage though. From what I can see, the canonical tags on pages just point to themselves as opposed to one master page that should be the catch-all for incoming links and social mentions.
With regards to the Title tags; unless there's a crowd of people agreeing with this, nearly everything I have found to try to prove this has fallen through - it seems having slightly similar title tags with brand name / locales included doesn't affect search results.
Some of the title tags you are using on pages are identical to one another, not "slightly similar". That's why I raised it.
Issue #2
_I don't believe this is the issue either as the actual pages still exist. _
Hm. I see. Those pages appear to be dynamically created, indexed, and canonicalized to themselves. Can you tag them as no-index?
-
Hi Donna, thanks for your reply.
Issue #1
Neither of the results that Google has indexed when executing the site operator are duplicated pages - we also have canonical URLs setup on all pages to avoid duplicated URLs.With regards to the Title tags; unless there's a crowd of people agreeing with this, nearly everything I have found to try to prove this has fallen through - it seems having slightly similar title tags with brand name / locales included doesn't affect search results.
Issue #2
I don't believe this is the issue either as the actual pages still exist.Thanks for your help though! Anything else you come up with, I'm open ears.
-
Issue #1:
You're right, you do seem to have a "variety of issues at the moment". The thing that stands out the most to me is duplicate content.
When I did a site search (site:http://www.localsearch.com.au/", Google indicates it has more than 5 million pages indexed on the site. When I did a site search for the specific URL in your example (site:http://www.localsearch.com.au/gosford,NSW/Plumbers), it found 2 results, neither of which the page in question. Yet your keywords were replicated in the page URLs, content, meta tags, and internal links. Google is probably having a heck of time figuring out which page to rank for what.
It also looks like you have your entire site replicated because URLs are indexed with and without a trailing "/".
Many of the title tags for Gosford pages are replicated containing "Gosford, NSW - LocalSearch" for example, www.localsearch.com.au/Gosford,NSW/Carriers-Light-Transport, www.localsearch.com.au/Gosford.../Radio-Communication-Equipment, www.localsearch.com.au/Gosford,NSW/Hair-Treatment-Replacement, www.localsearch.com.au/Gosford,NSW/Hobbies-Models-Accessories, www.localsearch.com.au/Gosford,NSW/Stone-Masons-Monumental, and so on. Can you see why Google might be confused.
That's probably the first thing you need to fix, duplicate content.
Issue #2:
This is a guess. These might be errors caused by pages that have been renamed or removed from the site and not properly redirected. Google can't find them. I'll be interested to hear if anyone else has any ideas.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
No Index thousands of thin content pages?
Hello all! I'm working on a site that features a service marketed to community leaders that allows the citizens of that community log 311 type issues such as potholes, broken streetlights, etc. The "marketing" front of the site is 10-12 pages of content to be optimized for the community leader searchers however, as you can imagine there are thousands and thousands of pages of one or two line complaints such as, "There is a pothole on Main St. and 3rd." These complaint pages are not about the service, and I'm thinking not helpful to my end goal of gaining awareness of the service through search for the community leaders. Community leaders are searching for "311 request service", not "potholes on main street". Should all of these "complaint" pages be NOINDEX'd? What if there are a number of quality links pointing to the complaint pages? Do I have to worry about losing Domain Authority if I do NOINDEX them? Thanks for any input. Ken
Intermediate & Advanced SEO | | KenSchaefer0 -
Google suddenly indexing 1,000 fewer pages. Why?
We have a site, blog.example.org, and another site, www.example.org. The most visited pages on www.example.org were redesigned; the redesign landed May 8. I would expect this change to have some effect on organic rank and conversions. But what I see is surprising; I can't believe it's related, but I mention this just in case. Between April 30 and May 7, Google stopped indexing roughly 1,000 pages on www.example.org, and roughly 3,000 pages on blog.example.org. In both cases the number of pages that fell out of the index represents appx. 15% of the overall number of pages. What would cause Google to suddenly stop indexing thousands of pages on two different subdomains? I'm just looking for ideas to dig into; no suggestion would be too basic. FWIW, the site is localized into dozens of languages.
Intermediate & Advanced SEO | | hoosteeno0 -
Google Indexing our site
We have 700 city pages on our site. We submitted to google via a https://www.samhillbands.com/sitemaps/locations.xml but they only indexed 15 so far. Yes the content is similar on all of the pages...thought on getting them to index the remaining pages?
Intermediate & Advanced SEO | | brianvest0 -
Number of indexed pages dropped. No manual action though?
I have a client who had their WordPress site hacked. At that point there was no message from Google in webmaster tools and the search results for their pages still looked normal. They paid sitelock to fix the site. This was all about a month ago. Logging into Webmaster Tools now there are still no messages from Google nor anything on the manual actions page. Their organic traffic is essentially gone. Looking at the submitted sitemap only 3 of their 121 submitted pages are indexed. Before this all of them where in the index. Looking at the index status report I can see that the number of indexed pages dropped completely off the map. We are sure that the site is free of malware. This client has done no fishy SEO practices. What can be done?
Intermediate & Advanced SEO | | connectiveWeb0 -
Why isn't google indexing our site?
Hi, We have majorly redesigned our site. Is is not a big site it is a SaaS site so has the typical structure, Landing, Features, Pricing, Sign Up, Contact Us etc... The main part of the site is after login so out of google's reach. Since the new release a month ago, google has indexed some pages, mainly the blog, which is brand new, it has reindexed a few of the original pages I am guessing this as if I click cached on a site: search it shows the new site. All new pages (of which there are 2) are totally missed. One is HTTP and one HTTPS, does HTTPS make a difference. I have submitted the site via webmaster tools and it says "URL and linked pages submitted to index" but a site: search doesn't bring all the pages? What is going on here please? What are we missing? We just want google to recognise the old site has gone and ALL the new site is here ready and waiting for it. Thanks Andrew
Intermediate & Advanced SEO | | Studio330 -
Huge google index with un-relevant pages
Hi, i run a site about sport matches, every match has a page and the pages are generated automatically from the DB. pages are not duplicated, but over time some look a little bit similar. after a match finishes it has no internal links or sitemap entry, but it's reachable by direct URL and continues to be on google index. so over time we have more than 100,000 indexed pages. since past matches have no significance and they're not linked and a match can repeat and it may look like duplicate content....what you suggest us to do: when a match is finished - not linked, but appears on the index and SERP 301 redirect the match Page to the match Category which is a higher hierarchy and is always relevant? use rel=canonical to the match Category do nothing.... *301 redirect will shrink my index status, some say a high index status is good... *is it safe to 301 redirect 100,000 pages at once - wouldn't it look strange to google? *would canonical remove the past matches pages from the index? what do you think? Thanks, Assaf.
Intermediate & Advanced SEO | | stassaf0 -
Does Google index url with hashtags?
We are setting up some Jquery tabs in a page that will produce the same url with hashtags. For example: index.php#aboutus, index.php#ourguarantee, etc. We don't want that content to be crawled as we'd like to prevent duplicate content. Does Google normally crawl such urls or does it just ignore them? Thanks in advance.
Intermediate & Advanced SEO | | seoppc20120 -
NOINDEX listing pages: Page 2, Page 3... etc?
Would it be beneficial to NOINDEX category listing pages except for the first page. For example on this site: http://flyawaysimulation.com/downloads/101/fsx-missions/ Has lots of pages such as Page 2, Page 3, Page 4... etc: http://www.google.com/search?q=site%3Aflyawaysimulation.com+fsx+missions Would there be any SEO benefit of NOINDEX on these pages? Of course, FOLLOW is default, so links would still be followed and juice applied. Your thoughts and suggestions are much appreciated.
Intermediate & Advanced SEO | | Peter2640