Why this page doesn't get indexed?
-
Hi, I've just taken over development and SEO for a site and we're having difficulty getting some key pages indexed on our site. They are two clicks away from the homepage, but still not getting indexed. They are recently created pages, with unique content on. The architecture looks like this:Homepage >> Car page >> Engine specific pageWhenever we add a new car, we link to its 'Car page' and it gets indexed very quickly. However the 'Engine pages' for that car don't get indexed, even after a couple of weeks. An example of one of these index pages are - http://www.carbuzz.co.uk/car-reviews/Volkswagen/Beetle-New/2.0-TSISo, things we've checked - 1. Yes, it's not blocked by robots.txt2. Yes, it's in the sitemap (http://www.carbuzz.co.uk/sitemap.xml)3. Yes, it's viewable to search spiders (e.g. the link is present in the html source)This page doesn't have a huge amount of unique content. We're a review aggregator, but it still does have some. Any suggestions as to why it isn't indexed?Thanks, David
-
Hi David,
Apologies, in my haste I didnt take in it was the engine page. I hole heartedly agree with the other comments made here. I would also add that, although not the be all and end all, you may find that linking out to to other sites may help the page get indexed as well. If there are any additional, authoritative resources (that are not competition) it may be worth linking to a few of these. Adds value to the user as well.
-
A factor could be another page was not indexed on the previous version of the site, which is presently indexed. On this site, the "other" page was indexed first, leading to this page not being indexed.
I noticed your site has other 2.0-TSI pages, and they were VW pages as well.
-
Interesting in the previous (completely version of the site) that page used to get crawled fine, but it had more content (but exactly the same as the parent one).
I wonder whether I should accept some duplication, just bite the bullet and write lots of original content, maybe find a way to explain google (using microformats) that it's a page that aggregates reviews.
I still need to figure out whether is it more worthwhile focusing on long tail keywords that the engine page would allow (E.g. Audi A6 1.4 Bluemotion TDI reviews) or just merge it together with the parent one for a much more content rich (Audi A6 Reviews).
-
Completely agree with Ryan, as an in-house SEO of a company with hundreds of thousands of pages I can guarantee you that not every page you have will be indexed via your sitemaps. The best thing to do is just use the tips that Ryan stated above. Link earning or content building, content building being the easier of the two.
-
Google has specifically stated they do not guarantee they will index all pages of a website. For Google to index a page, they have to decide the page offers value.
When I examine the page I note the following:
-
the page's comments are all snippets from other pages which have the full comments. There is no unique content for the comments.
-
the page's content is a total of 6 sentences, several of which are common to other pages on your site or are otherwise generic such as "Use the filter above to see reviews for the other engines."
-
the page has no backlinks to it, and the linking page has no links either. There are no apparent off page factors to indicate this page is important.
If you were to earn links to the page, my bet is it would be indexed. If you were to add a decent amount of quality content on the page, it would probably be indexed as well.
-
-
That's not the problem. The car in question got crawled correctly. It's the engine below it that didn't get crawled.
Anyway the car-chooser page is not the only way to get to cars.
-
Hi David,
When you first load the "Car Page" http://www.carbuzz.co.uk/car-chooser only the first few cars show. THe rest are only accessible when you scroll down the page which then activates some AJAX (i think) which loads more cars i the listing.
I suspect this has something to do with is. The bots cant access the "more" listings.
You could use an HTML sitemap on the site to help them get indexed but it would be better if the pages were linked to by default from the listings page.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Indexed pages
Just started a site audit and trying to determine the number of pages on a client site and whether there are more pages being indexed than actually exist. I've used four tools and got four very different answers... Google Search Console: 237 indexed pages Google search using site command: 468 results MOZ site crawl: 1013 unique URLs Screaming Frog: 183 page titles, 187 URIs (note this is a free licence, but should cut off at 500) Can anyone shed any light on why they differ so much? And where lies the truth?
Technical SEO | | muzzmoz1 -
Is there a way to index important pages manually or to make sure a certain page will get indexed in a short period of time??
Hi There! The problem I'm having is that certain pages are waiting already three months to be indexed. They even have several backlinks. Is it normal to have to wait more than three months before these pages get an indexation? Is there anything i can do to make sure these page will get an indexation soon? Greetings Bob
Technical SEO | | rijwielcashencarry0400 -
How long after disallowing Googlebot from crawling a domain until those pages drop out of their index?
We recently had Google crawl a version of the site we that we had thought we had disallowed already. We have corrected the issue of them crawling the site, but pages from that version are still appearing in the search results (the version we want them to not index and serve up is our .us domain which should have been blocked to them). My question is this: How long should I expect that domain (the .us we don't want to appear) to stay in their index after disallowing their bot? Is this a matter of days, weeks, or months?
Technical SEO | | TLM0 -
Has Google stopped rendering author snippets on SERP pages if the author's G+ page is not actively updated?
Working with a site that has multiple authors and author microformat enabled. The image is rendering for some authors on SERP page and not for others. Difference seems to be having an updated G+ page and not having a constantly updating G+ page. any thoughts?
Technical SEO | | irvingw0 -
The 'On Page' section of SEOMOZ
How does SEOMOZ choose a keyword for a page, for example it has ranked one of my pages for a search term which does not really appear on that page and then given it an F - how do I change the key word association? Secondly, when I first started using SEOMOZ I could change the page and then click the button 'Grade my on-page optimization' and it would show an immediate update - does anyone know why this has been stopped, as it is very useful to know you have got the page right away to an A for example.
Technical SEO | | bowravenseo0 -
I add microdata but why Google don't show it in SERP?
Site is: http://www.lightinthebox.com/, I've already added microdata for all product pages a month ago. And I used google Rich Snippets Testing Tool which shows me everything is all right. Like: http://www.lightinthebox.com/ouku-horizon-3g-android-smart-phone-with-3-5-inch-capacitive-touchscreen-800mhz-wifi-gps_p225435.html But Google just don't show the Rich Snippets in SERP. Any idea?? Thanks!
Technical SEO | | Litb0 -
Google News not indexing .index.html pages
Hi all, we've been asked by a blog to help them better indexing and ranking on Google News (with the site being already included in Google News with poor results) The blog had a chronicle URL duplication problem with each post existing with 3 different URLs: #1) www.domain.com/post.html (currently in noindex for editorial choices as showing all the comments) #2) www.domain.com/post/index.html (currently indexed showing only top comments) #3) www.domain.com/post/ (very same as #2) We've chosen URL #2 (/index.html) as canonical URL, and included a rel=canonical tag on URL #3 (/) linking to URL #2.
Technical SEO | | H-FARM
Also we've submitted yesterday a Google News sitemap including consistently the list of URLs #2 from the last 48h . The sitemap has been properly "digested" by Google and shows that all URLs have been sent and indexed. However if we use the site:domain.com command on Google News we see something completely different: Google News has indexed actually only some news and more specifically only the URLs #3 type (ending with the trailing slash instead of /index.html). Why ? What's wrong ? a) Does Google News bot have problems indexing URLs ending with .index.html ? While figuring out what's wrong we've found out that http://news.google.it/news/search?aq=f&pz=1&cf=all&ned=us&hl=en&q=inurl%3Aindex.html gives no results...it seems that Google News index overall does not include any URLs ending with /index.html b) Does Google News bot recognise rel=canonical tag ? c) Is it just a matter of time and then Google News will pick up the right URLs (/index.html) and/or shall we communicate Google News team any changes ? d) Any suggestions ? OR Shall we do the other way around. meaning make URL #3 the canonical one ? While Google News is showing these problems, Google Web search has actually well received the changes, so we don't know what to do. Thanks for your help, Matteo0 -
Site just will not be reincluded in Google's Index
I asked a question about this site (www.cookinggames.com.au) some time ago http://www.seomoz.org/qa/view/38488/site-indexing-google-doesnt-like-it and had some very helpful answers which were great. However I'm still no further ahead. I have added some more content, submitted a new XML sitemap, removed the 'lorem ipsum...' Now it seems that even Bing have ditched the site too. The number 1 result in Australia for the search term 'cooking games' is now this one - http://www.cookinggames.net.au/ which surely is not so much better to deserve a #1 spot whilst my site is deindexed? I have just had another reconsideration request 'denied' and am absolutely out of ideas/. If anyone can help suggest what I need to do... or even suggest how I can get feedback from the search engines what's wring that would be fantastic. Thank you David
Technical SEO | | OzDave0