Why this page doesn't get indexed?
-
Hi, I've just taken over development and SEO for a site and we're having difficulty getting some key pages indexed on our site. They are two clicks away from the homepage, but still not getting indexed. They are recently created pages, with unique content on. The architecture looks like this:Homepage >> Car page >> Engine specific pageWhenever we add a new car, we link to its 'Car page' and it gets indexed very quickly. However the 'Engine pages' for that car don't get indexed, even after a couple of weeks. An example of one of these index pages are - http://www.carbuzz.co.uk/car-reviews/Volkswagen/Beetle-New/2.0-TSISo, things we've checked - 1. Yes, it's not blocked by robots.txt2. Yes, it's in the sitemap (http://www.carbuzz.co.uk/sitemap.xml)3. Yes, it's viewable to search spiders (e.g. the link is present in the html source)This page doesn't have a huge amount of unique content. We're a review aggregator, but it still does have some. Any suggestions as to why it isn't indexed?Thanks, David
-
Hi David,
Apologies, in my haste I didnt take in it was the engine page. I hole heartedly agree with the other comments made here. I would also add that, although not the be all and end all, you may find that linking out to to other sites may help the page get indexed as well. If there are any additional, authoritative resources (that are not competition) it may be worth linking to a few of these. Adds value to the user as well.
-
A factor could be another page was not indexed on the previous version of the site, which is presently indexed. On this site, the "other" page was indexed first, leading to this page not being indexed.
I noticed your site has other 2.0-TSI pages, and they were VW pages as well.
-
Interesting in the previous (completely version of the site) that page used to get crawled fine, but it had more content (but exactly the same as the parent one).
I wonder whether I should accept some duplication, just bite the bullet and write lots of original content, maybe find a way to explain google (using microformats) that it's a page that aggregates reviews.
I still need to figure out whether is it more worthwhile focusing on long tail keywords that the engine page would allow (E.g. Audi A6 1.4 Bluemotion TDI reviews) or just merge it together with the parent one for a much more content rich (Audi A6 Reviews).
-
Completely agree with Ryan, as an in-house SEO of a company with hundreds of thousands of pages I can guarantee you that not every page you have will be indexed via your sitemaps. The best thing to do is just use the tips that Ryan stated above. Link earning or content building, content building being the easier of the two.
-
Google has specifically stated they do not guarantee they will index all pages of a website. For Google to index a page, they have to decide the page offers value.
When I examine the page I note the following:
-
the page's comments are all snippets from other pages which have the full comments. There is no unique content for the comments.
-
the page's content is a total of 6 sentences, several of which are common to other pages on your site or are otherwise generic such as "Use the filter above to see reviews for the other engines."
-
the page has no backlinks to it, and the linking page has no links either. There are no apparent off page factors to indicate this page is important.
If you were to earn links to the page, my bet is it would be indexed. If you were to add a decent amount of quality content on the page, it would probably be indexed as well.
-
-
That's not the problem. The car in question got crawled correctly. It's the engine below it that didn't get crawled.
Anyway the car-chooser page is not the only way to get to cars.
-
Hi David,
When you first load the "Car Page" http://www.carbuzz.co.uk/car-chooser only the first few cars show. THe rest are only accessible when you scroll down the page which then activates some AJAX (i think) which loads more cars i the listing.
I suspect this has something to do with is. The bots cant access the "more" listings.
You could use an HTML sitemap on the site to help them get indexed but it would be better if the pages were linked to by default from the listings page.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Why does Google's search results display my home page instead of my target page?
Why does Google's search results display my home page instead of my target page?
Technical SEO | | h.hedayati6712365410 -
Local SEO: Spain - having trouble getting to first page
Dear, Moz community We are an online advertising website / direcotry. Lately we discovered that our website due to technical error was in English rather than Spanish and was not ranking in the local Spanish google.es at all. After changing the main language we quickly climbed to 2nd page with rankings in google.es . **My question is on top of regular SEO (Link building, content, blog) what could we do to help us rank quicker in local google.es. ** The same keyword ranks in google.com as #3 but not in Spain. Should we put in the effort to work with local directories? Google MyBusiness is not an option as we are not a bricks and mortar business.
Technical SEO | | advertisingcloud0 -
Sudden Drop in Indexed Pages and Images under Sitemap
Hello! Just a couple days back, realised that under the Google Webmaster Tool > Sitemap, my website www.bibliotek.co has a sudden drop in indexed pages and images. Previously, it was almost fully indexed. However, I checked and the Google Index > Index Status, it is still fully indexed Any reason why and how do I resolve? Any help is very much appreciated! Thanks in advance!
Technical SEO | | Bibliotek1230 -
WMT - Googlebot can't access your site
Hi On our new website which is just a few weeks old upon logging into Webmaster tools I am getting the following message Googlebot can't access your site - The overall error rate for DNS queries is 50% What do I need to do to resolve this, I have never had this problem before with any of the sites - where the domains are with Fasthosts (UK) and hosting is with Dreamhosts. What is the recommended course of action Google mention contacting your host in my case Dreamhost - but what do you need to ask them in a support ticket. When doing a fetch in WMT the fetch status is a success?
Technical SEO | | ocelot0 -
2 links on home page to each category page ..... is page rank being watered down?
I am working on a site that has a home page containing 2 links to each category page. One of the links is a text link and one link is an image link. I think I'm right in thinking that Google will only pay attention to the anchor text/alt text of the first link that it spiders with the anchor text/alt text of the second being ignored. This is not my question however. My question is about the page rank that is passed to each category page..... Because of the double links on the home page, my reckoning is that PR is being divided up twice as many times as necessary. Am I also right in thinking that if Google ignore the 2nd identical link on a page only one lot of this divided up PR will be passed to each category page rather than 2 lots ..... hence horribly watering down the 'link juice' that is being passed to each category page?? Please help me win this argument with a developer and improve the ranking potential of the category pages on the site 🙂
Technical SEO | | QubaSEO0 -
Indexed pages and current pages - Big difference?
Our website shows ~22k pages in the sitemap but ~56k are showing indexed on Google through the "site:" command. Firstly, how much attention should we paying to the discrepancy? If we should be worried what's the best way to find the cause of the difference? The domain canonical is set so can't really figure out if we've got a problem or not?
Technical SEO | | Nathan.Smith0 -
Panda or Penquin -Website Fell - Shouldn't this Recover?
On March 23rd our site fell 47% in one day. www.TranslationSoftware4u.com but we still held quite a few #1 to #7 rankings on Google and thought it would just recover. Our top keyword "translation software" was #4 , now we are #19 Over the next week I waited to see if it recovered. We have been online 10+ years and always stayed with white hat. I admit to learning as I go over the years but always felt content was king so I focused on information. I really do not see my site as using spam techniques but maybe I am missing something on the way I have it. March 23rd, major drop -47% On April 2nd I started with SEO MOZ and the Research tools showed we had duplicate content warning. This was from a blog we were trying to start that only had 7 posts but it had about 20 tags per post. I did not realize that tags actually created that post under that tag. I went in and deleted the tags again being stupid and not realizing it was then making that come up 404. The blog was so small we do not get hits on it anyway so hoping it just clears itself up. ( still get duplicate warning on our directory due to using "php Link Directory", but it's due to how it reuses the title tag and description, 2 instances per category page"). Still trying to fix the php directory issue. Seems many others are running it and did not have a drop. April 24th, we dropped another -10% It keeps falling -70% now. I have gone through the site and tried to clean up any warnings like duplicate title tags, meta descriptions. With regards to links I put up a small web directory with some reciprocal linking. Our product translates languages but software is not the same as a human so we often set clients up with human translators, the directory is a nice place to help our customers find a translator or see online tools that can help. The links were not excessive, there were maybe 100 links. After the fall I went in and found some translators had gone out of business so I deleted those, I am down to 65 links now, about 45 are exchanges. I have submitted to some online directories manually, but looking back through the links there is not really anything that makes me concerned. The link back to my site was really the most neglected SEO thing I did. Again concentrating on content. I did find a few links that I was not happy about but I did not put those links so had no control. I have been working on cleaning up my title tags, and making sure the content just reads better. I have been hoping that my site would just start recovering but it keeps sliding. Has anyone seen recovery from the updates. Should I see anything yet? I cannot seem to get Google to return to the site and reindex. Am I doing somethign spammy on my site and I do not realize it? Thanks for any advice in advance!
Technical SEO | | Force70 -
Why hasn't my sites indexed on opensiteexplorer.org changed in weeks?
Why hasn't my sites indexed on opensiteexplorer.org changed in weeks, even though I've done link-building like crazy?
Technical SEO | | AccountKiller0