Page missing from Google index
-
Hi all,
One of our most important pages seems to be missing from the Google index.
A number of our collections pages (e.g., http://perfectlinens.com/collections/size-king) are thin, so we've included a canonical reference in all of them to the main collection page (http://perfectlinens.com/collections/all).
However, I don't see the main collection page in any Google search result. When I search using "info:http://perfectlinens.com/collections/all", the page displayed is our homepage. Why is this happening?
The main collection page has a rel=canonical reference to itself (auto-generated by Shopify so I can't control that).
Thanks!
-
In general, for link value to transfer either through 301s or canonicals, the content of the page needs to be nearly identical. See Cyrus' post for more. And canonicals are not always followed by Google, they are just a "hint", so it's unlikely you'll pass much value that way.
-
Dan, thanks for that response! I wasn't aware that our homepage had a canonical reference to our category page. On closer examination, I found that our category page in return had a canonical reference to our homepage. Messed up!
I've fixed that, and now resubmitted that page to Google using Search Console. Hopefully that will fix our issues.
Just one last question - why do you prefer noindex over canonical? If I had some backlinks to a thin category page (e.g., /collections/twin), wouldn't it be better to 'transfer' those benefits to our main category page (/collections/all) using canonical references?
Thanks again
-
Hello
Ahh ok, missed that detail.
I created a quick video for you ---> http://screencast.com/t/IKkEikyr
I think this is a bit of a complicated situation which will be tough to diagnose and fix in a Q&A thread. I would suggest catalog the different settings of your site in a spreadsheet like I show in the video.
Essentially, the canonical settings are just "suggestions" for Google and not "directives" so they will ignore them if they think they have been set in error.
I would start by clearly defining the end result you want (what pages should be crawled, and what should be indexed) and work backwards from there to apply the right settings.
I would probably try to use noindex, robots.txt etc before resorting to a canonical.
-
Hi Dan,
Thanks for your response. The page that you see when you type in our category page is in fact, our home page. e.g., when I do info:page A, or cache: page A, the result is for page B. Why is this happening if page A does not have a canonical reference or a redirect of any kind to B?
Thanks.
-
FYI - to check if a page is indexed try typing site:http://perfectlinens.com/collections/all into the Google search bar, or cache:http://perfectlinens.com/collections/all into your browser.
-
Hi There!
That page is in fact indexed and cached for me! Can you check again? And let me know?
-Dan
-
Patrick, thank you for your response.
1. The reason we're using canonical references on those pages is because they are almost identical copies of each other. In the future, we'll create some content on them and they can then stand by themselves.
2. But the original question remains - why is the main page (http://perfectlinens.com/collections/all) missing from the Google index? It's been on the site for a long time, it's one of our most important pages, it's in our sitemap, and robots.txt is not blocking it.
Thank you for your other tips though - I appreciate them, and will put them on our to-do list.
-
Hi there
First, those pages (size-king) should be canonicalized to their own pages, not canonicaling back to the "all" pages. This could be a potentially bad customer experience and you could be missing out on a LOT of organic traffic if some of those product pages are targeting high volume, low competition keywords / variations.
I would work on expanding the content on those product pages and implementing Schema. You have a lot of opportunities to be implementing these tags which will also help your search visibility.
Lastly, depending on when you implemented these canonical tags and your sitemap, Google and other search engines could still be indexing them. When did you upload your sitemap / implement canonical tags? Also, have you submitted these sitemaps to Google and Bing? I recommend you do so if you didn't!
And always make sure your robots.txt and meta tags aren't inadvertently blocking key pages from search! This is an often overlooked area in SEO!
But more than anything - work on that content for your product, canonical tag them to their pages, and add schema. It will make a world a difference!
Hope this helps! Good luck!
Patrick
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Tough SEO problem, Google not caching page correctly
My web site is http://www.mercimamanboutique.com/ Cached version of French version is, cache:www.mercimamanboutique.com/fr-fr/ showing incorrectly The German version: cache:www.mercimamanboutique.com/de-de/ is showing correctly. I have resubmitted site links, and asked Google re-index the web site many times. The German version always gets cached properly, but the French version never does. This is frustrating me, any idea why? Thanks.
Technical SEO | | ss20160 -
Google only indexed 19/94 images
I'm using Yoast SEO and have images (attachments) excluded from sitemaps, which is the recommended method (but could this be wrong?). Most of my images are in my posts; here's the sitemap for posts: https://edwardsturm.com/post-sitemap.xml I also appear on p1 for some good keywords, and my site is getting organic traffic, so I'm not sure why the images aren't being indexed. Here's an example of a well performing article: https://edwardsturm.com/best-games-youtube-2016/ Thanks!
Technical SEO | | Edward_Sturm0 -
Pages Not Getting Indexed
Hey there I have a website with pretty much 3-4 pages. All of them had a canonical pointing to one page and the same content ( which happened by mistake ) I removed the canonical URL and added one pointing to its page. Also, I added the original content that was supposed to be there to begin with. It's been weeks but those pages are not getting indexed on the SERPS while the one that they use to point with the canonical does.
Technical SEO | | AngelosS0 -
Site indexed by Google, but (almost) never gets impressions
Hi there, I have a question that I wasn't able to give it a reasonable answer yet, so I'm going to trust on all of you. Basically a site has all its pages indexed by Google (I verified with site:sitename.com) and it also has great and unique content. All on-page grades are A with absolutely no negative factors at all. However its pages do not get impressions almost at all. Of course I didn't expect it to be on page 1 since it has been launched on Dec, 1st, but it looks like Google is ignoring (or giving it bad scores) for some reason. Only things that can contribute to that could be: domain privacy on the domain, redirect from the www to the subdomain we use (we did this because it will be a multi-language site, so we'll assign to each country a subdomain), recency (it has been put online on Dec 1st and the domain is just a couple of months old). Or maybe because we blocked crawlers for a few days before the launch? Exactly a few days before Dec 1st. What do you think? What could be the reason for that? Thanks guys!
Technical SEO | | ruggero0 -
Why Google ranks a page with Meta Robots: NO INDEX, NO FOLLOW?
Hi guys, I was playing with the new OSE when I found out a weird thing: if you Google "performing arts school london" you will see w w w . mountview . org. uk at the 3rd position. The point is that page has "Meta Robots: NO INDEX, NO FOLLOW", why Google indexed it? Here you can see the robots.txt allows Google to index the URL but not the content, in article they also say the meta robots tag will properly avoid Google from indexing the URL either. Apparently, in my case that page is the only one has the tag "NO INDEX, NO FOLLOW", but it's the home page. so I said to myself: OK, perhaps they have just changed that tag therefore Google needs time to re-crawl that page and de-index following the no index tag. How long do you think it will take to don't see that page indexed? Do you think it will effect the whole website, as I suppose if you have that tag on your home page (the root domain) you will lose a lot of links' juice - it's totally unnatural a backlinks profile without links to a root domain? Cheers, Pierpaolo
Technical SEO | | madcow780 -
YouTube & Google + Pages
Hello, Has anyone had luck associating their Google + business page with their YouTube channel? Our YouTube page is associated with our Google + profile (and we would like it to be associated with the Google + business page.) There are numerous articles out there that Google is working on an update to allow the Channel/Google+ business page association but I am wondering if there is news we might have missed. Or if there is a way to get around it? We want to implement video on some site pages and would rather use YouTube code as opposed to customizing a solution. Do most folks think Google will have an easy solution once it at arrives? Meaning if you upload videos to your channel that is currently associated with the profile page, do you think there will be a way to convert everything over to a Google + business page once they unveil an update. Thank you!
Technical SEO | | SEOSponge0 -
De-indexing millions of pages - would this work?
Hi all, We run an e-commerce site with a catalogue of around 5 million products. Unfortunately, we have let Googlebot crawl and index tens of millions of search URLs, the majority of which are very thin of content or duplicates of other URLs. In short: we are in deep. Our bloated Google-index is hampering our real content to rank; Googlebot does not bother crawling our real content (product pages specifically) and hammers the life out of our servers. Since having Googlebot crawl and de-index tens of millions of old URLs would probably take years (?), my plan is this: 301 redirect all old SERP URLs to a new SERP URL. If new URL should not be indexed, add meta robots noindex tag on new URL. When it is evident that Google has indexed most "high quality" new URLs, robots.txt disallow crawling of old SERP URLs. Then directory style remove all old SERP URLs in GWT URL Removal Tool This would be an example of an old URL:
Technical SEO | | TalkInThePark
www.site.com/cgi-bin/weirdapplicationname.cgi?word=bmw&what=1.2&how=2 This would be an example of a new URL:
www.site.com/search?q=bmw&category=cars&color=blue I have to specific questions: Would Google both de-index the old URL and not index the new URL after 301 redirecting the old URL to the new URL (which is noindexed) as described in point 2 above? What risks are associated with removing tens of millions of URLs directory style in GWT URL Removal Tool? I have done this before but then I removed "only" some useless 50 000 "add to cart"-URLs.Google says themselves that you should not remove duplicate/thin content this way and that using this tool tools this way "may cause problems for your site". And yes, these tens of millions of SERP URLs is a result of a faceted navigation/search function let loose all to long.
And no, we cannot wait for Googlebot to crawl all these millions of URLs in order to discover the 301. By then we would be out of business. Best regards,
TalkInThePark0 -
Google picking up wrong page title
Hi, When searching for "Tottenham Forum" on google.co.uk (link below) http://www.google.co.uk/search?q=tottenham+forum&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:en-GB:official&client=firefox-a The site I manage (THFCTalk.com) is listed as 4th in the search results, but was hacked a few months ago and the search results lists the page title as "Free Shipping. Order Cialis Online. - Online Pharmacy" when the actual page title of THFCTalk is not actually set at that. Any idea how to fix this so Google updates this header on the search results? - as it is surely putting people off from clicking on our search result
Technical SEO | | WalesDragon0