Specific question about pagination prompted by Adam Audette's Presentation at RKG Summit
-
This question is prompted by something Adam Audette said in this excellent presentation:
http://www.rimmkaufman.com/blog/top-5-seo-conundrums/08062012/
First, I will lay out the issues:
1. All of our paginated pages have the same URL. To view this in action, go here: http://www.ccisolutions.com/StoreFront/category/audio-technica , scroll down to the bottom of the page and click "Next" - look at the URL. The URL is: http://www.ccisolutions.com/StoreFront/IAFDispatcher, and for every page after it, the same URL.
2. All of the paginated pages with non-unique URLs have canonical tags referencing the first page of the paginated series.
3. http://www.ccisolutions.com/StoreFront/IAFDispatcher has been instructed to be neither crawled nor indexed by Google.
Now, on to what Adam said in his presentation: At about minute 24 Adam begins talking about pagination. At about 27:48 in the video, he is discussing the first of three ways to properly deal with pagination issues. He says [I am somewhat paraphrasing]: "Pages 2-N should have self-referencing canonical tags - Pages 2-N should all have their own unique URLs, titles and meta descriptions...The key is, with this is you want deeper pages to get crawled and all the products on there to get crawled too. The problem that we see a lot is, say you have ten pages, each one using rel canonical pointing back to page 1, and when that happens, the products or items on those deep pages don't get get crawled...because the rel canonical tag is sort of like a 301 and basically says 'Okay, this page is actually that page.' All the items and products on this deeper page don't get the love."
Before I get to my question, I'll just throw out there that we are planning to fix the pagination issue by opting for the "View All" method, which Adam suggests as the second of three options in this video, so that fix is coming.
My question is this: It seems based on what Adam said (and our current abysmal state for pagination) that the products on our paginated pages aren't being crawled or indexed. However, our products are all indexed in Google. Is this because we are submitting a sitemap? Even so, are we missing out on internal linking (authority flow) and Google love because Googlebot is finding way more products in our sitemap that what it is seeing on the site? (or missing out in other ways?)
We experience a lot of volatility in our rankings where we rank extremely well for a set of products for a long time, and then disappear. Then something else will rank well for a while, and disappear. I am wondering if this issue is a major contributing factor.
Oh, and did I mention that our sort feature sorts the products and imposes that new order for all subsequent visitors? it works like this: If I go to that same Audio-Technica page, and sort the 125+ resulting products by price, they will sort by price...but not just for me, for anyone who subsequently visits that page...until someone else re-sorts it some other way. So if we merchandise the order to be XYZ, and a visitor comes and sorts it ZYX and then googlebot crawls, google would potentially see entirely different products on the first page of the series than the default order marketing intended to be presented there....sigh.
Additional thoughts, comments, sympathy cards and flowers most welcome. Thanks all!
-
Hi Dana,
The problem when it comes to passing authority internally is that properly paginated and crawled listing pages can be one of the primary routes via which Google finds and assigns authority to internal pages. Unless those products are linked to elsewhere, they're not going to be found if they cannot be found on a URL like http://www.ccisolutions.com/StoreFront/category/audio-technica?page=2, ?page=3 etc.
The lack of a unique URL with content changed dynamically also means that there never could be a good flow of authority through the site as Google does not have new pages to crawl and new outbound links to index / follow on those pages.
Your diagram is correct - the second option (Page 1 ---authority---> page 2 ----authority---> page 3... ) is what you're looking for with pagination.
-
Thanks so much Jane. I believe that URL is blocked from being crawled by our .htaccess file, although that's something I need to verify with IT. I just know from past discussions that it is blocked from crawling and indexing and it isn't in Google's index.
Would you mind describing, perhaps with a diagram, how this setup is a problem for passing authority internally? I am thinking it breaks the flow kind of like this:
Page 1 of Series -----> passing authority to page 2 --------> authority stops dead in its tracks due to non-unique URL
Instead of looking like this:
Page 1 of Series -----> passing authority to page 2 --------> page 2 passes authority back to home page, page 1 and page 3 of the Series....and so on
Would that be a somewhat accurate description? Thanks so much for responding. It is greatly appreciated!
-
Hi Dana,
Just to be clear, what I'm seeing is that if I visit a page like http://www.ccisolutions.com/StoreFront/category/led-stage-lights with 107 products, and I click "next", I do not receive the same URL with different products, but rather the URL changes to http://www.ccisolutions.com/StoreFront/IAFDispatcher with the next set of results. I think I'm just being blind, but how did you block that URL from crawling and indexing? I can't see a meta tag or a line in the site's robots.txt file.
This set-up definitely needs a modern pagination solution - glad Ryan's post at Ayima helped! This current set-up is definitely very detrimental to passing authority throughout the site to all products effectively.
-
Thank you Jane.
To answer your first question, no, we don't plan on continuing using the current method, however, I am concerned that whatever we decide to change is implemented properly.
To answer the second question, yes, all of our unique product pages return 200 OK status codes. I think the scenario your described is a very plausible scenario and it makes perfect sense to me. Especially you describe something else that I have seen happen, which I didn't even mention in my question. This is, the swapping out of ranking pages. I have seen this many times when one page might rank for a particular term for a while, then drop, and another page will take it's place and may do a lot better or a lot worse, and then in a few months time they will flip flop again.
One thing you didn't speak to was the fact that we have "no crawl, noindex" set up on this URL that is home to all of our paginated series pages. I am wondering what kind of havoc that could be wreaking on our internal linking and authority flow throughout the site? I am thinking it can't possibly be a good thing, no matter how you slice it.
Thank you very much for the link to the Pagination for SEO post. It contained a most excellent flow chart for pagination that I think every SEO should blow up, print out and post on a wall somewhere where developers and IT can see it. I am sharing a small version here:
-
Wow, I had never heard of this method of sorting before! Is this something you plan on keeping?
I would not be surprised if this is somewhat related to why Google sometimes finds and ranks some products, only to drop them for others. If the CMS serves it a different canonical version of the website every time it visits, it will index different products at different times.
The subsequent paginated pages are canonicalised back to the first page, but do you have unique product pages still returning 200 OK? I take it you do, due to the wording of your question but just to clarify: To use Amazon as an example, if a page like this is paginated, is a product page it links to still available even if it is listed on a deeper paginated page? If so, Google won't necessarily drop the product page if it doesn't see it linked to by the paginated listing pages, but it might do if it never sees it again due to pagination and canonicalisation. So you might end up with a situation where Googlebot found the Glastonbury coffee mug in January, keeps ranking it well for a few months, has not seen it again by April and drops it. But a white coffee mug set was found a few times in March, so it ranks that instead.
Have a look at this pagination post from one of my former colleagues as well - it has some comprehensive solutions to ecommerce pagination problems.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google Search Console 'Change of Address' Just 301s on source domain?
Hi all. New here, so please be gentle. 🙂 I've developed a new site, where my client also wanted to rebrand from .co.nz to .nz On the source (co.nz) domain, I've setup a load of 301 redirects to the relevant new page on the new domain (the URL structure is changing as well).
Technical SEO | | WebGuyNZ
E.G. On the old domain: https://www.mysite.co.nz/myonlinestore/t-shirt.html
In the HTACCESS on the old/source domain, I've setup 301's (using RewriteRule).
So that when **https://www.mysite.co.nz/**myonlinestore/t-shirt.html is accessed, it does a 301 to;
https://mysite.nz/shop/clothes/t-shirt All these 301's are working fine. I've checked in dev tools and a 301 is being returned. My question is, is having the 301's just on the source domain only enough, in regards to starting a 'Change of Address' in Google's Search Console? Their wording indicates it's enough but I'm concerned, maybe I also need redirects on the target domain as well? I.E. Does the Search Console Change of Address process work this way?
It looks at the source domain URL (that's already in Google's index), sees the 301 then updates the index (and hopefully pass the link juice) to the new URL. Also, I've setup both source and target Search Console properties as Domain Properties. Does that mean I no longer need to specify that the source and target properties are HTTP or HTTPS? I couldn't see that option when I created the properties. Thanks!0 -
URL Structure On Site - Currently it's domain/product-name NOT domain/category/product name is this bad?
I have a eCommerce site and the site structure is domain/product-name rather than domain/product-category/product-name Do you think this will have a negative impact SEO Wise? I have seen that some of my individual product pages do get better rankings than my categories.
Technical SEO | | the-gate-films0 -
Pro's & contra's: http vs https
Hi there, We are planning to take the step and go from http to https. The main reason to do this, is to mean trustfull to our clients. And of course the rumours that it would be better for ranking (in the future). We have a large e-commerce site. A part of this site ia already HTTPS. I've read a lot of info about pro's and contra's, also this MOZ article: http://moz.com/blog/seo-tips-https-ssl
Technical SEO | | Leonie-Kramer
But i want to know some experience from others who already done this. What did you encountered when changing to HTTPS, did you had ranking drops, or loss of links etc? I want to make a list form pro's and contra's and things we have to do in advance. Thanx, Leonie0 -
Pagination when not needed
Hello Moz, Odd one for you today. I've a site with has pagination (rel= next / prev) however its not being used correctly. I'll give you some examples: lets assume its a 5 page site with a home page, about us etc. The home page has a rel="next" tag on it leading to the next tab (about us) this goes all the way down to the final tag (contact us). Normally you use these tags for pages e.g page 1 - 5 but how much will they affect being used in the way above I'm thinking site structure. Just to add there is no view all on it either though this would make no sense in the way it is being used. Normally I would remove but the client wants to know why and I wanted to articulate better then "because its wrong" As always Moz - thanks!
Technical SEO | | GPainter0 -
Why are my URL's with a trailing slash still getting indexed even though they are redirected in the .htaccess file?
My .htaccess file is set up to redirect a URL with a trailing / to the URL without the /. However, my SEOmoz crawl diagnostics report is showing both URL's. I took a look at my Google Webmaster account and saw some duplicate META title issues. Same thing, Google Webmaster is showing the URL with the trailing /. My website was live for about 3 days before I added the code to the .htaccess file to remove the trailing /. Is it possible that in those 3 days that both versions were indexed and haven't been removed even though the .htaccess file has been updated?
Technical SEO | | mkhGT0 -
Are Collapsible DIV's SEO-Friendly?
When I have a long article about a single topic with sub-topics I can make it user friendlier when I limit the text and hide text just showing the next headlines, by using expandable-collapsible div's. My doubt is if Google is really able to read onclick textlinks (with javaScript) or if it could be "seen" as hidden text? I think I read in the SEOmoz Users Guide, that all javaScript "manipulated" contend will not be crawled. So from SEOmoz's Point of View I should better make use of old school named anchors and a side-navigation to jump to the sub-topics? (I had a similar question in my post before, but I did not use the perfect terms to describe what I really wanted. Also my text is not too long (<1000 Words) that I should use pagination with rel="next" and rel="prev" attributes.) THANKS for every answer 🙂
Technical SEO | | inlinear0 -
Does a CMS inhibit a site's crawlability?
I smell baloney but I could use a little backup from the community! My client was recently told by an SEO that search engines have a hard time getting to their site because using a CMS (like WordPress) doesn't allow "direct access to the html". Here is what they emailed my client: "Word Press (like your site is built with) and other similar “do it yourself” web builder programs and websites are not good for search engine optimization since they do not allow direct access to the HTML. Direct HTML access is needed to input important items to enhance your websites search engine visibility, performance and creditability in order to gain higher search engine rankings." Bots are blind to CMSs and html is html, correct? What do you think about the information given by the other SEO?
Technical SEO | | Adpearance0 -
Redirect question
I would like to redirect http://example.com/index.html to http://www.example.com/ Is the code below correct ? RewriteEngine on RewriteCond %{HTTP_HOST}^example.comRewriteRule (.*) http://www.example.com/$1 [R=301,L] RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /index.html\ HTTP/ RewriteRule ^index.html$ http://www.example.com/ [R=301,L]
Technical SEO | | seoug_20050