Drop in indexation but increase in organic traffic

[ViviCa1]

We've had a puzzling drop in indexed pages on our ecommerce website. My crawl returns just over 25k items. Until 19/6 we had about 23-24k indexed. Then we experienced a sudden drop from 19/6 to 26/6: from 23,400 to 18,999, losing 4.4k pages from one week to the next.

At the same time, our organic traffic has not decreased, it actually increased, however, it's only been a couple of weeks so that may be coincidence.

A few things that have happened during the past few weeks:

• 31/5: we implemented pagination on category pages to avoid issues with duplicate content - could it be that this led to a decrease in indexed pages 3 weeks later? However, I can only find about 1.5k pages in my crawl that are page 2+
• 18-19/6: we had some website outages over the weekend; as a B2B business, we don't get much traffic over the weekend, so I can't see an impact to traffic. However, the following week, indexation dropped by another 250 (then stayed the same this past week), so I don't think this was a factor.
• 21/6: we retired another website and migrated it to our main website. However, all pages were redirected to existing pages so no new pages were created for the migration. This doesn't really explain a decrease in indexation, but may account for some of the increase in organic traffic; however not all as the retired website hardly got any organic traffic.

So, should we be worried? As our website is quite large, it would probably be quite difficult to pin point exactly which pages dropped off the index, but a loss of 19% of pages is quite significant. Then again, it doesn't appear to have negatively impacted organic traffic...

Have you got any suggestions for what I should be looking at to find out what happened? Should I be worried at this point? I will definitely continue to have an eye on how our organic traffic (and indexation) develops but I am not sure if there is anything I can do at this point.

I'd appreciate your advice on this, to make sure I am not missing something blindingly obvious. Thanks!

RmWaNib JJm4tC3

[ViviCa1]

Hi Marcus,

Thanks for the comprehensive answer.

I don't have an exact number of products on our website due to how they are added; we have some products that are parent products consisting of several child products, so it is one product page but several product records.

Our sitemaps list just over 18k pages total - 17.3 product pages, 650 categories and the remainder are other content - however, it lists parent and child products (where child products redirect to parent products). So that number and the number of pages indexed by Google actually match very closely (Google have 18.7k indexed pages).

The crawl finds only 9.5k product pages as it doesn't list the child product pages, just parent product pages. It finds 2.1k category pages (including pages 2+, which are not included in the sitemap) and a similar number of other content compared to the sitemap.

I know that we have some problems with duplicate content as some products are very similar - we are working on addressing this as far as possible but it is a long-term project and, as you can imagine, it will take a while. Unfortunately our back end doesn't allow us to combine into one parent product where a product comes in a different size and in different colours, for example. In this case, we can combine into a parent product all different colours but we will have e.g. 10 different products for the different sizes. Changing the product description here doesn't seem user-friendly (even though it may make sense for SEO purposes) so for now, they will stay as they are. I also know we have some older products which need to be combined into parent and child products but aren't at the moment.

We are already using canonical tags - but these have been in place for a while. No directives have been updated that could affect how the Googlebot crawls our website. We have implemented a small (under 100) number of 301 redirects for category pages. But otherwise the main change recently has been the pagination for categories and the website migration.

I will have a closer look at how our organic landing pages perform and keep an eye on it, but you are right, there are too many possible factors and it may simply Google weeding out old / bad content, which wasn't getting any organic traffic anyway.

Thanks again for the detailed reply.

[Marcus_Miller]

Hey

Very hard to respond to a question like this in general terms. It really needs a careful analysis of the analytics data. Also, we have a lot of moving parts here. Whilst it is tempting to assign the result to one of these changes it simply may not be the case.

With larger ecommerce sites like this we often have lots of pages that are just not that great. It could be that Google is culling some of these older / thinner / weaker pages and you are seeing improved results from what remains.

It could be that you are seeing the benefit of the increased domain authority from folding in this new site and the indexation drop is a red herring.

It could be that some of that 25k pages were not great pages or were even URL based duplicate content. Have you done anything else with directives or canonicals etc?

I would always like to know a few things to double check the figures here:

You could cross reference like by product - how many products are there? If you can answer this you can get a better idea of how many pages there should be rather than just relying on a head count from a large site crawl. If you find out there are only 12k products and a 25k crawl then odds are there is something on the URL based duplication front that needs investigating.

You can also compare the landing pages to see which pages have experienced a drop. It could be the pages that have been culled did no organic search traffic anyway.

Ultimately - I doubt there is much to worry about but I would want to get a headcount on products, types of pages, quality of pages etc as it could be a sign that some of your pages need improving.

To answer beyond that would need a look at the site and the analytics.

Hope that helps!
Marcus