Drop in indexation but increase in organic traffic
-
We've had a puzzling drop in indexed pages on our ecommerce website. My crawl returns just over 25k items. Until 19/6 we had about 23-24k indexed. Then we experienced a sudden drop from 19/6 to 26/6: from 23,400 to 18,999, losing 4.4k pages from one week to the next.
At the same time, our organic traffic has not decreased, it actually increased, however, it's only been a couple of weeks so that may be coincidence.
A few things that have happened during the past few weeks:
- 31/5: we implemented pagination on category pages to avoid issues with duplicate content - could it be that this led to a decrease in indexed pages 3 weeks later? However, I can only find about 1.5k pages in my crawl that are page 2+
- 18-19/6: we had some website outages over the weekend; as a B2B business, we don't get much traffic over the weekend, so I can't see an impact to traffic. However, the following week, indexation dropped by another 250 (then stayed the same this past week), so I don't think this was a factor.
- 21/6: we retired another website and migrated it to our main website. However, all pages were redirected to existing pages so no new pages were created for the migration. This doesn't really explain a decrease in indexation, but may account for some of the increase in organic traffic; however not all as the retired website hardly got any organic traffic.
So, should we be worried? As our website is quite large, it would probably be quite difficult to pin point exactly which pages dropped off the index, but a loss of 19% of pages is quite significant. Then again, it doesn't appear to have negatively impacted organic traffic...
Have you got any suggestions for what I should be looking at to find out what happened? Should I be worried at this point? I will definitely continue to have an eye on how our organic traffic (and indexation) develops but I am not sure if there is anything I can do at this point.
I'd appreciate your advice on this, to make sure I am not missing something blindingly obvious. Thanks!
-
Hi Marcus,
Thanks for the comprehensive answer.
I don't have an exact number of products on our website due to how they are added; we have some products that are parent products consisting of several child products, so it is one product page but several product records.
Our sitemaps list just over 18k pages total - 17.3 product pages, 650 categories and the remainder are other content - however, it lists parent and child products (where child products redirect to parent products). So that number and the number of pages indexed by Google actually match very closely (Google have 18.7k indexed pages).
The crawl finds only 9.5k product pages as it doesn't list the child product pages, just parent product pages. It finds 2.1k category pages (including pages 2+, which are not included in the sitemap) and a similar number of other content compared to the sitemap.
I know that we have some problems with duplicate content as some products are very similar - we are working on addressing this as far as possible but it is a long-term project and, as you can imagine, it will take a while. Unfortunately our back end doesn't allow us to combine into one parent product where a product comes in a different size and in different colours, for example. In this case, we can combine into a parent product all different colours but we will have e.g. 10 different products for the different sizes. Changing the product description here doesn't seem user-friendly (even though it may make sense for SEO purposes) so for now, they will stay as they are. I also know we have some older products which need to be combined into parent and child products but aren't at the moment.
We are already using canonical tags - but these have been in place for a while. No directives have been updated that could affect how the Googlebot crawls our website. We have implemented a small (under 100) number of 301 redirects for category pages. But otherwise the main change recently has been the pagination for categories and the website migration.
I will have a closer look at how our organic landing pages perform and keep an eye on it, but you are right, there are too many possible factors and it may simply Google weeding out old / bad content, which wasn't getting any organic traffic anyway.
Thanks again for the detailed reply.
-
Hey
Very hard to respond to a question like this in general terms. It really needs a careful analysis of the analytics data. Also, we have a lot of moving parts here. Whilst it is tempting to assign the result to one of these changes it simply may not be the case.
With larger ecommerce sites like this we often have lots of pages that are just not that great. It could be that Google is culling some of these older / thinner / weaker pages and you are seeing improved results from what remains.
It could be that you are seeing the benefit of the increased domain authority from folding in this new site and the indexation drop is a red herring.
It could be that some of that 25k pages were not great pages or were even URL based duplicate content. Have you done anything else with directives or canonicals etc?
I would always like to know a few things to double check the figures here:
You could cross reference like by product - how many products are there? If you can answer this you can get a better idea of how many pages there should be rather than just relying on a head count from a large site crawl. If you find out there are only 12k products and a 25k crawl then odds are there is something on the URL based duplication front that needs investigating.
You can also compare the landing pages to see which pages have experienced a drop. It could be the pages that have been culled did no organic search traffic anyway.
Ultimately - I doubt there is much to worry about but I would want to get a headcount on products, types of pages, quality of pages etc as it could be a sign that some of your pages need improving.
To answer beyond that would need a look at the site and the analytics.
Hope that helps!
Marcus
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Alternative tools for Keyword Traffic
Hi There, Wondering if anyone has any other tools they would recommend using for finding out keyword traffic on websites. Currently (and I'm sure like most), my website is connected to Google Analytics and Google Search Console. My biggest frustration becomes the "(not set)" variable that appears when I go to review the keywords section. It's always such a large number and I have no way of finding out what people might be typing in and coming across my website. Of course, I understand the privacy factor as to why Google must do this but it's certainly difficult to analyze what's working and what's not. Any tips, tricks or suggestions are greatly appreciated! Thanks, Lindsay
Reporting & Analytics | | MainstreamMktg0 -
Why only a few pages of my website are being indexed by google
Our website www.navisyachts.com has in its sitemap over 3000 pages of information, and this is all unique content written by our team. Now Google Webmaster central shows only 100 urls indexed from 3500 submitted. Can you help me understand why and how I can fix this issue? The website has 4 years old, is a Joomla 3.3 up to date. It has part of the content in the Joomla core content systems and part in K2. Thank you. Pablo
Reporting & Analytics | | FWC_SEO0 -
How to filter out client traffic
Hi, I would like to filter out traffic in GA that comes to my client site just to login into their application. Is there a way to create a filter or segment that if a visitor exits the site to a subdomain it will filter them out? For example, they land on the homepage (abc.com/), click the login tab (drop down screen) and login and go to their application dashboard (app.abc.com/). I would like to delete this traffic segment as it is creates high bounce rates and makes it difficult to test landing pages. Thanks so much, Donna 🙂
Reporting & Analytics | | WebMarketingHUB0 -
Mobile redirect causing traffic drop? Can I move analytics code to capture them as visitors before redirecting them?
Yesterday I implemented a mobile redirect as I don't have a mobile version of my site. As a result I've seen a drop in traffic which makes sense as my redirect script is at the top of the page. If I were to move the redirect script below the analytics code would that allow me to still track mobile users? Would this affect my bounce rate though?
Reporting & Analytics | | SamCUK0 -
Dip in traffic from Pune for our sites in Google Analytics
Hi, We have noticed dip in traffic from Pune after 6th May'14 in our Google Analytics account for few of our sites. Did anyone noticed the same for your site. Kindly let me know if you have any idea. Thank and Regards
Reporting & Analytics | | vivekrathore0 -
Re-classifying a Traffic Source in Google Analytics
Hey All, I think it's inside of the Admin section of Google Analytics, right now I have a traffic source from the domain of indeed.ca that is being classified as Organic traffic when it should be classified as referring traffic, how can I tell Google Analytics that all traffic from this source should be classified as referral traffic and not organic traffic? Furthermore, after I make this change, will Google Analytics re-classify my past data so I can do a proper analysis? I can't remember how to do this and any help would be appreciated! Thanks!
Reporting & Analytics | | EvansHunt1 -
Sudden Drop in Direct Traffic
We recently had a some changes to our site, adding new sub categories, moving products around, etc... We implemented the changes on Thursday of last week and had a normal day around 800+ visitors. The very next day our # of visitors dropped in half to around 400+ and has stayed that way all weekend. I thought it might be that some links may have changed but I checked all of the links and they are the same as before with the exception of new pages. I checked to make sure that the google analytics code was still in the code and it is. I read that it could be DNS propagation timing issue but it's been 3 days and nothing's changed. I'm fairly certain that we are not being penalized or pandalized by google as our referrals and organic search results have not changed only the amount of direct traffic. Anyone have any ideas on what to look at or what to do?
Reporting & Analytics | | LizIslCo0