Drop in indexation but increase in organic traffic
-
We've had a puzzling drop in indexed pages on our ecommerce website. My crawl returns just over 25k items. Until 19/6 we had about 23-24k indexed. Then we experienced a sudden drop from 19/6 to 26/6: from 23,400 to 18,999, losing 4.4k pages from one week to the next.
At the same time, our organic traffic has not decreased, it actually increased, however, it's only been a couple of weeks so that may be coincidence.
A few things that have happened during the past few weeks:
- 31/5: we implemented pagination on category pages to avoid issues with duplicate content - could it be that this led to a decrease in indexed pages 3 weeks later? However, I can only find about 1.5k pages in my crawl that are page 2+
- 18-19/6: we had some website outages over the weekend; as a B2B business, we don't get much traffic over the weekend, so I can't see an impact to traffic. However, the following week, indexation dropped by another 250 (then stayed the same this past week), so I don't think this was a factor.
- 21/6: we retired another website and migrated it to our main website. However, all pages were redirected to existing pages so no new pages were created for the migration. This doesn't really explain a decrease in indexation, but may account for some of the increase in organic traffic; however not all as the retired website hardly got any organic traffic.
So, should we be worried? As our website is quite large, it would probably be quite difficult to pin point exactly which pages dropped off the index, but a loss of 19% of pages is quite significant. Then again, it doesn't appear to have negatively impacted organic traffic...
Have you got any suggestions for what I should be looking at to find out what happened? Should I be worried at this point? I will definitely continue to have an eye on how our organic traffic (and indexation) develops but I am not sure if there is anything I can do at this point.
I'd appreciate your advice on this, to make sure I am not missing something blindingly obvious. Thanks!
-
Hi Marcus,
Thanks for the comprehensive answer.
I don't have an exact number of products on our website due to how they are added; we have some products that are parent products consisting of several child products, so it is one product page but several product records.
Our sitemaps list just over 18k pages total - 17.3 product pages, 650 categories and the remainder are other content - however, it lists parent and child products (where child products redirect to parent products). So that number and the number of pages indexed by Google actually match very closely (Google have 18.7k indexed pages).
The crawl finds only 9.5k product pages as it doesn't list the child product pages, just parent product pages. It finds 2.1k category pages (including pages 2+, which are not included in the sitemap) and a similar number of other content compared to the sitemap.
I know that we have some problems with duplicate content as some products are very similar - we are working on addressing this as far as possible but it is a long-term project and, as you can imagine, it will take a while. Unfortunately our back end doesn't allow us to combine into one parent product where a product comes in a different size and in different colours, for example. In this case, we can combine into a parent product all different colours but we will have e.g. 10 different products for the different sizes. Changing the product description here doesn't seem user-friendly (even though it may make sense for SEO purposes) so for now, they will stay as they are. I also know we have some older products which need to be combined into parent and child products but aren't at the moment.
We are already using canonical tags - but these have been in place for a while. No directives have been updated that could affect how the Googlebot crawls our website. We have implemented a small (under 100) number of 301 redirects for category pages. But otherwise the main change recently has been the pagination for categories and the website migration.
I will have a closer look at how our organic landing pages perform and keep an eye on it, but you are right, there are too many possible factors and it may simply Google weeding out old / bad content, which wasn't getting any organic traffic anyway.
Thanks again for the detailed reply.
-
Hey
Very hard to respond to a question like this in general terms. It really needs a careful analysis of the analytics data. Also, we have a lot of moving parts here. Whilst it is tempting to assign the result to one of these changes it simply may not be the case.
With larger ecommerce sites like this we often have lots of pages that are just not that great. It could be that Google is culling some of these older / thinner / weaker pages and you are seeing improved results from what remains.
It could be that you are seeing the benefit of the increased domain authority from folding in this new site and the indexation drop is a red herring.
It could be that some of that 25k pages were not great pages or were even URL based duplicate content. Have you done anything else with directives or canonicals etc?
I would always like to know a few things to double check the figures here:
You could cross reference like by product - how many products are there? If you can answer this you can get a better idea of how many pages there should be rather than just relying on a head count from a large site crawl. If you find out there are only 12k products and a 25k crawl then odds are there is something on the URL based duplication front that needs investigating.
You can also compare the landing pages to see which pages have experienced a drop. It could be the pages that have been culled did no organic search traffic anyway.
Ultimately - I doubt there is much to worry about but I would want to get a headcount on products, types of pages, quality of pages etc as it could be a sign that some of your pages need improving.
To answer beyond that would need a look at the site and the analytics.
Hope that helps!
Marcus
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How do you analyze a traffic drop with no historic Google Analytics data?
A client of mine has a large website with multiple sections (shop, forums, articles, etc.) that apparently had a significant reduction in rankings, traffic, and sales in the past. However, historic Google Analytics data is not available for the site, and I'm having troubles identifying anything concrete about the traffic drop, such as when it happened, what pages/sections it happened to, etc. The shop traffic drives most of the revenue, but it's a small number compared to the forums traffic, so it's hard to pick anything out of top-line trends like SEMrush offers. What tools or strategies might help in this situation?
Reporting & Analytics | | AdamThompson0 -
Main Website Redirects to Mobile Website, Mobile Website counts this as direct traffic, is there a way to tell what the source/medium is?
Hello, The situation is that someone is arriving on my main website https://www.example.com and being redirected to http://m.example.com. When this happens my analytics says that the traffic is all direct coming to my mobile site. However, I know people clicking on my google cpc, and some google organic users are hitting the main website and being redirected. Before we didn't have as good of a redirect on our main website so I could tell organic and cpc traffic coming in, now my main website has a huge drop in these categories because they are redirecting to mobile but I can't tell on my mobile how much traffic from each is going to the mobile site. Is there a way to fix this? Is it because my main website is https:// and mobile is a http:// (as I know that sometimes makes traffic direct) or is it a bigger problem that can't be resolved? Thanks
Reporting & Analytics | | oxfordseminars0 -
Self-Reffering Traffic After upgrading to Universal Analytics
Backstory: We have always had an issue with self-referring traffic but in waiting for the UA upgrade we put it on the backburner for getting fixed. We have now upgraded to UA and I was under the impression that GA would automatically exclude the domain associated with a property as a referral source. However this is not what I am seeing under my referral traffic source. With 10 websites having issues with this I need some help. Should I use the referral exclusion list? Also on a handful of our sites we have region specific URLs, I am also seeing these come in as self-referring traffic. I should also mention that about 85% of our sales are being attributed to the self-referring traffic. Here are two sites for example sake: ZootSports.com and K2snowboarding.com
Reporting & Analytics | | K2_Sports0 -
New e-commerce launched, drop in traffic
One of our clients launched a new e-commerce site and there was a significant (60%) drop in traffic in Google Analytics following the launch. We are about 10 days into the new site, and our suspicion is that the move has caused a decrease in their site authority which has lowered their search position. The old site had the store at sitename.com/store. The new site has a Shopify store at shop.sitename.com. There were very few external followed links to the store pages, so we thought that there would be little effect for overall site traffic by moving the store to a subdomain. Any recommendations for next steps on diagnosing the problem and working towards a solution?
Reporting & Analytics | | DesignHammer0 -
Organic Traffic Drop and Rank Increase After Video Thumbnail added
So my company has created a large amount of videos and I took a couple of them and created a test video sitemap to see what effect adding a video thumbnail/rich snippet to the SERPs would be. It worked on one page and gained 2 spots (position 6 to 4) for the highest keyword, but traffic didn't increase too much. Then a week later I tested it with the page that we get the most organic traffic for, which is ranking for a very big keyword. It worked and gained a bit of ranking, but traffic decreased 50% ever since according to Google Analytics. It seems to me that the traffic from users clicking on the video thumbnail is not be tracked as google / organic even though it lands them on the intended page/doesn't redirect anywhere else. I've looked to see if traffic to this page increased overall to see if it was being tracked via a referral or as something else, but couldn't find any traffic discrepancy. The only thing I did find is that impressions under SEO > Landing Pages > Video Property increased, but this could be from the page ranking in the Video SERPs now. Has anyone experienced a similar situation? Do you think having a video snippet could be that big of a turn off for customers that people just aren't clicking like they used to? I don't think so, that's why I'm leaning towards a tracking discrepancy in Google Analytics.
Reporting & Analytics | | OfficeFurn0 -
Google Making all searches secure - "Not provided" data to increase in Analytics
A lot of you might already be aware of the recent Google change at encrypting all search activity except for clicks on ads. Rand did a whiteboard session on this recently. How is everyone planning to adjust their research data to accommodate for this change?
Reporting & Analytics | | SEO5Team0 -
Senuke and traffic generator program is a good idea?? I think i got some problems now.
First of all thanks for reading, especially if you are the one whose bright ideas will help me out:) I started using senuke xcr about 3 months ago, obviously at the beginning i didnt make much success(not like I do now). Later i bought that inferno thingy and it actually works. First 2 weeks didnt make much difference(although i could see some little but stable uprising) but after 4th week ended, the average impression and queries doubled up, 6th went up again, its like every week or two it jumps up and keeps it there. Also the actual traffic from keywords went up! When about the second week finished, i started using a traffic generator program, first it leveled out the impressions and seemed to help a bit. Lately i think it messed it all up, plus about 2 weeks ago there was 2-3 dayswhen i sent a bit more traffic than usual and around that time the average rising of impressions didnt happened, it might even went down. Now i stopped using traffic g. and everything stayed the same no improvement!! Anyone could help me? I need to get it moving up again! Also im still nowhere near the top as the keywords are competitive well at least for me. What do i do wrong and what should i do? Also what about traffic generator? ps is it safe or/and or allowed to write that? Thanks
Reporting & Analytics | | Sugafree0 -
For an optimized site, any available stats / guesstimates on what is avg % of traffic to homepage vs. second-level pages?
I'm interested in passing this info on to a client who experienced a period of time when an incorrect GA code was installed on their homepage. They were able to get Google stats on second level pages only. This is a site that gets 80 + % of visits from organic search engine referrals. They do minimal advertising. Thanks in advance.
Reporting & Analytics | | alankoen1230