Sudden Change In Indexed Pages
-
Every week I check the number of pages indexed by google using the "site:" function. I have set up a permanent redirect from all the non-www pages to www pages.
When I used to run the function for the:
non-www pages (i.e site:mysite.com), would have 12K results
www pages (i.e site:www.mysite.com) would have about 36K
The past few days, this has reversed! I get 12K for www pages, and 36K for non-www pages.
Things I have changed:
I have added canonical URL links in the header, all have www in the URL.
My questions:
Is this cause for concern?
Can anyone explain this to me?
-
Maybe Google includes all sub-domains. I just tested my site, and got the following results.
site:handsomeweb.com 340 results (all have www)
site:www.handsomeweb.com 231 results (all have www)
The difference is the first query includes pages located at:blog.handsomeweb.com
-
I don't get both resolving, and yes, I did set a preferred domain in my Google account.
Any other ideas?
-
Have you done preferred domain in Google?
I don't know how you could have done the 301's and still get both resolving. Have you run xenu or some other program to insure the 301s are there?
-
We did the 301 redirects from non-www to www when we launched the site.
I have another site that I have done a 301 from www to non-www, and you get 0 results when you search "site:www.mysite.com".
They are both on the same platform, which makes it more confusing!!!
-
inhouseseo
I have looked at several of our sites and see no change in results for site:
You stated: **Things I have changed: **
I have added canonical URL links in the header, all have www in the URL.
I believe what is happening (assuming you changed the canonical URL prior to the change in results of site:) is the change is the result of the canonical application you have added. However, I am not sure how you could still have an aggregate of 48K pages, are you sure this is accurate?
If, you are showing 12K of www, and 36K of non www, I would guess that the 12K were duplicated within the 36K. Therefore, you would have only 36K pages on your site.
Typically, when we encounter a site that has both www and non-www we select a preferred domain in WMT and do the 301 redirect in .htaccess file. Once this is done, over a short period, we will have only what we have chosen as the preferred domain www or non www.
So, if we started with 1,000 pages of www and 2,000 of non www, then if preferred choice is non www. We will end up with 2,000 pages total.
My suggestion would be to go into WMT and select a preferred domain and do the 301 redirect in the .htaccess file. Once that is done, I believe your problem will be resolved. rel=canon will not accomplish this in and of itself. Give it a few weeks and check your results.
Best
-
This is something I've been noticing greatly over the past few months. I was literally just about to post this question:
site: command, y u no accurate?!
I feel like the site:domain.com command used to be very accurate in showing you total pages indexed. Recently, I've seen wildly varied results returned.
Of course, it varies based upon the inclusion of "www.", but even without it, I've seen such results as anywhere from 193k to 8million pages... and everything in-between.
Why the variance? Has anyone else seen this recently?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Site move-Redirecting and Indexing dynamic pages
I have an interesting problem I would like to pick someone else’s brain. Our business has over 80 different products, each with a dedicated page (specs, gallery, copy etc.) on the main website. Main site itself, is used for presentation purpose only and doesn’t offer a direct path to purchase. A few years ago, to serve a specific customer segment, we have created a site where customers can perform a quick purchase via one of our major strategic partners. Now we are looking to migrate this old legacy service, site and all its pages under the new umbrella (main domain/CMS). Problem #1 Redirects/ relevancy/ SEO equity Ideally, we could simply perform 1:1 - 301 redirect from old legacy product pages to the relevant new site products pages. The problem is that Call to action (buy), some images and in some cases, parts of the copy must be changed to some degree to accommodate this segment. The second problem is in our dev and creative team. There are not enough resources to dedicate for the creation of the new pages so we can perform 1:1 301 redirects. So, the potential decision is to redirect a visitor to the dynamic page URL where parent product page will be used to apply personalization rules and a new page with dynamic content (buy button, different gallery etc.) is displayed to the user (see attached diagram). If we redirect directly to parent URL and then apply personalization rules, URL will stay the same and this is what we are trying to avoid (we must mention in the URL that user is on purchase path, otherwise this redirect and page where the user lands, can be seen as deceptive). Also Dynamic pages will have static URLs and unique page/title tag and meta description. Problem #2 : Indexation/Canonicalization The dynamic page is canonicalized to the parent page and does have nearly identical content/look and feel, but both serve a different purpose and we want both indexed in search. Hope my explanation is clear and someone can chip in. Any input is greatly appreciated! vCm2Dt.jpg
Intermediate & Advanced SEO | | bgvsiteadmin1 -
Difficulty with Indexing Pages - Desperate for Help!
I have a website with product pages that use the same URL, but load different data based on what's passed to them with GET. I am using a Wordpress website, but all of the page information is retrieved from a database using PHP and displayed with PHP. Somehow these pages are not being indexed by Google. I have done the following: 1. Created a site map pointing to each page. 2. Defined URL parameters in Search Console for these type of pages. 3. Created a product schema using schema.org, and tested it without errors. I have requested re-indexing repeatedly and these pages and images on the pages are still not being indexed! Does anybody have any suggestions?
Intermediate & Advanced SEO | | jacleaves0 -
Robots.txt, Disallow & Indexed-Pages..
Hi guys, hope you're well. I have a problem with my new website. I have 3 pages with the same content: http://example.examples.com/brand/brand1 (good page) http://example.examples.com/brand/brand1?show=false http://example.examples.com/brand/brand1?show=true The good page has rel=canonical & it is the only page should be appear in Search results but Google has indexed 3 pages... I don't know how should do now, but, i am thinking 2 posibilites: Remove filters (true, false) and leave only the good page and show 404 page for others pages. Update robots.txt with disallow for these parameters & remove those URL's manually Thank you so much!
Intermediate & Advanced SEO | | thekiller990 -
What may cause a page not to be indexed (be de-indexed)?
Hi All, I have a main category page, a landing page, that does not appear in the SERPS at all (even if I serach for a whole sentence from it). This page once ranked high. What may cause such a punishment for a specific page? Thanks
Intermediate & Advanced SEO | | BeytzNet0 -
Home page sudden drop in rank, but not others.
Over the weekend, I noticed "www.thematstore.com" drop out of the search results for "chair mats", where previously it was between 10-12 ranked. The other pages on the domain have remained the same in regards to rankings for their intended keywords. The only thing I changed (on Nov 29th) is the title tag from "Chair Mats - Commercial Quality Chair Mats, Custom Made", to "Chair Mats - Best Custom Chair Mats, Commercial Quality". I know there is a lot that needs to be done on the site, and the site architecture is a bit of a mess (I recommended building a new site)... but I was hoping to gradually fix these issues without gambling with a rank drop. Not sure why the home page all of the sudden dropped out, any suggestions?
Intermediate & Advanced SEO | | Joes_Ideas0 -
Indexing specified entry pages
Hi,We are currently working on location based info.Basically, when someone searches from Florida they will get specific Florida results and when they search from California they will specific California results.How does this location based info affect crawling and indexing?Lets say we have location info for googlebot, sometimes they crawl from a New York ip address, sometimes they do it from Texas and sometimes from California. In this case google will index 3 different pages with 3 different prices and a bit different text, and I'm afraid they might see these as some kind of cloaking or suspicious movement because we serve different versions of the page. What's the best way to handle this?
Intermediate & Advanced SEO | | SEODinosaur0 -
Why are new pages not being indexed, and old pages (now in robots.txt) remain in the index?
I currently have a site that was recently restructured, causing much of its content to be reposted, creating new URL's for each page. To avoid duplicates, all of the existing pages were added to the robots file. That said, it has now been over a week - I know Google has recrawled the site - and when I search for term X, it is stil the old page that is ranking, with the new one nowhere to be seen. I'm assuming it's a cached version, but why are so many of the old pages still appearing in the index? Furthermore, all "tags" pages (it's a Q&A site, like this one) were also added to the robots a few months ago, yet I think they are all still appearing in the index. Anyone got any ideas about why this is happening, and how I can get my new pages indexed?
Intermediate & Advanced SEO | | corp08030 -
Number of Indexed Pages are Continuously Going Down
I am working on online retail stores. Initially, Google have indexed 10K+ pages of my website. I have checked number of indexed page before one week and pages were 8K+. Today, number of indexed pages are 7680. I can't understand why should it happen and How can fix it? I want to index maximum pages of my website.
Intermediate & Advanced SEO | | CommercePundit0
