Site: Query Question
-
Hi All,
Question around the site: query you can execute on Google for example. Now I know it has lots of inaccuracies, but I like to keep a high level sight of it over time.
I was using it to also try and get a high level view of how many product pages were indexed vs. the total number of pages.
What is interesting is when I do a site: query for say www.newark.com I get ~748,000 results returned.
When I do a query for www.newark.com "/dp/" I get ~845,000 results returned.
Either I am doing something stupid or these numbers are completely backwards?
Any thoughts?
Thanks,
Ben
-
Barry Schwartz posted some great information about this in November of 2010, quoting a couple of different Google sources. In short, more specific queries can cause Google to dig deeper and give more accurate estimates.
-
Yup. get rid of parameter laden urls and its easy enough. If they hang around the index for a few months before disappearing thats no big deal, as long as you have done the right thing it will work out fine
Also your not interested in the chaff, just the bits you want to make sure are indexed. So make sure thise are in sensibly titled sitemaps and its fine (used this on sites with 50 million and 100 million product pages. It gets a bit more complex at that number, but the underlying principle is the same)
-
But then on a big site (talking 4m+ products) its usually the case that you have URL's indexed that wouldn't be generated in a sitemap because they include additional parameters.
Ideally of course you rid the index of parameter filled URL's but its pretty tough to do that.
-
Best bet is to make sure all your urls are in your sitemap and then you get an exact count.
Ive found it handy to use multiple sitempas for each subfolder i.e. /news/ or /profiles/ to be able to quickly see exactly what % of urls are indexed from each section of my site. This is super helpful in finding errors in a specific section or when you are working on indexing of a certain type of page
S
-
What I've found the reason for this comes down to how the Google system works. Case in point, a client site I have with 25,000 actual pages. They have mass duplicate content issues. When I do a generic site: with the domain, Google shows 50-60,000 pages. If I do an inurl: with a specific URL param, I either get 500,000 or over a million.
Though that's not your exact situation, it can help explain what's happening.
Essentially, if you do a normal site: Google will try its best to provide the content within the site that it shows the world based on "most relevant" content. When you do a refined check, it's naturally going to look for the content that really is most relevant - closest match to that actual parameter.
So if you're seeing more results with the refined process, it means that on any given day, at any given time, when someone does a general search, the Google system will filter out a lot of content that isn't seen as highly valuable for that particular search. So all those extra pages that come up in your refined check - many of them are most likely then evaluated as less than highly valuable / high quality or relevant to most searches.
Even if many are great pages, their system has multiple algorithms that have to be run to assign value. What you are seeing is those processes struggling to sort it all out.
-
about 839,000 results.
-
Different data center perhaps - what about if you add in the "dp" query to the string?
-
I actually see 'about 897,000 results' for the search 'site:www.newark.com'.
-
Thanks Adrian,
I understand those areas of inaccuracy, but I didn't expect to see a refined search produce more results than the original search. That just seems a little bizarre to me, which is why I was wondering if there was a clear explanation or if I was executing my query incorrectly.
Ben
-
This is an expected 'oddity' of the site: operator. Here is a video of Matt Cutts explaining the imprecise nature of the site: operator.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Query on Not Set In Product List Performance in Google Analytics
Hi All, I have query for given below screenshot-1. What is Not Set here? For column no. 2 only purchase and revenue showing other column blank why? I have properly implemented enhance ecommerce via tag manager. And my product list impression, clicks all working fine for all categories now I don't know from where I am getting Not set - Please check screenshot-2. So what is Not set here? Thanks! QcBGT OCrEp
Reporting & Analytics | | Arnold30 -
Which Algorithm Change Hurt the Site? A causation/correlation issue
The attached graph is from google analytics, a correlation of about 14 months of Organic Google visits with algo changes, data from moz naturally 🙂 Is there any way to tell from this which will have affected the site? for example #1 or #2 seems to be responsible for the first dip, but #4 seems to fix it and it broke around 6, or is the rise between 4 and 7 an anomaly and actually 1 or 2 caused a slip from when it was released all the way to when 7 was released. Sorry if the graph is a little cloak and dagger, that is partly because we don't have permissions to reveal much about the identity, and partly because we were trying to do a kind of double blind, separating the data from our biases 🙂 We can say though the different between the level at the start and end of the graph is at least 10,000 visits per day JarMzoK.png
Reporting & Analytics | | Fammy0 -
Google Analytics Organic search queries aren't being updated, even though I'm still seeing results in all our typical results pages.
We pushed some new changes to the site and Google Analytics is no longer updating the Organic Search queries listing, even though traffic is consistent and and we're still landing results in all our typical keyword searches. Any ideas?
Reporting & Analytics | | unclekaos0 -
Getting traffic for another site
Hi Everyone, Our website url/brand is very close to another website url/brand. We are non-competing entities. It appears as though this other company has begun a marketing program which has resulted in our traffic skyrocketing. However, it seems to have also resulted in our Pages/Visit and Visit Duration to decrease and our Bounce Rate to increase. Can anyone suggest how to deal with this type of scenario? Thanks,
Reporting & Analytics | | AC_Pro
Robert0 -
How to find out which URLs are NOT indexed on a site
Is there a way to easily find out which URLs on a store-type site are NOT being indexed in Google? For example, if my sitemap information in Google Webmaster tools shows I have 7342 URLs in my sitemap and 5699 of those indexed, how do I find out what the 1643 non-indexed URLS are? Thanks for any help!
Reporting & Analytics | | GregWalt0 -
Open Site Explorer Discrepency
Hi, I'm just starting to look into this tool as a result of the need to analyse some work that is being carried out on my site by an external SEO company that is charging absolute top dollar. I'm sure it's a simple answer, but looking at the attached image you will see that the total links is shown as 219 which is described as links from all sources. However the report that is on the page shows only 44 links which would appear to be a report based on the same criteria. What is the difference in the numbers please? Simon siteExplorer.jpg
Reporting & Analytics | | simonphumphries0 -
No Internal Followed Links on My Site?
I just ran the Site Explorer on my site (myeyedr.com) and it reported that there are no internal links on www.myeyedr.com or myeyedr.com. However, there were 20 internal followed links on www.myeyedr.com/index.aspx. I'm guessing this is hurting my SEO. Any suggestions?
Reporting & Analytics | | lance-1744490 -
Sort referring sites by visit change over time comparison in GA
I can't believe I've never done this before, so I'm going to assume that I previously must have figured it out via excel, but I'm hoping there's an easier way. So I want to compare the referring sites between April and May and see which have sent (specifically) less traffic. The problem with doing a comparison in GA is that it only sorts by the highest traffic for May, when actually I want to see the largest negative change (by number, not percentage) between April and May. Is there a way to do this via the dashboard or am I just going to have to play about in excel for 10 minutes?
Reporting & Analytics | | StalkerB0