Is the Google results serp broken?
-
Hi everyone!
We've been trying to decipher how many of our pages are indexed by google at the moment. If we do the usual "site:https://www.hobbydb.com" search term, the serp says that we have more than 740,000 pages indexed. However, when I do a deep dive and click through to the last page of results, I can only get to page 54, and then there are no more results. This would mean that I only have 540 pages indexed, not 740,000. We have also done other queries for other sub-sections of our website, and the results also truncate at 50 pages. Has anyone run into this problem? Any suggestions are appreciated!
Best,
Alex
-
Google search console Index Status report
Google Webmaster tools tell you in the
https://support.google.com/webmasters/answer/2642366?hl=en
if you want to know without a doubt you have to connect Google Analytics to Deepcrawl it will say exactly how many URLs are in the index.
- https://www.deepcrawl.com
- https://www.deepcrawl.com/knowledge/best-practice/how-to-measure-indexed-pages-more-accurately/
- https://www.deepcrawl.com/knowledge/best-practice/how-can-non-indexable-pages-receive-google-organic-traffic/
- http://www.stateofdigital.com/40-deepcrawl-tweaks/
Now that Google Analytics can be added to screaming frog you can do something very similar.
- https://www.screamingfrog.co.uk/seo-spider/
- https://www.screamingfrog.co.uk/seo-spider/user-guide/general/#crawling
- https://www.screamingfrog.co.uk/seo-spider/faq/
- https://moz.com/community/q/how-to-determine-which-pages-are-not-indexed
I Hope this helps,
Tom
-
Michael, the site: command typically has not "worked" or provided accurate results for years now. It's just they way it is--and I don't expect Google to really give us the number of pages that are actually indexed anytime soon.
That said, typically I recommend using the site:hobbydb.com command to see the pages, and then click to the final page of results (if you can). I'm currently seeing about 766,000 pages indexed.
-
That index query returns dubious results. URLProfiler wrote an excellent article last year sharing the results of some experimentation they did around the accuracy of that command and other tools used to help determine the extent of your site indexation. I recommend you give it a read - http://urlprofiler.com/blog/google-index-checker/. It will take some some work and access to their tool, but I think you'll gain a lot of insight.
Good luck!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google + and Google Knoladge Graph
I am trying to get things to match up for the company brand websearch and the Google + page and we have had it for years now The knowledge graph on Google is showing the map, address and name (shown in attached image), but is not linked to a G+ page, as when i click the "Are you the business owner?" its is trying to make me create a new G+ business page. Anyone have any ideas on this? Also does the wiki name have to be exact for it to show? As for phone number would that be coming from the DNS record as that is nowhere in the markup rich snippet or normal markup Thanks in advance LC9cWdG
Technical SEO | | David-McGawn0 -
Getting Recrawled by Google
I have been updating my site a lot and some of the updates are showing up in Google and some are not. Is there a best practice in getting your site fully recrawled by Google?
Technical SEO | | ShootTokyo0 -
No Ranking for bathroom vanities in organic results for google
My website is www.tanyas.ca and I noticed that I can't find a result in the organic directory of google for my main keyword, "bathroom vanities". Your help is greatly appreciated. Cam
Technical SEO | | camc0 -
Homepage disappeared from Google Serp
I redirected my domain using this code in .htaccess : RewriteCond %{HTTP_HOST} ^xxxx.com
Technical SEO | | digitalkiddie
RewriteRule (.*) http://www.xxxx.com/$1 [R=301,L]
<ifmodule mod_rewrite.c="">RewriteEngine On
RewriteBase /
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L] RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /([^/]/)index.(html?|php)(?[^\ ])?\ HTTP/
RewriteRule ^(([^/]/)*)index.(html?|php)$ http://www.xxxx.com/$1 [R=301,L]</ifmodule> A day after I did it, got an error in GWMT "Google can't find your site's robots.txt" and my homepage disappeared from the result pages. When I try to open Google cache of the homepage I got an error 404. I generated new robots.txt, uploaded it , now the error doesnt show but still my homepage is not in the serps. Its been 3 days. What should I do ? Thanks in advance "Google can't find your site's robots.txt" error? - Pro ...0 -
Google autorship in specific field?
Hi, I want to ask you about something I 've read about google and authorship. It is written that it is better to show yourself as a author in a specific field. I myself have knowledge and interest in many fields - like SEO, vegan living, martial arts. And I want to be seen as specialist in all of them. Does it mean that we are limited to mark with autorship articles in only one field, in order to be seen as expert in a specific field? f.e. Should I mark with "rel=author" the articles that are about SEO because I want to be seen as author in that specific field for sure. Iif I mark with "rel=author" articles also about martial arts would these affect the understanding about my expertise in SEO?
Technical SEO | | vladokan0 -
Google webmaster tools
I have linked webmaster tools to Google analytics account. My question is where can i see Webmaster reports in Google analytics ?
Technical SEO | | seoug_20050 -
Not ranking well in Google
Hi, I am new to Seomoz,I have some little doubts regarding <title>tag.</p> <p>Can i target 3 words in the title tag. Currently i am on top for one keyword, and i cant get the rest two in top positions. Here is my website, can anyone review my site please.</p> <p>xxx(dot)ridpiles(dot)com with keyword hemorrhoids treatment</p> <p>I have good amount of backlinks, but still something i am missing. I have 100% unique content.</p> <p> </p> <p>Regards</p></title>
Technical SEO | | Dexter22387874870 -
Google dropping pages from SERPS
The website for my London based plumbing company has thousands of specifically tailored pages for the various services we provide to all the areas in London. It equates to approximately 6000 pages in total. When google has all these pages indexed, we tend to get a fair bit of traffic - as they cater pretty well for long tail searches. However, every once in a while Google will drop the vast majority of our indexed pages from SERPs for a few days or weeks at a time - for example at the moment Google is only indexing 613 whereas last week it was back at the normal ~6000. Why does this happen? We of course lose a lot of organic traffic when these pages don't displayed - what are we doing wrong? Website: www.pgs-plumbers.co.uk
Technical SEO | | guy_andrews0