How accurate are the index figures in GWT?
-
I've been looking at a site in GWT and the number of indexed urls is very low when compared with the number or submitted urls on the xml sitemaps.
The site has several stores which are all submitted using different sitemaps.
When you perform a search in Google, eg
site:domain.com/store1
site:domain.com/store2
site:domain.com/store3
The results are similar to the webmaster urls.
However, looking in the analytics for landing pages used for organic traffic from Google shows a much higher number of pages.
If these pages aren't indexed as reported in GMT, how could they be found in the results and be recorded as landing pages?
-
Why are you using more than 1 site map per domain?
the answer could be so many things but I think it's this you're using more than one site map per domain Google gets confused and does not index your entire website.
Your server could be too slow if it's e-commerce are probably not speeding up your site fast enough to have Google actually index the links properly. Remember the faster your website the deeper Google goes when indexing the site.
E-commerce sites with more than one site map on possibly slow hosting it sounds about right that Google would not actually index every single one of the pages that you have submitted over and over again.
Clear out the multiple sign-ups then pick a single site map if you're using plug-ins choose just one if you're using generated choose just one.
Add it to your website remove the other site maps then submit the site map to Google webmaster tools when it says index this one page or all pick all and if that doesn't work you can use fetch with the Google bot to get your individual webpages crawled then have them submitted by hand.
I would use a content delivery network if you're using e-commerce make sure your site speed is fast.
Check out your site speed using this http://www.webpagetest.org/
then use the tool below to figure out why and what you can do about it.
http://torbit.com/site-optimizer/
I strongly suggest you invest in a content delivery network
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Site shows up after re-indexing, then disappears.
I have a site, natvest.com, with which I sell real estate in Alabama and Georgia. I need to show up in an "Alabama Land for Sale" search. Same thing for Georgia. If I re-index my site, I show up for roughly one day, before disappearing again. Happens every time I re-index. Ideas?
Intermediate & Advanced SEO | | natvest0 -
Homepage meta title not indexing correctly on google
Hello everyone! We're having a spot of trouble with our website www.whichledlight.com The meta title is coming up wrong on google. In Google it currently reads out
Intermediate & Advanced SEO | | TrueluxGroup
'Which LED Light: LED Bulbs & Lamps Compared'
when it should be
'LED Bulbs & Lamps Compared | Which LED Light' Last snapshot of the page from google was yesterday (5th April 2016) Anyone got any ideas?
Is all the markup correct in the ?0 -
GWT, Editing URL Parameters for Ecommerce Features
I have had the setting of "let googlebot decide" on managing my URL parameters on an Ecommerce site in Magento. The products I sell come in different sizes and colors and finishes etc. These parameters are showing up in Google Webmaster Tools and set for "let googlebot decide". Some of them have as many as 8 million urls monitored. I changed the editing option to clam these parameters as "narrow searches", but still left the option to "let googlebot decide" (versus block urls). Will blocking these erroneous urls serve any benefit? Does blocking these help with the crawl/seo?
Intermediate & Advanced SEO | | nat88han0 -
GWT Message - CMS Update Available
Howdy Moz, Just received a message in Google Webmaster Tools about a CMS update: "Joomla Update Available As of the last crawl of your website, you appear to be running Joomla 1.5. One or more of the URLs found were: http://www.website/custom-url/article5034 Google recommends that you update to the latest release. Older or unpatched software may be vulnerable to hacking or malware that can hurt your users. To download the latest release, visit the Joomla download page. If you have already updated to the latest version of Joomla, please disregard this message. If you have any additional questions about why you are receiving this message, Google has provided more background information in a blog post about this subject." Read through the associated blog post. According to the post a generator meta tag is created in Joomla that notes the CMS version. Here's the oddity: The site was on Joomla 1.5 over 2 years ago. 1 Year ago it was updated to Joomla 2.5. About a week ago it was converted completely to Wordpress. According to GWT the last date the Google bot accessed the site was the day before (5/1/14) the email. I went through the code, css/html, and the database and found no reference of Joomla 1.5. Has anyone seen this message? If so, how did you rectify it? Were there any adverse effects on rankings?
Intermediate & Advanced SEO | | AaronHenry0 -
Google is Really Slow to Index my New Website
(Sorry for my english!) A quick background: I had a website at thewebhostinghero.com which had been slapped left and right by Google (both Panda & Penguin). It also had a manual penalty for unnatural links which had been lifted in late april / early may this year. I also had another domain, webhostinghero.com, which was redirecting to thewebhostinghero.com. When I realized I would be better off starting a new website than trying to salvage thewebhostinghero.com, I removed the redirection from webhostinghero.com and started building a new website. I waited about 5 or 6 weeks before putting any content on webhostinghero.com so Google had time to notice that the domain wasn't redirecting anymore. So about a month ago, I launched http://www.webhostinghero.com with 100% new content but I left thewebhostinghero.com online because it still brings a little (necessary) income. There are no links between the websites except on one page (www.thewebhostinghero.com/speed/) which is set to "noindex,nofollow" and is disallowed to search engines in robots.txt. I made sure the web page was deindexed before adding a "nofollow" link from thewebhostinghero.com/speed => webhostinghero.com/speed Since the new website launch, I've been publishing new content (from 2 to 5 posts) daily. It's getting some traction from social networks but it gets barely any clicks from Google search. It seems to take at least a week before Google indexes new posts and not all posts are indexed. The cached copy of the homepage is 12 days old. In Google Webmaster Tools, it looks like Google isn't getting the latest sitemap version unless I resubmit it manually. It's always 4 or 5 days old. So is my website just too young or could it have some kind of penalty related to the old website? The domain has 4 or 5 really old spammy links from the previous domain owner which I couldn't get rid of but otherwise I don't think there's anything tragic.
Intermediate & Advanced SEO | | sbrault740 -
Few questions regarding wordpress and indexing/no follow.
I'm using Yoast's Wordpress SEO plugin on my wordpress site which allows you to quickly set up nofollow / no index on specific taxonomies. I wanted to see what you guys thought was the best practice in setting up my various taxonomies. Would you noidex, but follow all of these, none of these, or just some of these: Categories, tags, media, author archives ( (My blog is mainly a single author blog (me) but my wife does sometimes write posts. So I didn't know how this effected everything. Also I could simply make the blog a single user blog and just have her posts be guest posts, but I'd rather leave her as a user.), and date archives. The example I read on line only no-index's the date archives. Just curious what you guys thought. Thanks.
Intermediate & Advanced SEO | | NoahsDad0 -
Latent Semantic Indexing and Direct Match Domains
I wondered if anyone had any opinions as to whether LSI plays any part in the ranking of a direct match domain? For example :- would www.search-engine-optimisation.com be more likely to rank better for search terms such as 'SEO Services' or 'SEO Experts' than www.some-random-domain.com Does having 'Search Engine Optimisation' in the domain name mean that you would rank better for 'SEO'?
Intermediate & Advanced SEO | | AdeLewis0 -
Disallowed Pages Still Showing Up in Google Index. What do we do?
We recently disallowed a wide variety of pages for www.udemy.com which we do not want google indexing (e.g., /tags or /lectures). Basically we don't want to spread our link juice around to all these pages that are never going to rank. We want to keep it focused on our core pages which are for our courses. We've added them as disallows in robots.txt, but after 2-3 weeks google is still showing them in it's index. When we lookup "site: udemy.com", for example, Google currently shows ~650,000 pages indexed... when really it should only be showing ~5,000 pages indexed. As another example, if you search for "site:udemy.com/tag", google shows 129,000 results. We've definitely added "/tag" into our robots.txt properly, so this should not be happening... Google showed be showing 0 results. Any ideas re: how we get Google to pay attention and re-index our site properly?
Intermediate & Advanced SEO | | udemy0
