Google is not indexing all URLs
-
My website have company and events profile from 200 countries. So it does have lots of URL. Earlier in August 2014, Google used to crawl 90% of URLs we submit. Thing goes wrong when we shifted from http to https. We lost traffic. But we are gaining it slowly. Main concern is that, It still does not indexed all submitted URLs. It have crawled merely 8% of all URLs submitted. site address is businessvibes.com
Any help would be appreciated.
-
Hi,
It sounds like something may be getting blocked or not crawled correctly, possibly because of the move from http --> https. I assume this is handled by the .htaccess file?
I would also check to see if there are https page issues on the site. If it's bit, you may have to do some digging around, but you can see this by using Google Chrome --> right click on the page --> inspect element --> console. This will show you the errors on the page.
-Andy
-
Hi Mossa,
Thanks for advice. We have already 2 millions of pages which are crawled. and good number of page views and session/day.
I will go with your point
-
You can see, almost all company and event profile have internal linking for other profile and events
-
We are doing that but we have pages in billion which is literally time taking as it have countries and events from 200 countries
-
It is very difficult to get backlining for billions of URLs, so branding and other activities are there to take care of that.
-
http to https have 301 redirection and all sitemap have been updated.
-
-
Before I start, why your website is using “_blank” even on internal links? This does not make any sense to me and I was literally pissed after three clicks.
Anyways, if you are interested in increasing the indexed pages here are the few things you could do:
- Improve internal linking of the website. Especially when possible link pages that are not indexed to the pages that are indexed.
- Get some social media shares (tweets) to pages that are not indexed.
- Build some quality links to the pages that are not indexed in Google.
Also, make sure that the 301 redirections from http tp https versions are correct and working and xml sitemap is updated.
If you do that I am sure that Google will not only index your pages but will also rank better from the desired key phrases.
Hope this helps!
-
Hi Martijn,
We have various sitemap
www.businessvibes.com/sitemap.companyprofile5.xml
www.businessvibes.com/sitemap.companyprofile6.xml
www.businessvibes.com/sitemap.companyprofile7.xml
And many More
-
Hi,
Do you have sitemaps that could provide some insights into your indexation rate for the data that you've submitted?
M.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Funnel Problem in Google Analytics - Please help
Hello, I did set up a several funnel steps in my analytics account to get the funnel visualization report. But it's look fishy, something's clearly wrong that it shows only 10 visitors at the first step but actually more than 5k visitors has visited first goal page and followed through the another all steps. Everyday we are receiving 5k to 10k goal conversions but it not showing in the entire funnel steps through first step to last step. Please see the attached screenshot URL for funnel setup (Admin) and funnel visualization report comparing last two months traffic. Total goal conversions received 3,605 in last two months but in funnel chart (report) it showing only 2% conversions in last step. Anybody can tell me what is the problem with that or analytics not tracking properly?si65Y
Reporting & Analytics | | flightcentre200 -
No-indexed pages are still showing up as landing pages in Google Analytics
Hello, My website is a local job board. I de-indexed all of the job listing pages on my site (anything that starts with http://www.localwisejobs.com/job/). When I search site:localwisejobs.com/job/, nothing shows up. So I think that means the pages are not being indexed. When I look in Google Analytics at Acquisition > Search Engine Optimization > Landing Pages, none of the job listing pages show up. But when I look at Acquisition > Channels > Organic and then click Landing Page as the primary dimension, the /job pages show up in there. Why am I seeing this discrepency in Organic Landing pages? And why would the /job pages be showing up as landing pages even though they aren't indexed?
Reporting & Analytics | | mztobias0 -
Google Analytics "Undetermined" Keyword
Google analytics is suddenly showing a new keyword "undetermined". Traffic Sources > Sources > All Traffic with the primary dimension set to Keyword. It's currently our 13th keyword and it is displayed without parentheses around that you see with (not provided) or (not set). Anyone have any idea what this is? Thanks!
Reporting & Analytics | | Morris770 -
Is Google Webmaster Tools Index Status Completely Correct?
So I was thrilled when Webmaster tools released the little graph telling you how many pages are indexed. However, I noticed with one of my sites, that it was actually going down over the past year. I've only been doing the SEO on the site for a few months, so it wasn't anything I did. The chart is attached to this post. Now here's the funny part. I haven't really noticed anything out of the ordinary for keyword ranking dropping off. I also tested my most recent page that I put up. 3 days after I posted it, I could find it in a Google search. Shouldn't this mean that's it's indexed? I can also find any other page I've posted in the last few months. Another oddity is that I submitted a sitemap a while ago when the site was only 22 pages. The sitemap index count says 20 of those pages are indexed. The chart only says that there are 3 indexed pages right now. However, I can clearly find dozens of pages in Google searches. Is there something I'm missing? Is my chart for this website broken? Should I report this to Google? Has anyone had a similar issue? decreaseIndex.png
Reporting & Analytics | | mytouchoftech0 -
Google Analytics & AdWords Conversion Tracking
I have Google Analytics implemented on my site and have goals/funnels set up to track the conversion process. I have just started an AdWords campaign but can't see/track conversions from it. Do I need to implement both Google Analytics AND AdWords scripits on the page?
Reporting & Analytics | | mj7750 -
Duplicate content? Split URLs? I don't know what to call this but it's seriously messing up my Google Analytics reports
Hi Friends, This issue is crimping my analytics efforts and I really need some help. I just don't trust the analytics data at this point. I don't know if my problem should be called duplicate content or what, but the SEOmoz crawler shows the following URLS (below) on my nonprofit's website. These are all versions of our main landing pages, and all google analytics data is getting split between them. For instance, I'll get stats for the /camp page and different stats for the /camp/ page. In order to make my report I need to consolidate the 2 sets of stats and re-do all the calculations. My CMS is looking into the issue and has supposedly set up redirects to the pages w/out the trailing slash, but they said that setting up the "ref canonical" is not relevant to our situation. If anyone has insights or suggestions I would be grateful to hear them. I'm at my wit's end (and it was a short journey from my wit's beginning ...) Thanks. URL www.enf.org/camp www.enf.org/camp/ www.enf.org/foundation www.enf.org/foundation/ www.enf.org/Garden www.enf.org/garden www.enf.org/Hante_Adventures www.enf.org/hante_adventures www.enf.org/hante_adventures/ www.enf.org/oases www.enf.org/oases/ www.enf.org/outdoor_academy www.enf.org/outdoor_academy/
Reporting & Analytics | | DMoff0 -
Links and index missing from Google Webmaster Tools!
I've been using a wordpress blog and using google analytics and webmaster tools. I'd never done very much optimizing of my site, my strategy had always been to just keep writing relevant posts and let google do the rest. At some point I installed a SEO plugin to wordpress and did a tiny amount of tweaking, but I noticed that in my webmaster tools page, all my "links to this site" links are gone! My site map says 42 pages submitted, 42 pages indexed, but all the links that I've built over the past few years are missing, and I also notice that I can't find my main landing page in the index anymore. I know there are indexed sites out there that link to me, but they aren't being reflected anymore. Did I anger google by using SEO plugins within wordpress? I'm fairly certain I wasn't doing anything sketchy. Http://brodiegraphics.com RTOH6.png
Reporting & Analytics | | HELPeR0