Google is not indexing all URLs
-
My website have company and events profile from 200 countries. So it does have lots of URL. Earlier in August 2014, Google used to crawl 90% of URLs we submit. Thing goes wrong when we shifted from http to https. We lost traffic. But we are gaining it slowly. Main concern is that, It still does not indexed all submitted URLs. It have crawled merely 8% of all URLs submitted. site address is businessvibes.com
Any help would be appreciated.
-
Hi,
It sounds like something may be getting blocked or not crawled correctly, possibly because of the move from http --> https. I assume this is handled by the .htaccess file?
I would also check to see if there are https page issues on the site. If it's bit, you may have to do some digging around, but you can see this by using Google Chrome --> right click on the page --> inspect element --> console. This will show you the errors on the page.
-Andy
-
Hi Mossa,
Thanks for advice. We have already 2 millions of pages which are crawled. and good number of page views and session/day.
I will go with your point
-
You can see, almost all company and event profile have internal linking for other profile and events
-
We are doing that but we have pages in billion which is literally time taking as it have countries and events from 200 countries
-
It is very difficult to get backlining for billions of URLs, so branding and other activities are there to take care of that.
-
http to https have 301 redirection and all sitemap have been updated.
-
-
Before I start, why your website is using “_blank” even on internal links? This does not make any sense to me and I was literally pissed after three clicks.
Anyways, if you are interested in increasing the indexed pages here are the few things you could do:
- Improve internal linking of the website. Especially when possible link pages that are not indexed to the pages that are indexed.
- Get some social media shares (tweets) to pages that are not indexed.
- Build some quality links to the pages that are not indexed in Google.
Also, make sure that the 301 redirections from http tp https versions are correct and working and xml sitemap is updated.
If you do that I am sure that Google will not only index your pages but will also rank better from the desired key phrases.
Hope this helps!
-
Hi Martijn,
We have various sitemap
www.businessvibes.com/sitemap.companyprofile5.xml
www.businessvibes.com/sitemap.companyprofile6.xml
www.businessvibes.com/sitemap.companyprofile7.xml
And many More
-
Hi,
Do you have sitemaps that could provide some insights into your indexation rate for the data that you've submitted?
M.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google Analytics & Google Sheets
Hi guys, I'm looking to use google analytics google sheets extension to pull data into a google sheet with metrics from other tools e.g. semrush, moz, etc. From my understanding with google analytics google sheet extension its designed to run reports and doesn't really allow you to add other metrics. Basically what i'm trying to do is this: https://www.useloom.com/share/c1e42bfa60bd46fca2b1120018969ce8 Any suggestions/advice on how to do this would be great. Cheers.
Reporting & Analytics | | jayoliverwright0 -
How do you adjust the Google Analytics date range?
Hello! Every time I log into Google Analytics (via desktop), the date range is set to October 1, 2013 to October 28, 2013. I've looked throughout the GA settings to find a place where I change this default setting to anything else ("Last week?") but have not be able to find where that setting is. Can you help? Losing my mind,
Reporting & Analytics | | SmileMoreSEO
Erik0 -
Google Analytics shows wrong traffic source
I was reviewing out traffic sources on GA for this past week's traffic history, and noticed a couple of visits & conversions that GA cited as coming from campaigns from emails that were sent out in 2012 and early 2013. it seems odd that we would have traffic & conversions from these older campaigns - on a regular basis. it's not even from one campaign, but from multiple campaigns. is it possible that it is a cookie issue? - i wonder if these visits were really from the email campaigns - or if they were from organic searches or other sources. Thoughts? Thanks!
Reporting & Analytics | | S.S.N0 -
How Google measure website bounce rate ?
Bounce rate is a SEO signal, but how Google measures it ? There is any explanation about this ? Does Google uses Analytics ? Maybe time between 2 clics in search results ? Thanks
Reporting & Analytics | | Max840 -
Google Custom Report Filters are a pain in the arse
Greetings from high weed pollen count Wetherby UK 🙂 Ok ive been having extreme custom reporting filter frustration with Google analytics. My objective was to set up a "custom" google analytics report that detailed: 1. Referral traffic from external sights in one reporting tab
Reporting & Analytics | | Nightwing
2. Visitor generic traffic report including direct, search & referral traffic e.g: http://i216.photobucket.com/albums/cc53/zymurgy_bucket/reporting-tabscopy.jpg But Ive hit the provebial buffers 😞 Initailly i added an advanced filter as shown here: http://i216.photobucket.com/albums/cc53/zymurgy_bucket/custom-reporting-with-filtercopy.jpg but the advanced filters do not stick / save. As soon as you exit the custom report the advanced filter is cleared as shown here: http://i216.photobucket.com/albums/cc53/zymurgy_bucket/custom-reporting-filter-removedcopy.jpg I then added a filter under a specific custom tab but the filter cascades into all the reporting tabs see below: http://i216.photobucket.com/albums/cc53/zymurgy_bucket/custom-reporting-cascading-filterscopy.jpg So my question is please... "How do can you set up filters that effect the data in one reporting tab and does not cascade into the others" Any insights welcome 🙂0 -
Duplicate content? Split URLs? I don't know what to call this but it's seriously messing up my Google Analytics reports
Hi Friends, This issue is crimping my analytics efforts and I really need some help. I just don't trust the analytics data at this point. I don't know if my problem should be called duplicate content or what, but the SEOmoz crawler shows the following URLS (below) on my nonprofit's website. These are all versions of our main landing pages, and all google analytics data is getting split between them. For instance, I'll get stats for the /camp page and different stats for the /camp/ page. In order to make my report I need to consolidate the 2 sets of stats and re-do all the calculations. My CMS is looking into the issue and has supposedly set up redirects to the pages w/out the trailing slash, but they said that setting up the "ref canonical" is not relevant to our situation. If anyone has insights or suggestions I would be grateful to hear them. I'm at my wit's end (and it was a short journey from my wit's beginning ...) Thanks. URL www.enf.org/camp www.enf.org/camp/ www.enf.org/foundation www.enf.org/foundation/ www.enf.org/Garden www.enf.org/garden www.enf.org/Hante_Adventures www.enf.org/hante_adventures www.enf.org/hante_adventures/ www.enf.org/oases www.enf.org/oases/ www.enf.org/outdoor_academy www.enf.org/outdoor_academy/
Reporting & Analytics | | DMoff0 -
How Can I Record .zip Traffic in Google Analytics
The company I am with shows traffic coming in to .zip files in Google Analytics. The traffic being recorded stopped a while back, but I know people are still downloading/visiting these .zip ULRs. I'm not sure how they were recording/tracking the .zip URLs (in Analytics) before, but I'd like to track them once again. Does anyone know how to do this? Thanks
Reporting & Analytics | | poolguy0 -
Regular Expressions in Google Analytics
I want to use the Google Analytics landing page reports to look at the bounce rate of top level pages excluding the homepage. So pages with urls: www.example.com/example Does anyone know a regular expression that will allow me to do this? Just to clarify I do not want to look at the bounce rate of the homepage or any pages deeper than www.example.com/example e.g: www.example.com/example/example www.example.com/example/example/example etc Thanks in advance
Reporting & Analytics | | CPLDistribution0