Google is not indexing all URLs
-
My website have company and events profile from 200 countries. So it does have lots of URL. Earlier in August 2014, Google used to crawl 90% of URLs we submit. Thing goes wrong when we shifted from http to https. We lost traffic. But we are gaining it slowly. Main concern is that, It still does not indexed all submitted URLs. It have crawled merely 8% of all URLs submitted. site address is businessvibes.com
Any help would be appreciated.
-
Hi,
It sounds like something may be getting blocked or not crawled correctly, possibly because of the move from http --> https. I assume this is handled by the .htaccess file?
I would also check to see if there are https page issues on the site. If it's bit, you may have to do some digging around, but you can see this by using Google Chrome --> right click on the page --> inspect element --> console. This will show you the errors on the page.
-Andy
-
Hi Mossa,
Thanks for advice. We have already 2 millions of pages which are crawled. and good number of page views and session/day.
I will go with your point
-
You can see, almost all company and event profile have internal linking for other profile and events
-
We are doing that but we have pages in billion which is literally time taking as it have countries and events from 200 countries
-
It is very difficult to get backlining for billions of URLs, so branding and other activities are there to take care of that.
-
http to https have 301 redirection and all sitemap have been updated.
-
-
Before I start, why your website is using “_blank” even on internal links? This does not make any sense to me and I was literally pissed after three clicks.
Anyways, if you are interested in increasing the indexed pages here are the few things you could do:
- Improve internal linking of the website. Especially when possible link pages that are not indexed to the pages that are indexed.
- Get some social media shares (tweets) to pages that are not indexed.
- Build some quality links to the pages that are not indexed in Google.
Also, make sure that the 301 redirections from http tp https versions are correct and working and xml sitemap is updated.
If you do that I am sure that Google will not only index your pages but will also rank better from the desired key phrases.
Hope this helps!
-
Hi Martijn,
We have various sitemap
www.businessvibes.com/sitemap.companyprofile5.xml
www.businessvibes.com/sitemap.companyprofile6.xml
www.businessvibes.com/sitemap.companyprofile7.xml
And many More
-
Hi,
Do you have sitemaps that could provide some insights into your indexation rate for the data that you've submitted?
M.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google Analytics Real Time Not Working! :(
Hello Everyone, Today, the real time feature in my google analytics stopped working. I am able to see that traffic is visiting my site, but not in real time. The real time count is usually at 0. But, there are some instances where real time will come back online, but there will be only 1 unique visitor. At any given time, our website usually has 20 visitors or so..Has anyone encountered this issue? Where should i start looking for fixes? What are the possible problems? Thanks!
Reporting & Analytics | | KarlMarxTheBear0 -
Linking My Google analytics with visual.ly
Hello, I have a quick question: Is it secure and safe to link my Google Analytics account with Visual.ly here's the link: https://create.visual.ly/graphic/google-analytics I want to create easy-to-understand report from Google Analytics and if you have any experience with this service can you share it with me please. Thanks.
Reporting & Analytics | | JonsonSwartz0 -
Google Tag Manager breaking integration
Using Live Chat's (www.livechatinc.com) Google Analytics integration was populating events and virtual pageviews into my analytics account. I've since added Tag Manager and moved my analytics tracking code into there, but since doing so, the integration no longer seems to work as there is no population of either events or pageviews anymore. Anyone else had any experience of something similar? Any other suggestions (beyond not using GTM for analytics code anymore)? I was considering setting up the event tracking code manually in GTM, but not really sure how to do so seeing as I'm not sure what to fire the different events on. This is the live chat JS code:
Reporting & Analytics | | AdrianCordiner0 -
Google analytics - landing pages
Hello, I have a website and it contains landing pages. after tracking and analyzing the data in Google Analytics , I found out that the traffic for the whole site is affected by the visits to the landing page. I mean, the bounce rate is less than one minute. this is because users fill their details in the landing pages and leave my website. to improve my website, I need clear data about the traffic, bounce rate etc. in my website. I also need can you help me solve this problem I thought about create another account in Google Analytics and set it to my websites pages (not including the landing pages). and keep the current code for the landing pages. my question is regarding this: Is it legal to use 2 different GA code (each one of them belong to different account) in the same domain ? can you provide me with more information about Multi Account in Google Analytics, and how can I use it to divide the traffic in my website between the traffic and the data for the landing pages and for the website itself ?
Reporting & Analytics | | JonsonSwartz0 -
Google Bot blocked
Google says that they are blocked from crawling my site. I have tried to get my web masters to look into this and they say the site apears fine. However, my SERP results have been falling and the errors are increasing. http://www.RealEstateMarketLeaders.com https://support.google.com/webmasters/bin/answer.py?hl=en&answer=2409441 I have sent an e mail into Google asking for help and I have been looking into every possible channel to find a solution for this problem. The site does have a 301 re direct pointing in from my old site http:///www.TriCitiesRealEstateAgent.com into my new site http://www.RealEstateMarketLeaders.com. However, I am not sure if that is the problem or if something else happend to make me see such a huge drop in SERP results.
Reporting & Analytics | | Brandon_Patton0 -
Google Product Feed help
Hello, We are working with a particular service that uses our Google product feed to pull information. We wanted to add product info such as dimensions and hex colors for each product to better service the third party we are using, but we not sure if this is something Google supports in their feed. Would it cause any problems if we added that info in? Thanks for any help! Also, my knowledge regarding a Google product feed is lacking so if the question is confusing or I didn't give enough info on our situation just let me know and I'll do what I can to better explain.
Reporting & Analytics | | ClaytonKendall0 -
Why do I have few different index URL addresses?
Yes I know, sorry guys but I also have a problem with duplicate pages. It shows that almost every page of my site has a duplicate content issue and looking at my folders in the server, I don't see all these pages... This is a static Website with no shopping cart or anything fancy. The first on the list is my [index] page and this is giving me a hint about some sort of bad settings on my end with the SEOMOZ crawler??? Please advice and thank you! index-variations.jpg
Reporting & Analytics | | cssyes0 -
How to measure number of visits from Google News coming from Google Universal Search (NOT referral coming directly coming from news.google.com) with google analyitcs
I'm running a news site, and I have a problem of accuratly measuring which traffic is REALLY coming from google news. I analyzed a lot of individual articles and I come to the conclusion, that the visits, that come from the google news section in the universal search results are counted as "normal" search engine traffic in google analytics. So if you do a Google search for a topic that includes links from Google news, you don't get an accurate referral count. As an example, if you do a search for "eBay", incorporated into the page 1 search results you may also see Google news results as well.
Reporting & Analytics | | Mulle
If someone clicks on that Google news link that appears in Google search, it shows up in Google analytics as a referral from Google search, when it was actually from a Google news referral. I was already checking google analytics and google news help forums and searched SEO blogs for this. But I wasn't able to find a working solution. Can anybody help me out with this problem? Thanks so much, Matthias0