Google is not indexing all URLs
-
My website have company and events profile from 200 countries. So it does have lots of URL. Earlier in August 2014, Google used to crawl 90% of URLs we submit. Thing goes wrong when we shifted from http to https. We lost traffic. But we are gaining it slowly. Main concern is that, It still does not indexed all submitted URLs. It have crawled merely 8% of all URLs submitted. site address is businessvibes.com
Any help would be appreciated.
-
Hi,
It sounds like something may be getting blocked or not crawled correctly, possibly because of the move from http --> https. I assume this is handled by the .htaccess file?
I would also check to see if there are https page issues on the site. If it's bit, you may have to do some digging around, but you can see this by using Google Chrome --> right click on the page --> inspect element --> console. This will show you the errors on the page.
-Andy
-
Hi Mossa,
Thanks for advice. We have already 2 millions of pages which are crawled. and good number of page views and session/day.
I will go with your point
-
You can see, almost all company and event profile have internal linking for other profile and events
-
We are doing that but we have pages in billion which is literally time taking as it have countries and events from 200 countries
-
It is very difficult to get backlining for billions of URLs, so branding and other activities are there to take care of that.
-
http to https have 301 redirection and all sitemap have been updated.
-
-
Before I start, why your website is using “_blank” even on internal links? This does not make any sense to me and I was literally pissed after three clicks.
Anyways, if you are interested in increasing the indexed pages here are the few things you could do:
- Improve internal linking of the website. Especially when possible link pages that are not indexed to the pages that are indexed.
- Get some social media shares (tweets) to pages that are not indexed.
- Build some quality links to the pages that are not indexed in Google.
Also, make sure that the 301 redirections from http tp https versions are correct and working and xml sitemap is updated.
If you do that I am sure that Google will not only index your pages but will also rank better from the desired key phrases.
Hope this helps!
-
Hi Martijn,
We have various sitemap
www.businessvibes.com/sitemap.companyprofile5.xml
www.businessvibes.com/sitemap.companyprofile6.xml
www.businessvibes.com/sitemap.companyprofile7.xml
And many More
-
Hi,
Do you have sitemaps that could provide some insights into your indexation rate for the data that you've submitted?
M.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Pandora in Google Analytics
Hi Mozers, in Google Analytics our ads are registering a 10 second page duration and a 90% bounce rate, which is pretty atrocious. My questions are: Organic page duration is 3:15 minutes, and Social + Referral are similar. Yet Pandora is bringing the average time on site way down, to less than a minute. Will those paid ad metrics be factored to the User Behavior and therefore negative affect our overall rankings? In other words does Google now see our page duration as less than a minute? The Pandora rep says that Google Analytics and Pandora have "systemic issues" such that GA is very inaccurate with Pandora metrics. I find that difficult to believe. Any insights here? Thanks all! Jane
Reporting & Analytics | | CalamityJane770 -
In google analytic for google /cpc it is showing url with 404 which even not exists in my database
Hello All, In google analytic for google /cpc it is showing url with 404 which even not exists in my database that also more than 300 per day. How can it is possible? it is showing /black-friday-offers but I don't have such page. Thanks!
Reporting & Analytics | | pragnesh96390 -
Dynamic URL parameter generator
Hi Is any one privy to an online tool that will let me create a dynamic URL parameter string but will allow me to generate multiples of the URL and add a distinct key for each one. E.g Campaign Source, Medium, Name, a Keywords etc. are all the same in the string but then I want to generate a unique id code at the end. Then export them as a csv and integrate into my database lists. Looking to run this into a few thousand as well. I was going to just do this in Excel and combine two columns withe the string and the number count in the other column but if there is a tool that does it all that would be interesting to know.
Reporting & Analytics | | David-E-Carey0 -
Cross domain tracking Google Analytics
Hi there, Got a question on cross domain tracking: We have a couple of TLD's to serve localized content to our visitors, next to our main .com TLD where our app is running as well in a subdomain. Situation is this: Local sites:
Reporting & Analytics | | jorisbrabants
marketingsite.be
marketingsite.com.br
marketingsite.fr Main site: marketingsite.com
app.marketingsite.com Conversion gets triggered when somebody ends up in app.marketingsite.com/firstuse for example. People can sign up at the local site filling in their email but they end up in app.marketingsite.com/firstuse Reading this article on cross domain tracking I'm getting a bit confused on the setup of the tracking code itself. The sample code provided shows these two lines: ga('require', 'linker'); ga('linker:autoLink', ['maindomain.com','targetdomain.com']); Now the question 🙂 Is it correct when I think that maindomain should be replaced with our local TLD's on every one of those, and that targetdomain is where the conversion happens? In this case the .com site?0 -
Some goal conversion in Google analytics showing under referral
Yesterday I have created Google analytics account for a new website but few goal conversions comes from payment gateway site (paypal.com, epdq.co.uk) and showing under referral. How to fix this issue so I can know the real source of Goal conversion. *Note - utm_nooverride=1 on thank you page applied, payment gateway URL is already placed in Referral Exclusion List. So please don't suggest either of them. Thanks
Reporting & Analytics | | Alick3000 -
Where does the organic keyword information come from in Google Analytics?
I know that Google switch to all encrypted search, but I still show some keywords. Are those keywords that slipped through? Or are they all from Yahoo/Bing?
Reporting & Analytics | | EcommerceSite0 -
Google Analytics Content Experiments
Has anyone else found that Google Analytics Content Experiments seems to quite quickly favor the best performing variant in an experiment, and then show that variant many times more often than other/s - not split the traffic evenly? What is Google's thinking behind 'optimizing' during an experiment? It seems odd to me.
Reporting & Analytics | | David_ODonnell0 -
Google Bot blocked
Google says that they are blocked from crawling my site. I have tried to get my web masters to look into this and they say the site apears fine. However, my SERP results have been falling and the errors are increasing. http://www.RealEstateMarketLeaders.com https://support.google.com/webmasters/bin/answer.py?hl=en&answer=2409441 I have sent an e mail into Google asking for help and I have been looking into every possible channel to find a solution for this problem. The site does have a 301 re direct pointing in from my old site http:///www.TriCitiesRealEstateAgent.com into my new site http://www.RealEstateMarketLeaders.com. However, I am not sure if that is the problem or if something else happend to make me see such a huge drop in SERP results.
Reporting & Analytics | | Brandon_Patton0