Google is not indexing all URLs
-
My website have company and events profile from 200 countries. So it does have lots of URL. Earlier in August 2014, Google used to crawl 90% of URLs we submit. Thing goes wrong when we shifted from http to https. We lost traffic. But we are gaining it slowly. Main concern is that, It still does not indexed all submitted URLs. It have crawled merely 8% of all URLs submitted. site address is businessvibes.com
Any help would be appreciated.
-
Hi,
It sounds like something may be getting blocked or not crawled correctly, possibly because of the move from http --> https. I assume this is handled by the .htaccess file?
I would also check to see if there are https page issues on the site. If it's bit, you may have to do some digging around, but you can see this by using Google Chrome --> right click on the page --> inspect element --> console. This will show you the errors on the page.
-Andy
-
Hi Mossa,
Thanks for advice. We have already 2 millions of pages which are crawled. and good number of page views and session/day.
I will go with your point
-
You can see, almost all company and event profile have internal linking for other profile and events
-
We are doing that but we have pages in billion which is literally time taking as it have countries and events from 200 countries
-
It is very difficult to get backlining for billions of URLs, so branding and other activities are there to take care of that.
-
http to https have 301 redirection and all sitemap have been updated.
-
-
Before I start, why your website is using “_blank” even on internal links? This does not make any sense to me and I was literally pissed after three clicks.
Anyways, if you are interested in increasing the indexed pages here are the few things you could do:
- Improve internal linking of the website. Especially when possible link pages that are not indexed to the pages that are indexed.
- Get some social media shares (tweets) to pages that are not indexed.
- Build some quality links to the pages that are not indexed in Google.
Also, make sure that the 301 redirections from http tp https versions are correct and working and xml sitemap is updated.
If you do that I am sure that Google will not only index your pages but will also rank better from the desired key phrases.
Hope this helps!
-
Hi Martijn,
We have various sitemap
www.businessvibes.com/sitemap.companyprofile5.xml
www.businessvibes.com/sitemap.companyprofile6.xml
www.businessvibes.com/sitemap.companyprofile7.xml
And many More
-
Hi,
Do you have sitemaps that could provide some insights into your indexation rate for the data that you've submitted?
M.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google Analytics Landing Page Report Discrepancy
I have noticed that when I run a landing page report and use the advanced option so I can view only the landing pages that include a particular string in the URL, have noticed that I in the report, the graph at the top will say one thing, but the data below says something else. For example, the graph for one particular search shows 200 Impressions, but the info below says 700 impressions and 610 clicks. Anyone seen anything similar or have any ideas why? Thanks! Craig
Reporting & Analytics | | TheCraig0 -
Strange Traffic Spike In Google Analytics
Over the last two days I have seen a huge spike in the pageviews to my site www.learnbonds.com. Up to this point my average page views per visit has been less than 2. For the last couple of days it has jumped to 5 but the average time on the site has not increased. Upon further investigation under the browser and OS page of google analytics I found that traffic coming from internet explorer and IE with Chrome Frame is reporting 10+ pageviews per visit. Below is a screenshot. Any insight into what might be causing this and how to fix it that the community can provide would be appreciated. Thanks
Reporting & Analytics | | davidwaring
DaveZUaQWJd.png?1?21800 -
Google analytics Goals and Funnels??
Hey - how's things? I've got 2 goals setup, each of which use the same thank you page but with different required funnels. The results have been identical for the last few weeks - which tells me I've got something wrong!! The first goal has a contact form on every page in the /products/ directory set up like this:
Reporting & Analytics | | agua
destination equal to: /thanks-enquiry/funnel step1 page: ^/products/.* required: yes The first goal has a form on 1 page called /on-sale set up like this:
destination equal to: /thanks-enquiry/
funnel step1 page: /on-sale required: yes Any ideas?0 -
What's the problem with this code for Google Analytics
Hello, I'm using this code in my thank you page to track ecommerce sales in GA. The problem that I'm having is that in the google event the _trackPageview doesn't register the proper page but puts the url of the ecommerce website. The transaction is properly tracked. Any ideas of what could be the issue looking at this code? Maybe there could be other code could be causing the problem? Thanks a lot.
Reporting & Analytics | | nagar0 -
Have we been penalised by Google?
Hello guys, Looking for some help here hope someone will be able to point us to the right direction, we have been scratching our heads for months! Basically our Google rankings and organic traffic from Google have dropped signficantly in the last 12 months. At one point we ranked as high as page 3 with the term "holiday lettings". Now we are on page 16, and sometimes not found at all... This has been the case for months now. Curiously, Bing and Yahoo have been much more generous to us, we have been and still consistently rank at No. 2 with the same term. We are in the vacation rental industry - our website is www.alphaholidaylettings.com and most of the sites that rank above us are of much less relevancy, quality and authority. We just could not understand why we would rank so poorly on Google (both US and UK). Organic traffic has also dropped by more than 50% steadily in the last 12 months. One thing we thought might have affected our site is the quality of content. Most content/listings on our site are user generated, and it is not uncommon for users to copy and paste details from an existing listing that they have on another site. Having said that, the same would apply to our competitors too. In this case, might Panda or Penguin have had any negative effects on our seo? We have been adding unique content to the site in the form of a blog and "Holiday Feature" articles. Could the actual design/structure/coding of our pages contribute to our poor rankings too? Any opinions or feedback is much appreciated! Thanks guys, Tom
Reporting & Analytics | | forgottenlife0 -
How do I manually add transactions to Google Analytics
We are seeing Google Analytic's drop transaction on our site so therefore all the figures are skewed. Is there a way I can manually add transactions to GA to cover the missing one?
Reporting & Analytics | | Towelsrus0 -
Google Analytics bug?
Hi, I'm getting hits to this page, it's screwing up my Google Analytics stats. I know it's related with Google Website Optimizer, cuz it correlates with the tests I ran at the same dates. I'm just wondering if you've seen this before and what I should do to clean my stats? And is this affecting my SEO? Because it's still registering hits to these pages but I already stopped the tests a while go. Should I disallow this in my robots.txt file? Thanks! ga_bug3.jpg
Reporting & Analytics | | rhenster990 -
Drop in google referral traffic
Hi guys, As we know, GA shows google as traffic source in two ways: google / organic for organic searches and google.TLD / referral for everything else: google groups, base.google.com, static pages, google reader, google image search, google search appliance/mini. What we noticed is that around Oct 20th there's a huge drop of google.TLD / referral traffic to our site. Do you experience something similar? I couldn't find anything Google-related that happened around this specific date. We use GSA for our site search and I'm wondering if this could be the reason - maybe someone from our development team made changes to GSA settings that affected this traffic source. Looking forward to hearing from you! Thanks.
Reporting & Analytics | | lgrozeva0
