Google is not indexing all URLs
-
My website have company and events profile from 200 countries. So it does have lots of URL. Earlier in August 2014, Google used to crawl 90% of URLs we submit. Thing goes wrong when we shifted from http to https. We lost traffic. But we are gaining it slowly. Main concern is that, It still does not indexed all submitted URLs. It have crawled merely 8% of all URLs submitted. site address is businessvibes.com
Any help would be appreciated.
-
Hi,
It sounds like something may be getting blocked or not crawled correctly, possibly because of the move from http --> https. I assume this is handled by the .htaccess file?
I would also check to see if there are https page issues on the site. If it's bit, you may have to do some digging around, but you can see this by using Google Chrome --> right click on the page --> inspect element --> console. This will show you the errors on the page.
-Andy
-
Hi Mossa,
Thanks for advice. We have already 2 millions of pages which are crawled. and good number of page views and session/day.
I will go with your point
-
You can see, almost all company and event profile have internal linking for other profile and events
-
We are doing that but we have pages in billion which is literally time taking as it have countries and events from 200 countries
-
It is very difficult to get backlining for billions of URLs, so branding and other activities are there to take care of that.
-
http to https have 301 redirection and all sitemap have been updated.
-
-
Before I start, why your website is using “_blank” even on internal links? This does not make any sense to me and I was literally pissed after three clicks.
Anyways, if you are interested in increasing the indexed pages here are the few things you could do:
- Improve internal linking of the website. Especially when possible link pages that are not indexed to the pages that are indexed.
- Get some social media shares (tweets) to pages that are not indexed.
- Build some quality links to the pages that are not indexed in Google.
Also, make sure that the 301 redirections from http tp https versions are correct and working and xml sitemap is updated.
If you do that I am sure that Google will not only index your pages but will also rank better from the desired key phrases.
Hope this helps!
-
Hi Martijn,
We have various sitemap
www.businessvibes.com/sitemap.companyprofile5.xml
www.businessvibes.com/sitemap.companyprofile6.xml
www.businessvibes.com/sitemap.companyprofile7.xml
And many More
-
Hi,
Do you have sitemaps that could provide some insights into your indexation rate for the data that you've submitted?
M.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google Analytics Not Properly Attributing Goals
My GA has been working fine for years and is now suddenly attributing 100% of my goals/conversions to a "referral" source. In this case "crm.zoho.com". (see attached url) Our contact form is a zoho CRM form. When prospect fills it out online, it dumps them to a thank-you page and the conversion is counted. That all works just fine, but it is not attributing the conversion to Organic or PPC or Direct as it was a month or so ago. I'm not sure if this may be the cause, but I was cleaning up GA about a month ago, deleting some filters I didn't think I needed any more. Thank you for your help. 9CRUx
Reporting & Analytics | | sanctuary2420 -
How often does the indexing take place?
Hi there, I am currently looking at one of my campaigns and have noticed that since i last checked on the 31st Jan there have been no change in analytics despite me having done work on the campaign itself. I saw in the external links section that it said "38 down since last index" and I was just wondering how often the index takes place. Any help with this would be greatly appreciated! Many thanks.
Reporting & Analytics | | VoodooCreativeLtd0 -
Google Analytics reporting traffic for 404 pages
Hi guys, Unique issue with google analytics reporting for one of our sites. GA is reporting sessions for 404 pages (landing pages, organic traffic) e.g. for this page: http://www.milkandlove.com.au/breastfeeding-dresses/index.php the page is currently a 404 page but GA (see screenshot) is reporting organic traffic (to the landing page). Does anyone know any reasons why this is happening? Cheers. http://www.milkandlove.com.au/breastfeeding-dresses/index.php GK0zDzj.jpg
Reporting & Analytics | | jayoliverwright2 -
Question about setting up Google Webmaster on Network Solutions?
I'm trying to set myself up as a Webmaster on my company's site. We use Network Solutions. I am following Google's directions on how to do this. However, I am a bit wary. Here are the directions. Underlined is the part I am having trouble with: Log in to your account for howlatthemoon.com at www.networksolutions.com by clicking theManage Account icon. In the left navigation bar, open the nsWebAddress (Domains) menu by clicking the **+ **icon. Click Manage Domain Names. On the Domain Details page for the domain you're using, select the Designated DNS radio button (to the right of Change domain to point to) and click the Apply Changes button. If you've previously modified your advanced DNS settings, click Edit (to the right ofDomain currently points to). Under the Advanced DNS Manager heading, click Manage Advanced DNS Records. Under the Text (TXT Records) heading, click Add/Edit. In the Host field, enter @. Leave the TTL field set to the default value. In the Text field, copy and paste the following unique security token:
Reporting & Analytics | | howlusa
(security token removed for obvious reasons) Click Continue. Review your changes and click Save Changes. When you've done saving the TXT record, click the Verify button below on this page. There is already a host of @ (None). The text for it reads: v=spf1 include:_spf.google.com ~all I called Network Solutions and the guy I was speaking with told me to delete it and replace it with my Google Webmasters code. However, I think this is setting up our email. Do I just add the Webmasters in and have two hosts of @ (None)? Thanks!0 -
Google Analytics
Good Morning, I am trying to understand 2 issues in Google Analytics. 1. When look at : Traffic Source --> SEO --> Quesrios - i see the impressions column and its always a whole number 1550, 500, 5500, etc.. I never saw (for example) 702, 313, etc... impressions Can anyone explain why and how does it work? 2. In the same report i see my AVG. position for each query, the question is how come i have AVG, position of (290, 230, 190) for some of the queries and still i get clicks on these queries. My guess is that from time to time these queries have better position and the clicks are from these time. Do you familiar with a way to the the distribution of a specific query over time? for example: 1.3.2013 avg position = 4 2.3.2013 avg position = 7 3.3.2013 avg position = 2 4.3.2013 avg position = 8 etc... 3. This report say its for: "Top 1,000 daily queries" - What does it mean? Thank you and sorry for this long question SEOwiseUs
Reporting & Analytics | | iivgi0 -
View entrance keyworsd for a subpage in google analytics
I have an element from ringcentral that is tracked by google analytics, yet for the life of me I cannot figure out to how to view the entrance keywords to that element. I can view the search metric and see which keywords and how many visits from each keyword onto the site in general, but how do I see which keywords were actually responsible for this 'click to call' code which has GA? I can see the total number of clicks into that code, but not which keywords led to it.
Reporting & Analytics | | ilyaelbert0 -
Google Analytics - multiple counters
Hey there Mozzers! One of our customers wants to seperate one Google Analytics account into multiple accounts. The website is divided in three parts: Main: www.website.nl Sub1: www.website.nl/sub1 Sub2: http://www.website.nl/sub2 And they would like 4 different reports under one account. R1: Total count R2: Website.nl (without Sub 1 & Sub2) R3: Sub1 R4: Sub2 I know multiple counters will get in conflict with each other, so I have to implement some filters. E.g: We can configure a filter for R3 on "astmakids" in URL. My question is: is there a safe way to implement multiple Analytics filters on one website? And how will R3 see visitors that come from the root domain astmafonds.nl? Are they referrals? Thanks a lot in advance!! Partouter
Reporting & Analytics | | Partouter0 -
Google Analytics Goal Funnel Visualization Issue
I've setup a goal funnel but am having an issue when I look at the funnel visualization. It doesn't appear to be recognizing the 1st step of the funnel that I've defined in the goal edit page. The "Property Listing page view" is located at /listings/xxx where xxx is the number of the property. Within the funnel, I've added /listings/*, but when I go to see the funnel visualization, I see 0 counts for this step (even though it clearly shows on the entrance page to the left "/listings/622, etc". I've attached a .pdf with a few images to help make this clearer. Any thoughts? CRD-Funnel.pdf
Reporting & Analytics | | chrisfree0