My wepgages aren't crawled by google
-
Most of my webpages aren't crawled by google.
Why is that and what can i do to make google index at least most of my webpages? -
Well, Google does have a crawl budget, they might be using that for your most popular pages. As long as your indexed pages number is going up, that means google is working its way through the backlog.
-
My website is a yellow pages site from Greece www.vreite.gr
It has registered more than 175.000 businesses and every business has 6 profile pages(main page,product page,feed page etc.)
Many visitors engage with these pages and are absolutely dunamic pages.
Is that a problem? -
The only site I can think of that would legitimately have 400,000 pages is Amazon.com. Google probably thinks your site is full of a ton of low quality content. Why in the world do you have that many pages? Are they low quality garbage? Do any visitors actually engage this even a fraction of them?
-
Hi
Yes my site is crawlable.
I checked if robots.txt or noindex tags and canonical urls and everything is fine.
Maybe is it because my website has over 400.000 pages -
Hi
You didn't answer the first part of Zoe's question - are you sure that your site is crawlable and that there are no issues with the robots.txt / noindex tags, ip detection systems, canonicals on all pages pointing to the home and so on. It's not because you can see all pages of your site in a browser that they are accessible/crawlable/indexable by google.
Try a crawl with Screaming Frog and user agent Googlebot to see if your pages can be crawled and indexed.
Backlinks are needed to have your site ranked for keywords - but it's not a prerequisite to have your site crawled. (noticed that a few times when a dev site was indexed by accident)
Without the actual url it's impossible to give a more detailed answer.
Dirk
-
Hi,
Backlinks certainly help, if there's no links at all to your site that could be a reason, but it's hard to say without looking deeper.
Are your internal pages all linked to each other? Does your website have a structured navigation system? This is also really necessary to ensure Google will index your whole site, not just a couple of pages.
Zoe
-
Hi
I added my website to Google Webmaster Tools and i checked my website and i don't have any crawling issues.
I added my website to Dmoz but the backling didn't appear yet.My site is live for about a year and google doesn't crawl most of my webpages yet.
Is it because i don't have quality backlinks?Thank you
-
Hi,
Firstly I'd check that Google can index your website. Have you added your site to Google Webmaster Tools? I'd start there and check for any crawl issues, especially your robots.txt file and any no-indexing of pages.
Secondly, if your website is brand new, I'd add your website to Dmoz & some relevant good quality sites like Yelp, Yell, Yellowpages, Google Plus (where relevant). Make sure the details you add to each match exactly with the details on your website. It will take some time for your site to appear in Google's index- sometimes a few days, sometimes a week or so- you can check by typing site:yourdomainname.com into a Google search to find the pages.
If your website is not new & has been indexed by Google before, I'd investigate whether you have a penalty. This post on penalties from white.net is really useful!
Hope this helps,
Zoe
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google Search Console 'Change of Address' Just 301s on source domain?
Hi all. New here, so please be gentle. 🙂 I've developed a new site, where my client also wanted to rebrand from .co.nz to .nz On the source (co.nz) domain, I've setup a load of 301 redirects to the relevant new page on the new domain (the URL structure is changing as well).
Technical SEO | | WebGuyNZ
E.G. On the old domain: https://www.mysite.co.nz/myonlinestore/t-shirt.html
In the HTACCESS on the old/source domain, I've setup 301's (using RewriteRule).
So that when **https://www.mysite.co.nz/**myonlinestore/t-shirt.html is accessed, it does a 301 to;
https://mysite.nz/shop/clothes/t-shirt All these 301's are working fine. I've checked in dev tools and a 301 is being returned. My question is, is having the 301's just on the source domain only enough, in regards to starting a 'Change of Address' in Google's Search Console? Their wording indicates it's enough but I'm concerned, maybe I also need redirects on the target domain as well? I.E. Does the Search Console Change of Address process work this way?
It looks at the source domain URL (that's already in Google's index), sees the 301 then updates the index (and hopefully pass the link juice) to the new URL. Also, I've setup both source and target Search Console properties as Domain Properties. Does that mean I no longer need to specify that the source and target properties are HTTP or HTTPS? I couldn't see that option when I created the properties. Thanks!0 -
Redirects and site map isn't showing
We had a malware hack and spent 3 days trying to get Bluehost to fix things. Since they have made changes 2 things are happening: 1. Our .xml sitemap cannot be created https://www.caffeinemarketing.co.uk/sitmap.xml we have tried external tools 2. We had 301 redirects from the http (www and non www versions) nad the https;// (non www version) throughout the whole website to https://www.caffeinemarketing.co.uk/ and subsequent pages Whilst the redirects seem to be happening, when you go into the tools such as https://httpstatus.io every version of every page is a 200 code only whereas before ther were showing the 301 redirects Have Bluehost messed things up? Hope you can help thanks
Technical SEO | | Caffeine_Marketing0 -
Some bots excluded from crawling client's domain
Hi all! My client is in healthcare in the US and for HIPAA reasons, blocks traffic from most international sources. a. I don't think this is good for SEO b. The site won't allow Moz bot or Screaming Frog bot to crawl it. It's so frustrating. We can't figure out what mechanism they are utilizing to execute this. Any help as we start down the rabbit hole to remedy is much appreciated. thank you!
Technical SEO | | SimpleSearch0 -
My Website's Home Page is Missing on Google SERP
Hi All, I have a WordPress website which has about 10-12 pages in total. When I search for the brand name on Google Search, the home page URL isn't appearing on the result pages while the rest of the pages are appearing. There're no issues with the canonicalization or meta titles/descriptions as such. What could possibly the reason behind this aberration? Looking forward to your advice! Cheers
Technical SEO | | ugorayan0 -
How Many Words To Make Content 'unique?'
Hi All, I'm currently working on creating a variety of new pages for my website. These pages are based upon different keyword searches for cars, for example used BMW in London, Used BMW in Edinburgh and many many more similar kinds of variations. I'm writing some content for each page so that they're completely unique to each other (the cars displayed on each page will also be different so this would not be duplicated either). My question is really, how much content do you think that I'll need on each page? or what is optimal? What would be the minimum you might need? Thank for your help!
Technical SEO | | Sandicliffe0 -
Why doesn't SEOmoz see internal/external links on my site?
My SEOmoz analysis that my site contains neither external or internal lnks. I have used other tools and they have all seen the internal and external links on the pages. There aren't many but they are there. Why isn't SEOmoz seeing them?
Technical SEO | | iain0 -
Negative effect on google SEO with 301's?
Cleaning up the website by consolidating pages - each with a little bit of useful info - into one definitive page that is really useful and full of good content. Doing 301's from the many old pages to the one new really good one. Didn't want to do rel canonicals because I don't want the old pages around, I want to get rid of them. Will google see the 301s and go nuts or see that there is one definitive, really good page with no duplicate content? The change is very good from a user perspective. Also, On-Page Report Cards on SEOMoz suggests that you put a rel canonical on a page to itself to tell google that this page is the definitive page. What do you think? Thanks so much for anyone who has time to answer - so many gurus - this is a great forum. - jean
Technical SEO | | JeanYates0 -
Cantags within links affect Google's perception of them?
Hi, All! This might be really obvious, but I have little coding experience, so when in doubt - ask... One of our client site's has navigation that looks (in part) like this: <a <span="">href</a><a <span="">="http://www.mysite.com/section1"></a> <a <span="">src="images/arrow6.gif" width="13" height="7" alt="Section 1">Section 1</a><a <span=""></a> WC3 told us the tags invalidate, and while I ignored most of their comments because I didn't think it would impact on what search engines saw, because thesetags are right in the links, it raised a question. Anyone know if this is for sure a problem/not a problem? Thanks in advance! Aviva B
Technical SEO | | debi_zyx0