My wepgages aren't crawled by google
-
Most of my webpages aren't crawled by google.
Why is that and what can i do to make google index at least most of my webpages? -
Well, Google does have a crawl budget, they might be using that for your most popular pages. As long as your indexed pages number is going up, that means google is working its way through the backlog.
-
My website is a yellow pages site from Greece www.vreite.gr
It has registered more than 175.000 businesses and every business has 6 profile pages(main page,product page,feed page etc.)
Many visitors engage with these pages and are absolutely dunamic pages.
Is that a problem? -
The only site I can think of that would legitimately have 400,000 pages is Amazon.com. Google probably thinks your site is full of a ton of low quality content. Why in the world do you have that many pages? Are they low quality garbage? Do any visitors actually engage this even a fraction of them?
-
Hi
Yes my site is crawlable.
I checked if robots.txt or noindex tags and canonical urls and everything is fine.
Maybe is it because my website has over 400.000 pages -
Hi
You didn't answer the first part of Zoe's question - are you sure that your site is crawlable and that there are no issues with the robots.txt / noindex tags, ip detection systems, canonicals on all pages pointing to the home and so on. It's not because you can see all pages of your site in a browser that they are accessible/crawlable/indexable by google.
Try a crawl with Screaming Frog and user agent Googlebot to see if your pages can be crawled and indexed.
Backlinks are needed to have your site ranked for keywords - but it's not a prerequisite to have your site crawled. (noticed that a few times when a dev site was indexed by accident)
Without the actual url it's impossible to give a more detailed answer.
Dirk
-
Hi,
Backlinks certainly help, if there's no links at all to your site that could be a reason, but it's hard to say without looking deeper.
Are your internal pages all linked to each other? Does your website have a structured navigation system? This is also really necessary to ensure Google will index your whole site, not just a couple of pages.
Zoe
-
Hi
I added my website to Google Webmaster Tools and i checked my website and i don't have any crawling issues.
I added my website to Dmoz but the backling didn't appear yet.My site is live for about a year and google doesn't crawl most of my webpages yet.
Is it because i don't have quality backlinks?Thank you
-
Hi,
Firstly I'd check that Google can index your website. Have you added your site to Google Webmaster Tools? I'd start there and check for any crawl issues, especially your robots.txt file and any no-indexing of pages.
Secondly, if your website is brand new, I'd add your website to Dmoz & some relevant good quality sites like Yelp, Yell, Yellowpages, Google Plus (where relevant). Make sure the details you add to each match exactly with the details on your website. It will take some time for your site to appear in Google's index- sometimes a few days, sometimes a week or so- you can check by typing site:yourdomainname.com into a Google search to find the pages.
If your website is not new & has been indexed by Google before, I'd investigate whether you have a penalty. This post on penalties from white.net is really useful!
Hope this helps,
Zoe
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Why can't google mobile friendly test access my website?
getting the following error when trying to use google mobile friendly tool: "page cannot be reached. This could be because the page is unavailable or blocked by robots.txt" I don't have anything blocked by robots.txt or robots tag. i also manage to render my pages on google search console's fetch and render....so what can be the reason that the tool can't access my website? Also...the mobile usability report on the search console works but reports very little, and the google speed test also doesnt work... Any ideas to what is the reason and how to fix this? LEARN MOREDetailsUser agentGooglebot smartphone
Technical SEO | | Nadav_W0 -
I can't crawl the archive of this website with Screaming Frog
Hi I'm trying to crawl this website (http://zeri.info/) with Screaming Frog but because of some technical issue with their site (i can't find what is causing it) i'm able to crawl only the first page of each category (ex. http://zeri.info/sport/) and then it will go to crawl each page of their archive (hundreds of thousands of pages) but it won't crawl the links inside these pages. Thanks a lot!
Technical SEO | | gjergjshala0 -
Site's IP showing WMT 'Links to My Site'
I have been going through, disavowing spam links in WMT and one of my biggest referral sources is our own IP address. Site: Covers.com
Technical SEO | | evansluke
IP: 208.68.0.72 We have recently fixed a number of 302 redirects, but the number of links actually seems to be increasing. Is this something I should ignore / disavow / fix using a redirect?0 -
Why isn't my homepage number #1 when searching my brand name?
Hi! So we recently (a month ago) lunched a new website, we have great content that updates everyday, we're active on social platforms, and we did all that's possible, at the moment, when it comes to on site optimization (a web developer will join our team this month and help us fix all the rest). When I search for our brand name all our social profiles come up first, after them we have a few inner pages from our different news sections, but our homepage is somewhere in the 2nd search page... What may be the reason for that? Is it just a matter of time or is there a problem with our homepage I'm unable to find? Thanks!
Technical SEO | | Orly-PP0 -
How Many Words To Make Content 'unique?'
Hi All, I'm currently working on creating a variety of new pages for my website. These pages are based upon different keyword searches for cars, for example used BMW in London, Used BMW in Edinburgh and many many more similar kinds of variations. I'm writing some content for each page so that they're completely unique to each other (the cars displayed on each page will also be different so this would not be duplicated either). My question is really, how much content do you think that I'll need on each page? or what is optimal? What would be the minimum you might need? Thank for your help!
Technical SEO | | Sandicliffe0 -
Google Published Date - Does Google Lie?
Here's the scenario. I create a page called "ABC" and it gets published and found by Google lets say on the 13th of April. on the 15th (or 14th) i decide to update the URL, page Title, and content. (Redirect old URL to new URL as well) Will Google still show this page as being published on the 13th? or would it update the publish date according to the new URL? Greg | | | | | | <a id="question_reply-to-question-36769-description_codeblock" class="mceButton mceButtonEnabled mce_codeblock" style="color: #000000; border: 1px solid #f0f0ee; margin: 0px 1px 0px 0px; padding: 0px; background-color: transparent; cursor: default; vertical-align: baseline; width: 20px; border-collapse: separate; display: block; height: 20px;" title="Create Code Block" tabindex="-1"></a>Create Code Block | | | | | | | | | | | | | | |
Technical SEO | | AndreVanKets0 -
Walking into a site I didn't build, easy way to fix this # indexing problem?
I recently joined a team with a site without a) Great content b) Not much of any search traffic I looked and all their url's are built in this way: Normal looking link -> not actually a new page but # like: /#content-title And it has no h1 tag. Page doesn't refresh. My initial thought is to gut the site and build it in wordpress, but first have to ask, is there a way to make a site with /#/ content loading friendly to search engines?
Technical SEO | | andrewhyde0 -
Just relaunched a website - why did it fall in Google's SERPs?
I work for a marketing agency that just redesigned, rewrote and relaunched a client's website. They used to rank #4 on Google for the company's name (which is a fairly common one, for what it's worth). Now they're at #10 and want to know why. I'd like to explain to them what happened but don't know myself. Can someone explain it to me? And can I tell them if/when their ranking might go back up? In case this matters, I can tell you that it looks like Google hasn't yet crawled the new site. Anyway, thanks in advance for any help you can provide.
Technical SEO | | matt-145670