Only 11 pages being crawled
-
Hi,
Can some one have a look and see why out of 400+ pages we only have 11 being crawled on here??
http://www.lifetimelegal.co.uk
Kind Regards
Elissa
-
When I search your site in Google it shows me 82 results against my query “site:domain.com” so it seems like Google is slowly crawling your site but you are right the rate of crawling pages is quite slow and to increase this here are few things that you should follow
- Strong Internal Linking
It is important to have a strong internal linking between the pages if you want Google and other search engine to crawl your page. Please check if you have a strong and powerful internal linking between the pages
- Xml Sitemap
The other thing that is missing on your site is xml sitemap. The idea is to add an xml sitemap and add it in Google webmaster tool and this will indicate Google to crawl different areas of your website with no hassle
Hope this will help!
-
Just looking around your site, the code seems ok - to a large extent.
But you are missing a sitemap (handy to point to all urls) and a robots.txt file
Also just clicking around i don't see 400 pages? is there a subdomain or folder you aren't linking to from the main site?
-
Hi Elisa,
It looks like it's pretty simple, when I crawl your Web site with Screaming Frog I only found 11 pages too. The reason is that the other pages aren't linked from the 11 pages found during a crawl. I would recommend to link to your other pages as well so these pages are found.
Hope this helps!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Help Me With These Horrible Crawl Errors
I am currently using Opencart to run a fairly popular ecommerce website. I have done quite well in terms of rankings so far. However, the Opencart platform only goes so far. There are endless amounts of crawl errors, from duplicate content down to missing meta tags. There is no easy way to add canonical tags. The pages that are being listed in the crawl errors seem to be mainly search URL's that i have no idea how Moz found them. An example of the page is: .....com/index.php?route=product/search&tag=home+office+flooring Which has ten duplicate pages all missing meta tags. I have used Google webmaster tools and htaccess to deny access to these pages with the slug "route=product/search" but it doesn't seem to work. So my question to you guys is as follows: Is it worth trying to fix these errors and do they have an effect on SEO? If so, how do I prevent these errors that seem to grow at every crawl? Cheers, Danny
Search Behavior | | DannyHoodless0 -
Location Pages
Hi. A client of mine offers multiple services and covers a region of the UK. They want to target each major town/city within this area. However, there are 20 cities and services offered range from 5-15 services. I am in the process of creating a location page for each city, so it can be optimised separately however I am not sure if there is a better way to do it? Or should I create a page for each city & service. So I for example I end up with 10 London pages with each one offering a different service? These can all be optimised for different services within London then? Any suggestions? Thanks
Search Behavior | | YNWA0 -
Is putting/removing Adsense ads on the site affects crawling for seo?
Because I noticed when we totally remove Adsense ads on our site, the pages crawled per day on the google webmastertools suddenly dropped into noticable amount and we tested again to turn it on for a singe day, it jumped up again. So if Adsense affects crawling then, if we have adsense ad on all pages of the site the more chances it gets crawled by the bot? WS76R8w
Search Behavior | | CruiseControl0 -
Does Page Load Time Affect SEO Rankings?
I was curious about how much page load times affect rankings. Here's what I did: I put together a lot of interactive media on specific landing pages Time-on-Page from organic visitors went from 50 seconds to average of 34 minutes Bounce Rate decreased by 20% Page Load time increased from 1 second to 6 seconds and at peak times to 8 seconds (on 56KB test) In the meantime the page was re-indexed and re-cached My question is three-fold: Would the time on page give higher rankings for keyword Would decreased bounce rate enhance rankings? Would the page load time decrease rankings? Did anyone do a similar test? What were the results?
Search Behavior | | HMCOE0 -
Irratic page rank results - why?
Hi, Hi- I've changed the title and reposted this question as I didn't get any insight. I really am going around in circles for this; so any help that can be given would be really appreciated.I know that you guys have the knowledge so really hoping you can share it! Thanks - background: I run two sites: www.ragsmetals.com and www.yourgoldfund.com: They offer similar products but to different markets: one is predominently trade, one predominently customer focused. They vie for the same keywords.e.g. Buy Gold My Issue: I have been trying to build www.yourgoldfund.com up from an SEO point of view; it is FALLING in the rankings; I am doing NOTHING with www.ragsmetals.com and it is rising in the rankings. I checked webmaster to see if there were any warnings as a result of penguin (found out how to do this on here - thanks for that!) there are none. I have optimized the pages a lot. I have put time into getting in bound traffic, and even more excitingly some experts (from reputable domains) have just linked to our site because they like our content! So to me things should be improving . NOTE: My OSE listed inbound links are incredibly diferent from google Webmaster. What can I do. I am a scientific person so I love the fact that I have 2 sites to compare...interested to hear your thoughts. Many thanks in advance
Search Behavior | | girlie0 -
Dating Blog Posts & How Fast Google Picks up on New Pages
I had until a few months ago included the original post date of a new blog post on the site. I then removed it and none of my results in Google now include the blog post date, although for some (for articles written about events) Google includes the date of the event where you would usually see the post date. Since I did this, it seems like new blog posts are taking longer to rank on Google, some results are ranking well, and others declined relative to what I would have previously expected. What's the best thing to be doing? To include a date (considering a lot of my content is not time-relevant) or to keep it as it is now? The second thing, is I often go through and update my articles with new information and re-post it in my rss feed etc - ie the date becomes new again. How does Google treat this? Any ideas or comments would be great! Thanks
Search Behavior | | ben10001 -
Hi guys.. post-penguin my website coming in and out of serps every other day... what reason for that? ie. #11 --> #300+
We had one of the pre-penguin "unnatural links " messages in WMT - then Penguin hit April 24th/25th.. wee then set to work on Link profile... since 19th May we have had numerous search phrases come 'back' into rank pre-penguin for a day ie.. #5 - #10 and then drop the next.. ie 300+ this has been happening across a wide range of key phrases.... in fact it may well be that the key phrases are added to the serps 'briefly' or for a set amount of time... but our ranking checker checks daily so while it appears to be every other day in and out there may be another pattern... but the question is What is Google doing here and Why? any suggestions?
Search Behavior | | Geminineil1 -
Forced Page Views and Search Engines?
I have a website that was built for the primary purpose of showing HTML 5 capabilities. With this, we have to create forced page views within analytics in order to receive any data about consumer behavior on the site. Are search engines viewing these forced page views as actual webpages? Does it even effect SEO efforts?
Search Behavior | | HughesDigital0