Value of having a good crawl budget?
-
Hi,
I've seen several questions where people give advice on how to increase the crawl budget. What I haven't seen anyone comment is what the value of this really is if you have many pages that doesn't get updated very often.
Take for example the typical agency page - 50 pages, most of them rarely gets updates. In a monthly basis normally 10% of the website gets updated.
Is there really any value then of having 100% of the website crawled on a daily basis?
-
Check out this awesome post by AJ Kohn
To answer your question...
"What I’ve observed over the last few years is that pages that haven’t been crawled recently are given less authority in the index. To be more blunt, if a page hasn’t been crawled recently, it won’t rank well."
"The pages that aren’t crawled as often are pages with little to no PageRank. CrawlRank is the difference in this very large pool of pages. You win if you get your low PageRank pages crawled more frequently than the competition."
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Is it a good idea to direct a new url to a subfolder?
Hello everyone, I have a client who is in public relations. He is wanting to integrate SEO into his business and is hiring me to help. He purchased a domain that is separate from his existing website, but wants to keep the services and existing blog (which will include SEO) on his current site. For example, he has: www.example.com and wants to add a SEO folder which will contain our services: www.example.com/prseo the domain he purchased (as another example) is: www.prseo.com but wants to direct it to the folder he has set up @ example.com/prseo Can anyone offer advise? Is this a good idea?
Technical SEO | | visabelmedia0 -
Is Tag Manager a good option to insert text in websites?
Is Tag Manager a good option to insert text in websites? When a website doesn't have an administration panel adding text is a very big problem.
Technical SEO | | propertyshark0 -
The W3C Markup Validation Service - Good, Bad or Impartial?
Hi guys, it seems that now days it is almost impossible to achieve 0 (Zero) Errors when testing a site via (The W3C Markup Validation Service - https://validator.w3.org). With analytic codes, pixels and all kind of tracking and social media scripts gunning it seems to be an unachievable task. My questions to you fellow SEO'rs out there are 2: 1. How important and to what degree of effort do you go when you technically review a site and make the decision as to what needs to be fixed and what you shouldn't bother with. 2. How do you argue your corner when explaining to your clients that its impossible to active 100% validation. *As a note i will say that i mostly refer to Wordpress driven sites. would love ot hear your take. Daniel.
Technical SEO | | artdivision0 -
IBM Websphere Commerce - Good Platform Or Not?
I have recently started working with a fairly large company that is rethinking their web paltform. Currently they use IBM Websphere Commerce, but I'm finding a lot of limitations with it as far as SEO, content managment etc., it seems that the simplest things take a lot of resources and time. On top of that, IT has had developed other modules that integrate with it to make it do what it needs to do, so we are using multiple systems. The question we are evaluating is if Websphere is capable of providing what we need and just needs developed and implemented, or is in a platform that is not friendly for our needs? It may be easy to say to scrap it and move on, but everyone knows how difficult it is to change platforms, especially for a multi million dollar company that is doing a lot of business. So if the current system is capable, we want to make use of it, but we're not sure if it is even capable. Has anyone had direct experience with this platform?
Technical SEO | | rock220 -
How does Google Crawl Multi-Regional Sites?
I've been reading up on this on Webmaster Tools but just wanted to see if anyone could explain it a bit better. I have a website which is going live soon which is going to be set up to redirect to a localised URL based on the IP address i.e. NZ IP ranges will go to .co.nz, Aus IP addresses would go to .com.au and then USA or other non-specified IP addresses will go to the .com address. There is a single CMS installation for the website. Does this impact the way in which Google is able to search the site? Will all domains be crawled or just one? Any help would be great - thanks!
Technical SEO | | lemonz0 -
Do 301 redirects now allow most of the bad value to pass through?
I heard after the 3.2 update that most of your bad history passes though the 301 redirect.. What do you guys think out there?
Technical SEO | | Merta19801 -
Does CSS position effect the value of a link?
For example, take a look at http://www.dueds.com and scroll all the way to the bottom of the page. See the link in the bottom left? Does the fact that it is pushed all the way down to the bottom make the link worth less than if it was directly under the social media buttons?
Technical SEO | | adriandg0 -
What loss of value would this link experience?
What loss of value would this link experience? If the link is actually a link to the from site that is 301'd to your site like this example below: i.e., www.domain.com/29834?=www.yourdomain.com My thought is that simply because you're going through a redirect (In this case a 301) you will lose slight value there. But I'd love to hear your thoughts and reasoning on any other affects if any (direct or indirect) you think it may have.
Technical SEO | | Webfor1