Increase number of pages crawled
-
Only one page is being crawled, how do I increase the number to include most of our site?
-
Hi! I am guessing that you're referring to SEOmoz only crawling one page of your site? The list of possible reasons for that is at https://seomoz.zendesk.com/entries/409821-why-isn-t-my-site-being-crawled-you-re-not-crawling-all-my-pages. If none of those are applicable, go to the help hub at http://www.seomoz.org/help and contact our help team and we'll work on figuring it out with you.
-
make sure you are not unintentionally blocking pages in your robots.txt or meta robots tags and that the links on your site are search engine friendly links that the crawlers can follow.
submit the URL(s) to be crawled in google webmaster tools to tell the bots to come around again.
increase the crawlrate manually in WMT
-
Hi Lisa,
OSE will crawl enough pages of your site that are found/relevant at the time of the last crawl.
OSE is not up to date at any one time.
This FAQ shows that some data may be delayed up to 60 days, however in my experience listing new websites, this has been more like a couple of weeks.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How important is Lighthouse page speed measurement?
Hi, Many experts cite the Lighthouse speed as an important factor for search ranking. It's confusing because several top sites have Lighthouse speed of 30-40, yet they rank well. Also, some sites that load quickly have a low Lighthouse speed score (when I test on mobile/desktop they load much quicker than stated by Lighthouse). When we look at other image rich sites (such as Airbnb, John Deere etc) the Lighthouse score can be 30-40. Our site https://www.equipmentradar.com/ loads quickly on Desktop and Mobile, but the Lighthouse score is similar to Airbnb and so forth. We have many photos similar to photo below, probably 30-40, many of which load async. Should we spend more time optimizing Lighthouse or is it ok? Are large images fine to load async? Thank you, Dave bg_05.jpg
Reporting & Analytics | | erdev0 -
UTM Links Showing Up as Separate Pages in Google Analytics
Hey everyone, I was just looking at landing pages in Google Analytics, and in addition to just the URL of the landing page, the UTM links are being listed as separate pages. Is this normal? I anticipated seeing the landing page URL and then using the secondary dimension to see source/medium. If this isn't normal, what would I check next?
Reporting & Analytics | | rachelmeyer0 -
Is there a way to map your on-page SEO changes with the organic growth?
Hi Mozzers, I was just wondering if there's a way we can map our on-page SEO changes with the increase/decrease in organic traffic. For instance, I introduced brand pages' link the product page breadcrumbs and suddenly organic traffic for my brand pages increase from X to 2X in 1 couple of weeks. Now, this can be because of this breadcrumb change purely or because of some algorithm update or may be, bots started finding the content interesting and hence, started ranking them up (in case the brand pages were launched recently). So, you can't say which change should be mapped to what increase/decrease in organic traffic. Or, is there a way to map this?
Reporting & Analytics | | _nitman0 -
Site re-crawled?
I've fixed many of my errors, but they're still showing in my dashboard. When will the site be crawled again?
Reporting & Analytics | | sakeith0 -
Free Online XMl Site Map Creators up sites over 1000 pages
Does anybody know of a XML site map creator that is free for websites over 1000 pages?
Reporting & Analytics | | AppleCapitalGroup0 -
Why is the page title sometimes missing in the API results?
Why is the page title sometimes missing in the API results? It exists on the page and in the "keyword difficulty tool", but missing in the API. It is never returned when upa= 1For example:stdClass Object(
Reporting & Analytics | | kirza
[fmrp] => 7.05087881216
[fmrr] => 5.05983022077E-6
[pda] => 91.758307504
[ueid] => 0
[ufq] => www.yellowpages.com/
[uid] => 0
[umrp] => 0
[umrr] => 0
[upa] => 1
[upl] => yellowpages.com/
[us] => 0
[ut] =>
[uu] =>
) {"fmrp":5.679443875178763,"fmrr":2.052551993215746e-07,"pda":70.05750830306422,"ueid":0,"ufq":"education-portal.com/","uid":0,"umrp":0,"umrr":0,"upa":1,"upl":"education-portal.com/","us":0,"ut":"","uu":""}How to get the titles for these pages?0 -
Site crawler hasn't crawled my site in 6 days!
On 4.23 i requested a site crawl. My site only has about 550 pages. So how can we get faster crawls?
Reporting & Analytics | | joemas990 -
Never Crawled!
This page has not been crawled for five months: http://www.flowerpetal.com/index.jsp?info=13 1/2 that time it was linked to from the homepage of a pr5 site: http://flowerpetal.com/ Why is this the case?
Reporting & Analytics | | tylerfraser0