Increase number of pages crawled
-
Only one page is being crawled, how do I increase the number to include most of our site?
-
Hi! I am guessing that you're referring to SEOmoz only crawling one page of your site? The list of possible reasons for that is at https://seomoz.zendesk.com/entries/409821-why-isn-t-my-site-being-crawled-you-re-not-crawling-all-my-pages. If none of those are applicable, go to the help hub at http://www.seomoz.org/help and contact our help team and we'll work on figuring it out with you.
-
make sure you are not unintentionally blocking pages in your robots.txt or meta robots tags and that the links on your site are search engine friendly links that the crawlers can follow.
submit the URL(s) to be crawled in google webmaster tools to tell the bots to come around again.
increase the crawlrate manually in WMT
-
Hi Lisa,
OSE will crawl enough pages of your site that are found/relevant at the time of the last crawl.
OSE is not up to date at any one time.
This FAQ shows that some data may be delayed up to 60 days, however in my experience listing new websites, this has been more like a couple of weeks.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Any idea why this page is an absolute magnet for bots?
This page on our client's website seems to be an absolute magnet for bots, and it's skewing our Google Analytics stats: https://cbisonline.com/us/catholic-socially-responsible-esg-investing/proxy-voting/ We already filter out lots of bots in GA, primarily through a segment we created several years ago and continue to build upon, but plenty of spam traffic still manages to slip through – mostly to the page above. Last quarter, almost all of it came from two random cities in Europe, so we're going to filter out traffic from those places. (At least for now – not an ideal solution, I know.) But I'm really wondering what drives so many bots to that page in particular. Any insights would be greatly appreciated!
Reporting & Analytics | | matt-145670 -
Google Analytics Landing Page Report Discrepancy
I have noticed that when I run a landing page report and use the advanced option so I can view only the landing pages that include a particular string in the URL, have noticed that I in the report, the graph at the top will say one thing, but the data below says something else. For example, the graph for one particular search shows 200 Impressions, but the info below says 700 impressions and 610 clicks. Anyone seen anything similar or have any ideas why? Thanks! Craig
Reporting & Analytics | | TheCraig0 -
Is there an automated way to determine which pages of your website are getting 0 traffic?
I'm doing a content audit on my company website and want to identify pages with zero traffic. I can use GA for low traffic, but not zero traffic. I can do this manually, but it would take a long time. Are there any tools to help me determine these pages?
Reporting & Analytics | | Ksink0 -
What about this (google crawl)?
Recently we did a serious effort on SEO with SEO Yoast (Wordpress). And after a few months of tweaking old articles we get this impact on crawl search.. Is this graph normal? s1TQgv9.png
Reporting & Analytics | | noodweerbenelux0 -
Best way to handle duplicate title on Home page?
Moz reports two links to the same Home page ad duplicate titles ... http://myhjhome.com/index.php
Reporting & Analytics | | ElykInnovation
http://myhjhome.com I'm not sure if I should just 301 redirect http://myhjhome.com/index.php to http://myhjhome.com, or if there is a better way to handle that? Or should I comb the website and make sure all links to the Home page dont include index.php? Just looking for some extra help here, learning as I'm going, thanks!!0 -
How to get metrics on Home page image link to youtube video in lightbox
My home page has a explainer video. Is there a way to get metrics or analytics data about who watch or don't watch the video? http://www.furnacefilterscanada.com For example, if I have 100 unique visitors on this page and my video get 33 views, 33% saw the video! I think the data provide by youtube are not accurate, my site has a avarage of 40 visitors/day and my video was saw 84 times sinces it is online, 14 days ago. On that 84 clicks, I most have click 30 times!!! When you lend on my home page, you can't miss it! http://screencast.com/t/ZbIWYl0W I don't believe only a few visitors saw it. Conversation rate has increase. I use Google Analytics, is there a code or something that can be done to get metrics or data more accurate then the .Ananalytics" provide in my youtube control panel? Please HELP! thank you, BigBlaze
Reporting & Analytics | | BigBlaze2050 -
Page Rank - logarithmic or exponential
Possibly a really stupid question. Is Page Rank logarithmic or exponential? I've seen a lot of people talking about Page Rank saying it's logarithmic but when they describe it they're actually talking about an exponential scale. (Apologies if I'm showing a basic misunderstanding in mathematical knowledge - I studied Drama)
Reporting & Analytics | | BenFox0 -
Is 10 Keyword Targets for Page Rank too many?
My client has selected 10 keywords that they want to rank on the first page of Google for. Is 10 keywords too many to try and rank for? I have heard that you should focus on top 5 keywords instead.
Reporting & Analytics | | dseasterling0
