Increase number of pages crawled
-
Only one page is being crawled, how do I increase the number to include most of our site?
-
Hi! I am guessing that you're referring to SEOmoz only crawling one page of your site? The list of possible reasons for that is at https://seomoz.zendesk.com/entries/409821-why-isn-t-my-site-being-crawled-you-re-not-crawling-all-my-pages. If none of those are applicable, go to the help hub at http://www.seomoz.org/help and contact our help team and we'll work on figuring it out with you.
-
make sure you are not unintentionally blocking pages in your robots.txt or meta robots tags and that the links on your site are search engine friendly links that the crawlers can follow.
submit the URL(s) to be crawled in google webmaster tools to tell the bots to come around again.
increase the crawlrate manually in WMT
-
Hi Lisa,
OSE will crawl enough pages of your site that are found/relevant at the time of the last crawl.
OSE is not up to date at any one time.
This FAQ shows that some data may be delayed up to 60 days, however in my experience listing new websites, this has been more like a couple of weeks.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Any idea why this page is an absolute magnet for bots?
This page on our client's website seems to be an absolute magnet for bots, and it's skewing our Google Analytics stats: https://cbisonline.com/us/catholic-socially-responsible-esg-investing/proxy-voting/ We already filter out lots of bots in GA, primarily through a segment we created several years ago and continue to build upon, but plenty of spam traffic still manages to slip through – mostly to the page above. Last quarter, almost all of it came from two random cities in Europe, so we're going to filter out traffic from those places. (At least for now – not an ideal solution, I know.) But I'm really wondering what drives so many bots to that page in particular. Any insights would be greatly appreciated!
Reporting & Analytics | | matt-145670 -
Avg Page Load Time (sec) Comppared to site average - what does it mean?
Hi All, In google analytic In Site Speed -> Page Timings we have two columns a) Page Views & b) Avg Load Time (sec) compared to site average. Now in "b" column I am able to below % one in green and another in brown so what does it mean? Can anyone please explain me? Image attached Thanks! bNbBA
Reporting & Analytics | | amu1230 -
What is the most effective way of selecting a top keyword per page on a site?
We are creating fresh content for outdated sites and I need to identify the most significant keyword per page for the content developers, What is the best way to do this?
Reporting & Analytics | | Sable_Group0 -
How can you add a rel canonical tag if you haven't created the wrong pages?
For one of our white paper campaigns we are getting multiple URL's some how but we only have one version of the page. So do I put the rel canonical tag on that one single page? Will that fix the other url's from being indexed? I'm assuming people are typing in the urls with underscores and capital or non-capital letters and it's showing up that way in analytics. Thanks!
Reporting & Analytics | | Sika220 -
SEO Working GREAT! Although Keyword Rankings Only Directing to Home Page.
Since I've become a fellow MOZer I've seen an increase on my sites and proud to say that ALL of my clients have now accomplished 1st page rankings on at least 2 of their top 3 keywords. Yes, I am as stoked as them. Although I am a bit worried because as I rank keywords and test them on Google and engines alike all of our rankings direct to the homepage. I am essentially new to SEO. So as happy as I am to be doing something right I'm worried I'm losing the end user. By having them need to Search our site/menus/blog for the keywords/phrase that got them to us will turn most searchers away. They want the content now. What am I doing wrong? My pages are pretty specific so I never cared to use rel-canonical's...I dont have duplicate content, other than your typical footer/side bar consistent info. So I'm curious where I've gone wrong. Little history on my sites: I Use Joomla 2.5 & 3.x. I manage seo details through sh404seo. I've cleaned all duplicates I can up other than what these dynamic cms generate by just using one. I've done fairly well and all my pages rank A for moz's onpage grader for the keyword optimized around. Any help is appreciate, THANK YOU!
Reporting & Analytics | | Funk-Creative-Media0 -
Irrelevant page with high bounce rate
I have a page on my site, www.waikoloavacationrentals.com/kolea-rentals/floor-plans, that gets me roughly 17% of my traffic. That being said it is not really relevant traffic because it comes from the search term "floor plans", which really has nothing to do with Hawaii vacation rentals, which is what I do. My question is does Google know how to figure that out when they are looking at my stats or is there a way to let google know that that page probably should not show up for that search phrase? On the positive, they are nice floor plans and if someone is searching for ideas for floor plans and see one of them in google images then it probably could help them, but it really is not relevant to my business. It has a 80% bounce rate, but does have an average time on page of 1.5 minutes, which is a fair amount for what is there.
Reporting & Analytics | | RobDalton0 -
Google Analytics - In-Page Analytics
I had a strange thought waking up this morning, and was curious to hear other people's opinions on it. In Google Analytics, under Content > In-Page Analytics, Google shows what links on your site pages get clicked and how many times plus other metrics. Do you think they use that data for ranking back links so-to-speak? What I mean is, say I had a back link to my site on example.com, and example.com had google analytics installed. Google can see through google analytics whether my link has been clicked on. Say that my link gets no clicks, do you think that Google would use that metric against my site deeming it "not popular" or "not a good resource", even if example.com was a very popular site? And it could work the other way. Say my link got thousands of clicks on example.com, do you think that Google might use that to promote my site? I couldn't find any other discussion on this anywhere, so am not sure if people have already thought about this.
Reporting & Analytics | | THB0 -
SEOMoz crawls skewing Avg Visit Duration in Google Analytics
Hello, We are a UK based company. Our Google Analytics account is showing a rise in Avg Visit Duration for 'Direct traffic' since we started using SEOMoz. Are any other users experiencing this issue? We tracked it down to City Seattle, and when doing an Advanced filter by removing Seattle, the results are normal again. What does SEOMoz or other users recommend we do besides continuously using advanced filters? We have enquired about excluding Roger's IP address, but have been told that Roger uses the Amazon cloud, so the IP is not static. See attachment for screenshot of our Google Analytics account of Avg Visit Duration since we began with SEOMoz. Rich Talbot 718g3.gif
Reporting & Analytics | | STL1