Increase number of pages crawled
-
Only one page is being crawled, how do I increase the number to include most of our site?
-
Hi! I am guessing that you're referring to SEOmoz only crawling one page of your site? The list of possible reasons for that is at https://seomoz.zendesk.com/entries/409821-why-isn-t-my-site-being-crawled-you-re-not-crawling-all-my-pages. If none of those are applicable, go to the help hub at http://www.seomoz.org/help and contact our help team and we'll work on figuring it out with you.
-
make sure you are not unintentionally blocking pages in your robots.txt or meta robots tags and that the links on your site are search engine friendly links that the crawlers can follow.
submit the URL(s) to be crawled in google webmaster tools to tell the bots to come around again.
increase the crawlrate manually in WMT
-
Hi Lisa,
OSE will crawl enough pages of your site that are found/relevant at the time of the last crawl.
OSE is not up to date at any one time.
This FAQ shows that some data may be delayed up to 60 days, however in my experience listing new websites, this has been more like a couple of weeks.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
High Temporary Redirects: Login required pages
Noticed something interesting, a high temporary redirect report from Moz. Reviewing the pages they are caused by the user having to login and getting redirected. I can see the returnto query in the URL too. My thoughts: Since a login is required and the user is being redirected, these should remain 302 and not 301. I tested my Google Analytics account to **Exclude URL Query Parameter **returnto, just to see if it affected traffic. It didn't, I mean I don't see urls duplicated with the parameter anymore, just grouped together, so traffic is still being counted. I'm going to wait 1 more day and see what impact the GA traffic is before applying the exclusion to my true Google Analytics profile. This got me thinking, I should probably exclude this parameter from Google and Bing Webmaster Tools, that way Google/bing won't read those urls. Now does Moz's crawler follow that? Do you think that would change my moz crawl diagnostic report because I told Google/Bing crawlers to exclude that parameter. What do you think of my approach to reduce these high temporary redirects reported by Moz? Will it work? Has it plagued you?
Reporting & Analytics | | Bio-RadAbs0 -
Conversion Rate Higher Than Landing Page Visits?
Interesting to see in Google Analytics that the conversion rate is higher than landing page visits - could it be attributed to a visitor clicking the CTA button multiple times? Or perhaps there is duplicate GA code on the conversion page since we utilize both Google Analytics and HubSpot. (see attached funnel screenshot) Screen-Shot-2014-09-26-at-10.49.09-AM.png
Reporting & Analytics | | W210 -
Google Analytics is treating my blog like all the content is just on the home page.
Hello all, I installed Google Analytics on a main website and a blog (blog.travelexinsurance.com) While it appears to be tracking correctly (and when I test it in real time it shows that I'm visiting) but it is treating the entire blog as though it's one page. So I can't see data on blog post X. All I see is that X visitors came to my blog in aggregate. So I see blog.travelex.com has 999 visitors, but it doesn't show that /travel-luggage got 50 visits, while /insurace-tips got 75 and so forth. I assume I screwed up the tracking somehow, but can't figure out where I went wrong. Tracking on the main domain works just fine. It's specific to the blog.
Reporting & Analytics | | Patrick_G0 -
Why such a high page rank with so low metrics in OSE
Hi,
Reporting & Analytics | | sebastiankoch
The website is:
www.s123parka123ble.com (remove the 123) All websites I have ever seen with similar metrics in OSE have normally a PR of 2 or 3 or max 4. This one is PR 5 and I would like to know why. I also noticed that they do not redirect the urls without www. to www. Which is normally bad ... can it be good for some reason in this case? It is PR 5 since a long time I just can´t get why it is PR 5. Please have a look. Thanks!0 -
Page Rank - logarithmic or exponential
Possibly a really stupid question. Is Page Rank logarithmic or exponential? I've seen a lot of people talking about Page Rank saying it's logarithmic but when they describe it they're actually talking about an exponential scale. (Apologies if I'm showing a basic misunderstanding in mathematical knowledge - I studied Drama)
Reporting & Analytics | | BenFox0 -
Difference between page/domain authority
could anyone explain the difference between Page Authority and Domain Authoity to me or give me a link to a site where it is explained? Sorry if It's really obvious and I'm just too stupid to find out, but I've searched and haven't found anything.
Reporting & Analytics | | mtueckcr0 -
If I change the URL of a page, but the old page canonicalizes to the new, do I need to change my Analytics goals to get data?
I changed the URLs of some pages recently (because the same thing that affects the internal anchor text also affects the URL - grr...) but considered it not a big deal because even if I looked at the source code of the old URL, the canonical tag was now pointing to the new one. The question is - if I had URL destination goals set up for those URLs in Google Anlaytics, do I now have to change them? Or does Google somehow know that anyone getting to the new URL is the equivalent of someone getting to the old URL because of the canonical tag that exists on the old URL source code? I still do see goal conversions for some of the old URLs even since I changed them - but it could be that people are still somehow finding the old URL somewhere - or that Google only reindexed it a week or so after I made the change. Any light to shed? Thanks in advance, Aviva B
Reporting & Analytics | | debi_zyx0 -
Site crawler hasn't crawled my site in 6 days!
On 4.23 i requested a site crawl. My site only has about 550 pages. So how can we get faster crawls?
Reporting & Analytics | | joemas990