Increase number of pages crawled
-
Only one page is being crawled, how do I increase the number to include most of our site?
-
Hi! I am guessing that you're referring to SEOmoz only crawling one page of your site? The list of possible reasons for that is at https://seomoz.zendesk.com/entries/409821-why-isn-t-my-site-being-crawled-you-re-not-crawling-all-my-pages. If none of those are applicable, go to the help hub at http://www.seomoz.org/help and contact our help team and we'll work on figuring it out with you.
-
make sure you are not unintentionally blocking pages in your robots.txt or meta robots tags and that the links on your site are search engine friendly links that the crawlers can follow.
submit the URL(s) to be crawled in google webmaster tools to tell the bots to come around again.
increase the crawlrate manually in WMT
-
Hi Lisa,
OSE will crawl enough pages of your site that are found/relevant at the time of the last crawl.
OSE is not up to date at any one time.
This FAQ shows that some data may be delayed up to 60 days, however in my experience listing new websites, this has been more like a couple of weeks.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Page Tracking using Custom URLs - is this viable?
Hi Moz community! I’ll try to make this question as easy to understand as possible, but please excuse me if it isn’t clear. Just joined a new team a few months ago and found out that on some of our most popular pages we use “custom URLs” to track page metrics within Google Analytics. NOTE: I say “custom URLs” because that is the best way for me to describe them. As an example: This page exists to our users: http://usnews.rankingsandreviews.com/cars-trucks/Ram_HD/2012/photos-interior/ But this is the URL we have coded on the page: cars-trucks/used-cars/reviews/2012-Ram-HD/photos-interior/ (within the custom variance script labeled as “var l_tracker=” ) It is this custom URL that we use within GA to look up metrics about this page. This is just one example of many across our site setup to do the same thing Here is a second example: Available page to user: http://usnews.rankingsandreviews.com/cars-trucks/Cadillac_ATS/2015/ Custom “var l_tracker=” /cars-trucks/2015-Cadillac-ATS/overview/ NOTE: There is a small amount of fear that the above method was implemented years ago as a work-around to a poorly structured URL architecture. Not validated, but that is a question that arose. Main Questions: Is the above implementation a normal and often used method to track pages in GA? (coming from an Omniture company before – this would not be how we handled page level tracking) Team members at my current company are divided on this method. Some believe this is not a proper implementation and are concerned that trying to hide these from Google will raise red flags (i.e. fake URLs in general = bad) I cannot find any reference to this method anywhere on the InterWebs - If method is not normal: Any recommendations on a solution to address this? Potential Problems? GA is currently cataloguing these tracking URLs in the Crawl Error report. Any concerns about this? The team wants to hide the URLs in the Robots.txt file, but some team members are concerned this may raise a red flag with Google and hurt us more than help us. Thank you in advance for any insight and/or advice. Chris
Reporting & Analytics | | usnseomoz0 -
Apple.com showing in Google Analytics "My Top Active Pages" - Why?
Hi Mozzers This has more than likely been asked before but my searches have left me empty handed so I have asked on here ( sorry! ) Please see attached screenshot, do you have any idea why apple.com is showing in my top pages? I'm just curious more than anything as this isn't the first time I've noticed it. Anybody know? Happy Friday people, cheers Jamie CiTCA5b.png
Reporting & Analytics | | SanjidaKazi1 -
Why is Google Analytics reporting 20% fewer goals than Unique pageviews of same thank you page?
This is really puzzling me and my research has not thrown out the answer. I have always understood URL goals to be unique pageviews of the thank you page you are tracking. UPVs and goals should both only be counted once per session... Has anyone else seen this issue? Goals were not set up historically so I wanted to use unique pageviews of the thank you page for year on year comparisons, but 20% is a big difference! Background There are multiple pages to track so goal is set up using Regex There is no mistake in the goal set up (honest!) The goal URLs all match the unique pageview URLs, there are no rogue URLs There has been no change to the site or the tracking set up Data is not being sampled It's a lead gen site in an area where multiple enquiries within one visit would be very unusual Thanks in advance!
Reporting & Analytics | | McCannSEO0 -
Relation between Page and Landing Page
Hi all, I am looking through the Analytics data on a specific 'page' (404.html which is the Not found page) and as my secondary dimensions, I have Landing Page data. Now am I not sure how the page and the landing page are related here. Is it basically saying 58 Page views of the 404.html were originated by the users who landed on brochures.html? If someone could provide any pointers on it, it would be great. Thank you for your time. uZQJ
Reporting & Analytics | | nirpan0 -
Why am i getting a flux of increase in Impressions on my site & then it decreases
They guys. Hope everyone is having a great week. I wanted to get some inputs from you guys in regards to what is happening to my site that i quite don't understand. Every month or so i get this influx of high visibility with impressions for my keywords and then the impressions go away but my rankings still keep going up. Has anyone experienced this before and can give me some insight on what is going . Why do i get such a big jump and then it dies off only to return again a month later or 2 months later. I know you guys want probably some info from my site or from analytics or webmaster tools so i will provide as much as i can . For now i have included a screen shot. ScreenShot2013-06-04at31220PM_zps0d02f5fc.png ScreenShot2013-06-04at31134PM_zps5bb81b68.png ScreenShot2013-06-04at31134PM_zps5bb81b68.png ScreenShot2013-06-04at31220PM_zps0d02f5fc.png
Reporting & Analytics | | BizDetox0 -
Magic UVs - PPC landing pages delivering organic traffic by magic...
I have checked and double checked this. GA is showing over the last couple of weeks mysite.com/ppc/landingpage1 as a landing page for organic traffic, where it shouldn't. Main facts: The entire /ppc/ folder is blocked from the googlebot, and doesn't appear on any internal site maps. As far as I can tell, these pages have never been cached for the main index. I cannot recreate any of the organic searches myself (i.e. typing in keywords that triggered the traffic, even the almost unique long-tail ones). We just don't appear in the organic listings with these pages. The analytics and adwords accounts are linked. We are not paying for this mystery traffic through our PPC - these keywords are not appearing in our AdWords account (though other keywords / traffic are). The traffic is real - we have received phone calls from these pages, tracked to the visits recorded as organic These pages should only receive PPC traffic. They are receiving organic traffic also, but I can't recreate it. Can anyone suggest what's going on? I'm concerned about duplicate content issues and also skewing the analysis of the PPC campaign. Thanks
Reporting & Analytics | | RobPell0 -
Page Optimization score is An A but Rank is dropping
New to company in SEO and they wanted a page to rank high for certain term : knowledge management on this page: http://www.apqc.org/knowledge-management Ran SEO Moz optimizer and did suggested changes. Moved score to A and ranking was on google first page but now it's dropped considerably last 2 months as the bounce rate has gone through the roof. I'm at my wits end and don't want the higher ups on me for the drop. I'm not sure what to try next.
Reporting & Analytics | | inhouseninja0 -
If a page bounces in the woods, can Google Panda hear it?
I have read that after the Panda update a site's bounce rate is an important ranking metric. However, can anyone confirm whether all pages count equally? For instance, my home page gets 5000% more traffic than Deep Page X. If Deep Page X has a poor bounce rate, does it matter as much as if my Homepage has a bad bounce rate? I am guessing not, but wanted to open it up for discussion. If not, it has me wondering on what to do for some of my database driven content. I have some dynamically created pages that have higher bounce rates and minimal unique content. They aren't pure spam or junk, but are likely only about 1% unique from one another. Sounds like a no brainer change post-Panda, right? Well, what if I was the only one targeting the keywords for these pages? The pages pull from info I stored on the U.S. government stimulus program (related to my industry). It then has just about every city, state and county combo in the country for my product. For instance, a page <title>might be "Flemington, NJ Widgets - Somerset County". Something that no one else is targeting and drives minimal traffic.</p> <p> </p> <p>Do I take this content down? I didn't have any affects, positive or negative from Panda, so I am hesitant to take down thousands of Google cached pages.</p></title>
Reporting & Analytics | | TheDude0