Site crawler hasn't crawled my site in 6 days!
-
On 4.23 i requested a site crawl. My site only has about 550 pages. So how can we get faster crawls?
-
Crawl In Progress
Subdomain
www.taxproblem.org
Submitted 5:47pm GMT
Apr 23rd 2011
Is it a good idea to delete this (can't see how in the tool). I was thinking that maybe it got lost or something, it's now 10 days. I wanted to make another request.
I was mistaken before, the site wasn't crawled, it was just a campaign update.
-
I listed my craws in progress with the dates, so yes.
-
That one uses the exact same technology without the pretty report. Have you used it lately? It was upgraded a week ago and I believe intended to replace the one in seomoz labs.
-
Actually that's the link to the worst one that doesn't do anything. Yesterday my site was crawled, so it actually did take a week, but as you can see the results here are lagging...
Crawl In Progress
Subdomain
www.tax-audit.us
Submitted 1:36am GMT
Apr 25th 2011
Crawl In Progress
Subdomain
www.taxproblem.org
Submitted 5:47pm GMT
Apr 23rd 2011
-
That seems long to me. If you run a crawl. You can run it here instead and get results in 24 hours. Up to 2 per day and 3000 pages. It won't give you that nice red, blue and yellow report but it basic info is the same. You just need to manually analyze it. http://pro.seomoz.org/tools/crawl-test
-
I'm not sure to be honest but before it slowed down a little I tended to get a crawl once a week per site, whether a site was large or small.
-
Thanks. When fixed, how long should we have to wait for crawls?
-
That's not actually that long, and there's a queue anyway... I had a small site wait a couple of weeks recently but it did get crawled, was just in the queue for a while first. Just give it a while longer and it'll be fine.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Avg Page Load Time (sec) Comppared to site average - what does it mean?
Hi All, In google analytic In Site Speed -> Page Timings we have two columns a) Page Views & b) Avg Load Time (sec) compared to site average. Now in "b" column I am able to below % one in green and another in brown so what does it mean? Can anyone please explain me? Image attached Thanks! bNbBA
Reporting & Analytics | | amu1230 -
Large event site - how should I structure my URLs?
Hi guys, I'm working on a new website which is consolidating a number of existing event sites into one. The existing sites use a variety of URL structures: www.eventsite1.com/events/event-name www.eventsite2.com/festival-program/event-name www.eventsite3.com/event-name This inconsistency has led to issues with tracking category usage properly in analytics - for instance, with eventsite3.com, events fall within categories (www.eventsite3.com/category-name) but as soon as you drill into an event detail page (www.eventsite3.com/event-name) from the category page, the category is lost to analytics. This is compounded when one event lives within multiple categories, as I can't figure out which category is the most effective for a particular event. I've seen other event sites establish a canonical URL for a primary category, display it in the URL (i.e. www.eventsite4.com/primary-category/event-name) yet still let that event get hit via the secondary categories (www.eventsite4.com/secondary-category/event-name). This way, the categories get passed to analytics without any duplicate content issues (i.e. via the setting of canonicals) Basically, I want to make sure that whatever instruction I give to the devs for the new site re: URL structure is correct from an SEO perspective and analytics perspective. Do I even need to worry about having the category in the URL? Can someone please help me with this? Hope this makes sense Cheers
Reporting & Analytics | | cos20300 -
Are organic search visitors always seen as organic in origin, even if their return to the site is direct?
Many of our conversions occur in a customers second visit to the site. Often, a customer will arrive at our site, submit a finance application, leave, and return at a later date to checkout. We are interested in tracking how many of our checkouts come from customers who originally found our site through an organic search result. If a customer enters the site through organic search, leaves, and returns later through an email link or directly entering our URL, will G analytics show that customer as direct or organic origin? Cheers, Ben
Reporting & Analytics | | WSPL0 -
How to get crawled pages indexed?
Hi, I've got over 1k pages crawled but approx 100 pages indexed. Although, i submit them on Google Fetch and the links are indexable,they are not indexed. What shall i do the get max pages indexed? Any input highly appreciated. Thanks!
Reporting & Analytics | | Rubix0 -
Does anyone know of a way to do a profile level filter to exclude all traffic if it enters the site via certain landing pages?
Does anyone know of a way to do a profile level filter to exclude all traffic if it enters the site via certain landing pages? The problem I have is that we have several pages that are served to visitors of numerous other domains but are also served to visitors of our site. We end up with inflated Google Analytics numbers because people are viewing these pages from our partners' domains but never actually entering our site. I've made an advanced segment that serves the purpose but I'd really like to filter it at the profile level so the numbers across the board are more accurate without having to apply an advanced segment to every report. The advanced segment excludes visits that hit these pages as landing pages but includes visits where people have come from other pages on our domain. I know that you can do profile filters to exclude visits to pages or directories entirely but is there a way to filter them only if they are a landing pages? Any other creative thoughts? Thanks in advance!
Reporting & Analytics | | ATIseo0 -
Google Analytics Site Search to new sub-domain
Hi Mozzers, I'm setting up Google's Site Search on a website. However this isn't for search terms, this will be for people filling in a form and using the POST action to land on a results page. This is similar to what is outlined at http://support.google.com/analytics/bin/answer.py?hl=en&answer=1012264 ('<a class="zippy zippy-collapse">Setting Up Site Search for POST-Based Search Engines').</a> However my approach is different as my results appear on a sub-domain of the top level domain. Eg.. user is on www.domain.com/page.php user fills in form submits user gets taken to results.domain.com/results.php The issue is with the suggested code provided by Google as copied below.. Firstly, I don't use query strings on my results page so I would have to create an artificial page which shouldn't be a problem. But what I don't know is how the tracking will work across a sub-domain without the _gaq.push(['_setDomainName', '.domain.com']); code. Can this be added in? Can I also add Custom Variables? Does anyone have experience of using Site Search across a sub-domain perhaps to track quote form values? Many thanks!
Reporting & Analytics | | panini0 -
Is there any way to find out how many people blocked our site in Google?
With it being a factor in Panda, I would like to know what the number is and whether it's high / low compared to other sites etc. I kind of doubt a lot of people would have blocked us, but I'd like to at least be aware of whether that's a problem for us.
Reporting & Analytics | | rango0 -
How to find out which URLs are NOT indexed on a site
Is there a way to easily find out which URLs on a store-type site are NOT being indexed in Google? For example, if my sitemap information in Google Webmaster tools shows I have 7342 URLs in my sitemap and 5699 of those indexed, how do I find out what the 1643 non-indexed URLS are? Thanks for any help!
Reporting & Analytics | | GregWalt0