Site crawler hasn't crawled my site in 6 days!
-
On 4.23 i requested a site crawl. My site only has about 550 pages. So how can we get faster crawls?
-
Crawl In Progress
Subdomain
www.taxproblem.org
Submitted 5:47pm GMT
Apr 23rd 2011
Is it a good idea to delete this (can't see how in the tool). I was thinking that maybe it got lost or something, it's now 10 days. I wanted to make another request.
I was mistaken before, the site wasn't crawled, it was just a campaign update.
-
I listed my craws in progress with the dates, so yes.
-
That one uses the exact same technology without the pretty report. Have you used it lately? It was upgraded a week ago and I believe intended to replace the one in seomoz labs.
-
Actually that's the link to the worst one that doesn't do anything. Yesterday my site was crawled, so it actually did take a week, but as you can see the results here are lagging...
Crawl In Progress
Subdomain
www.tax-audit.us
Submitted 1:36am GMT
Apr 25th 2011
Crawl In Progress
Subdomain
www.taxproblem.org
Submitted 5:47pm GMT
Apr 23rd 2011
-
That seems long to me. If you run a crawl. You can run it here instead and get results in 24 hours. Up to 2 per day and 3000 pages. It won't give you that nice red, blue and yellow report but it basic info is the same. You just need to manually analyze it. http://pro.seomoz.org/tools/crawl-test
-
I'm not sure to be honest but before it slowed down a little I tended to get a crawl once a week per site, whether a site was large or small.
-
Thanks. When fixed, how long should we have to wait for crawls?
-
That's not actually that long, and there's a queue anyway... I had a small site wait a couple of weeks recently but it did get crawled, was just in the queue for a while first. Just give it a while longer and it'll be fine.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Blocking Standard pages with Robots.txt (t&c's, shipping policy, pricing & privacy policies etc)
Hi I've just had best practice site migration completed for my old e-commerce store into a Shopify environment and I see in GSC that it's reporting my standard pages as blocked by robots.txt, such as these below examples. Surely I don't want these blocked ? is that likely due to my migrators or s defaults setting with Shopify does anyone know? : t&c's shipping policy pricing policy privacy policy etc So in summary: Shall I unblock these? What caused it Shopify default settings or more likely my migration team? All Best Dan
Reporting & Analytics | | Dan-Lawrence0 -
GA Internal Site Search Correct Query Parameter?
Hi Guys, Recently added GA internal site search to a sub-folder: http://tinyurl.com/jhm9cyl Just want to confirm would the query parameter be: /search/ Or different because it's a sub-folder? Cheers.
Reporting & Analytics | | jayoliverwright0 -
If i was to drastically improve 5 critical things on my site, what would you suggest?
I have put in a lot of improvements on my site both onsite and offsite, I was just wondering from a critical point of view, what 5 things would you suggest would require an improvement, that will consequently lead to both, a better user experience and better Rankings on Google? Open even to criticism 🙂 Thank You..... Find my site here:http://bit.ly/1vW4GGP
Reporting & Analytics | | ConnectMedia0 -
Which Algorithm Change Hurt the Site? A causation/correlation issue
The attached graph is from google analytics, a correlation of about 14 months of Organic Google visits with algo changes, data from moz naturally 🙂 Is there any way to tell from this which will have affected the site? for example #1 or #2 seems to be responsible for the first dip, but #4 seems to fix it and it broke around 6, or is the rise between 4 and 7 an anomaly and actually 1 or 2 caused a slip from when it was released all the way to when 7 was released. Sorry if the graph is a little cloak and dagger, that is partly because we don't have permissions to reveal much about the identity, and partly because we were trying to do a kind of double blind, separating the data from our biases 🙂 We can say though the different between the level at the start and end of the graph is at least 10,000 visits per day JarMzoK.png
Reporting & Analytics | | Fammy0 -
Open site explorer or other backlinkchecker https
Hi MozFans, I would like to check the backlink profile of https deeplinks on a site.
Reporting & Analytics | | MaartenvandenBos
In open site explorer this isn't working: Link Is there a other way to check the backlinks of a https website? cheers,
Maarten0 -
What is the best way to track mobile sites in Google Analytics?
Hello! I am wondering what the pros and cons of using the regular Google Analytics tracking code on a mobile site versus the tracking documentation from Google specifically on it found at http://code.google.com/mobile/analytics/docs/web/ which is still in labs mode. Does the mobile specific tracking have the same features as the regular one to be able to track events and report the same statistics? Thanks for the help on this one!
Reporting & Analytics | | CabbageTree0 -
Do Google penalise you for having too many 404's?
Hi There I have been doing some work reducing the number of 404's displayed in the Crawl Errors found in Googles Webmaster Tools. We had a lot of products that were no longer available so have now been removed to reduce the number of 404s that had been found. However, there are a number of URLs that have been crawled that do not exist on our website and have been flagged in the list of Crawl Errors. I want to know if Google will penalise us for this, perhaps affecting our quality score or if they can see that this is something out of our control. This site for example: http://sibd.com/com_offers_unique_gifts.html has generated a lot of truncated URLs on its site that link to pages that don't exist on our site: http://www.arenaflowers.com/flowers/pri… That is the exact link that it is trying to locate. Here is the report for that particular link. As you can see the content has been scraped by other sites which has spread the problem further. Pages that link to http://www.arenaflowers.com/flowers/pri.. URL Discovery Date
Reporting & Analytics | | ArenaFlowers.com
http://www.justsearchit.com.au/for_flowers_offers,3.html
Sep 12, 2011
http://sibd.com/offers_unique_gifts_for.html
Sep 11, 2011
http://sibd.com/offers_unique_gifts.html
Sep 11, 2011
http://sibd.com/com_offers_unique_gifts_for.html
Sep 11, 2011
http://www.flexfinder.com/flowers_offers_unique_gifts.html
Sep 10, 2011
http://sibd.com/offers_unique_gifts_with.html
Sep 10, 2011
http://sibd.com/com_offers_unique_gifts.html
Sep 10, 2011
http://sibd.com/of_flowers_offers_from.html
Sep 9, 2011
http://arama.frmpc.com/for_flowers_for_less_ltd_includes.html
Sep 9, 2011
http://arama.frmpc.com/flowers_for_less_than_do.html
Sep 9, 2011 I have spotted a lot of these and currently have around 3.3K 404s in total, a majority are from sites we don't control. Is there an acceptable number of 404s a site should aim for and is the above something we should address or are Google smart enough to work out that we can't fix this ourselves? Thanks! Sam.0 -
Google Analytics' Goals and Cold Fusion
We are having an issue tracking goal completions in GA with a cold fusion site. I would like to know if anyone as had similar issues and found a solution.
Reporting & Analytics | | JamesBarry0