Site crawler hasn't crawled my site in 6 days!
-
On 4.23 i requested a site crawl. My site only has about 550 pages. So how can we get faster crawls?
-
Crawl In Progress
Subdomain
www.taxproblem.org
Submitted 5:47pm GMT
Apr 23rd 2011
Is it a good idea to delete this (can't see how in the tool). I was thinking that maybe it got lost or something, it's now 10 days. I wanted to make another request.
I was mistaken before, the site wasn't crawled, it was just a campaign update.
-
I listed my craws in progress with the dates, so yes.
-
That one uses the exact same technology without the pretty report. Have you used it lately? It was upgraded a week ago and I believe intended to replace the one in seomoz labs.
-
Actually that's the link to the worst one that doesn't do anything. Yesterday my site was crawled, so it actually did take a week, but as you can see the results here are lagging...
Crawl In Progress
Subdomain
www.tax-audit.us
Submitted 1:36am GMT
Apr 25th 2011
Crawl In Progress
Subdomain
www.taxproblem.org
Submitted 5:47pm GMT
Apr 23rd 2011
-
That seems long to me. If you run a crawl. You can run it here instead and get results in 24 hours. Up to 2 per day and 3000 pages. It won't give you that nice red, blue and yellow report but it basic info is the same. You just need to manually analyze it. http://pro.seomoz.org/tools/crawl-test
-
I'm not sure to be honest but before it slowed down a little I tended to get a crawl once a week per site, whether a site was large or small.
-
Thanks. When fixed, how long should we have to wait for crawls?
-
That's not actually that long, and there's a queue anyway... I had a small site wait a couple of weeks recently but it did get crawled, was just in the queue for a while first. Just give it a while longer and it'll be fine.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
PDF best practices: to get them indexed or not? Do they pass SEO value to the site?
All PDFs have landing pages, and the pages are already indexed. If we allow the PDFs to get indexed, then they'd be downloadable directly from google's results page and we would not get GA events. The PDFs info would somewhat overlap with the landing pages info. Also, if we ever need to move content, we'd now have to redirects the links to the PDFs. What are best practices in this area? To index or not? What do you / your clients do and why? Would a PDF indexed by google and downloaded directly via a link in the SER page pass SEO juice to the domain? What if it's on a subdomain, like when hosted by Pardot? (www1.example.com)
Reporting & Analytics | | hlwebdev1 -
Stop getting info from Google analytics on purchases in our site
Hi guys, We have eCommerce.
Reporting & Analytics | | WayneRooney
We connected the site to the Google analytic eCommerce.
Everything was work fine until 3 weeks ago. Suddenly we stooped getting purchases information in the analytic although i see purchases in the website. We didn't change anything in the website and i really don't know how to solve this problem.
If someone here can point me where i can get some info on how to fix it it can be great. Thanks a lot!0 -
How can we stop Google analytics pulling in data from another site?
We have a few accounts under one Google login. They all have separate Google analytics codes but one of the sites is somehow pulling in some data from another site but the other site has not got the same analytics code on it. Not sure how this is happening and what we can do about this, is it a bug in the Google Analytics system? Any help would be appreciated.
Reporting & Analytics | | dentaldesign0 -
Redirects to external sites
Hi there, I have listings that all link to external sites. In order to track how many times each link was clicked I coded a redirect script that simply redirects to the respective url. www.mysite.com/go/123/ - redirects to another site www.mysite.com/go/541/ - redirects to a different site Is there anything I should or should not do from seo stand point such as blocking the /go directory in robots or adding a no-follow tag to links? I'd appreciate your advice. Thanks
Reporting & Analytics | | vmotuz0 -
Is Google able to determine duplicate content every day/ month?
A while ago I talked to somebody who used to work for MSN a couple of years ago within their engineering department. We talked about a recent dip we had with one of our sites.We argued this could be caused by the large amount of duplicate content we have on this particular website (+80% of our site). Then he said, quoted: "Google seems only to be able to determine every couple of months instead of every day if the content is actually duplicate content". I clearly don't doubt that duplicate content is a ranking factor. But I would like to know you guys opinions about Google being only able to determine this every couple of X months instead of everyday. Have you seen or heard something similar?
Reporting & Analytics | | Martijn_Scheijbeler0 -
Google Analytics internal Site Search - Destination pages dispaly Search results
Hi, Im having a bit of an issue with Google Analytics internal site search, I am able to currently track the search terms through my website internal search but when I click onto destination pages I just get the search result page. When clicking destination pages I would expect to get the pages on which the user ended up after the results page, instead I just get the results page which is pretty much useless ?submitsearchXXXXXX hope you can help, look forward to your response. Thanks,
Reporting & Analytics | | Tug-Agency1 -
Do Google penalise you for having too many 404's?
Hi There I have been doing some work reducing the number of 404's displayed in the Crawl Errors found in Googles Webmaster Tools. We had a lot of products that were no longer available so have now been removed to reduce the number of 404s that had been found. However, there are a number of URLs that have been crawled that do not exist on our website and have been flagged in the list of Crawl Errors. I want to know if Google will penalise us for this, perhaps affecting our quality score or if they can see that this is something out of our control. This site for example: http://sibd.com/com_offers_unique_gifts.html has generated a lot of truncated URLs on its site that link to pages that don't exist on our site: http://www.arenaflowers.com/flowers/pri… That is the exact link that it is trying to locate. Here is the report for that particular link. As you can see the content has been scraped by other sites which has spread the problem further. Pages that link to http://www.arenaflowers.com/flowers/pri.. URL Discovery Date
Reporting & Analytics | | ArenaFlowers.com
http://www.justsearchit.com.au/for_flowers_offers,3.html
Sep 12, 2011
http://sibd.com/offers_unique_gifts_for.html
Sep 11, 2011
http://sibd.com/offers_unique_gifts.html
Sep 11, 2011
http://sibd.com/com_offers_unique_gifts_for.html
Sep 11, 2011
http://www.flexfinder.com/flowers_offers_unique_gifts.html
Sep 10, 2011
http://sibd.com/offers_unique_gifts_with.html
Sep 10, 2011
http://sibd.com/com_offers_unique_gifts.html
Sep 10, 2011
http://sibd.com/of_flowers_offers_from.html
Sep 9, 2011
http://arama.frmpc.com/for_flowers_for_less_ltd_includes.html
Sep 9, 2011
http://arama.frmpc.com/flowers_for_less_than_do.html
Sep 9, 2011 I have spotted a lot of these and currently have around 3.3K 404s in total, a majority are from sites we don't control. Is there an acceptable number of 404s a site should aim for and is the above something we should address or are Google smart enough to work out that we can't fix this ourselves? Thanks! Sam.0 -
Setting up Goals in Google Analytics that involve a 3rd party site
I've set up several goals for one of my clients in Google Analytics. The ones that relate to things on the site -- such as clicking on the "Contact Us" button -- work just fine. However, I set up one that is tracking when someone clicks on a purchase button, which sends the user to a third party site (PayPal). This one doesn't seem to work. (I purchased and item and the goal was not recorded). Looking to see if I have to do anything different when setting up the goal.
Reporting & Analytics | | EricVallee340