Campaign Not Crawling
-
I set up my first 5 campaigns and one is not crawling beyond one-page. It's been over 48 hours.
This site has nearly 3.500 pages the others much less, however, this shouldn't make any difference.
I searched for the problem and couldn't find it so I hope this question isn't redundant.
Comments and advice would be appreciated.
-
OOHH-LA-LA
That pesky Amazon cloud. Maybe a big Bingo. I will remove the blocked IPs from Amazon hosting. I had someone from those IPs slamming Google Ads so I blocked a big range all related to Amazon.
If the problem exists I'll reconstitute the block but for now let's see if removing them helps.
OK - I removed the motherload of IPs and I will come back and respond to the result after about 48 hours or sooner if I see positive results.
UPDATE: After removing the blocked IPs the site was crawled the max 10K pages.
UPDATE 2: The Adsense slammer returned under several different Amazon Cloud IPs so I had to block the full range again as I don't want to get banned by Google. No money registered only numerous ad clicks. This is unacceptable so I won't crawl this site.
Thanks,
Don
-
Hi Don,
So sorry to hear about the problems. Check out this help topic addressing what to do when only one page is crawled. If that doesn't solve your problem, send an email to help@seomoz.org
http://seomoz.zendesk.com/entries/409821-why-isn-t-my-site-being-crawled-you-only-crawled-one-page
-
Ah, ok, sorry - I misread your question Don.
I think I had the same problem with a campaign for a while, luckily it wasn't urgent so I left it, but it took a while to sort itself out
-
Hi Shelly:
The other 4 campaigns look to have gone through that starter crawl just fine, ergo, my question about that 5th campaign. I suppose if I don't see any action in a couple of more days I will indeed direct a question to the SEOMoz team but will leave this question open for a while also.
Thanks.
-
As I understand it, it takes a week for a complete crawl to be carried out on any new campaigns, but you now get a starter crawl when setting up a campaign - the starter crawl isn't as comprehensive and may explain why your sites haven't been crawled properly yet.
It may be worth directing a request to the SEOMoz team via the help function
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Problems with Crawl Diagnostics/New Campaign
Hi all I added a new domain to my campaigns yesterday and got the usual message saying we will have a small set of results ready in a couple of hours, and the rest will be done in a week or so. 24 hours later, this same message is still showing. Anyone else experiencing this or is it just related to my domain? Many thanks. Carl
Moz Pro | | GrumpyCarl0 -
Campaign Churning for Two Days
I started a campaign 5/11/2012. It's still churning. Here's what the Crawl Diagnostics Summary has said for the two days. Starter Crawl in Progress! Processing Crawl Data Started: May 11th, 2012 To get you started quickly Roger is crawling up to 250 pages on your site. You should see these results within two hours. The full crawl will complete within 7 days. Should I / can I stop this campaign? How?
Moz Pro | | lbohen0 -
How long does it typically take MOZBOT to crawl a site?
Our site has had "crawl in progress" for over 24 hours now without an update, we're dying for the results since our last changes :).
Moz Pro | | absoauto0 -
Websites First Crawl - Over 2 Hour Suggested Wait
Hello SEOMoz! We recently signed up for a free trial and on the pro dashboard it states the following. "To get you started quickly Roger is crawling up to 250 pages on your site. You should see these results within two hours. The full crawl will complete within 7 days." It's been nearly 24 hours and we see no results under Crawl Diagnostics however we do under rankings. Is this normal? Thanks
Moz Pro | | hostsurfuk0 -
Too Many On-Page Links: Crawl Diag vs On-Page
I've got a site I'm optimizing that has thousands of 'too many links on-page' warnings from the SeoMoz crawl diagnostic. I've been in there and realized that there are indeed, the rent is too damned high, and it's due to a header/left/footer category menu that's repeating itself. So I changed these links to NoFollow, cutting my total links by about 50 per page. I was too impatient to wait for a new crawl, so I used the On Page Reports to see if anything would come up on the Internal Link Count/External Link Count factors, and nothing did. However, the crawl (eventually) came back with the same warning. I looked at the link Count in the crawl details, and realized that it's basically counting every single '<a href'="" on="" the="" page.="" because="" of="" this,="" i="" guess="" my="" questions="" are="" twofold:<="" p=""></a> <a href'="" on="" the="" page.="" because="" of="" this,="" i="" guess="" my="" questions="" are="" twofold:<="" p="">1. Is no-follow a valid strategy to reduce link count for a page? (Obviously not for SeoMoz crawler, but for Google)</a> <a href'="" on="" the="" page.="" because="" of="" this,="" i="" guess="" my="" questions="" are="" twofold:<="" p="">2. What metric does the On-Page Report use to determine if there are too many Internal/External links? Apologies if this has been asked, the search didn't seem to come up with anything specific to this.</a>
Moz Pro | | icecarats0 -
Why is the SEOmoz crawler crawling the old version of our website?
Hello, I'm a new SEOmoz member. On Dec. 2nd, after completely redesigning our website, we migrated to a new hosting company by switching our DNS to the new server. The vast majority of the URLs have changed and we configured redirects of the old URLs to the new ones. Although, this task is not completed yet. After the migration, I created an account on SEOmoz to be able to track our progress and find the issues to fix to optimize our SEO. For some reason, in the SEOmoz reports it is the old URLs that show up. Unless the crawler does not actually crawl the pages and only uses the indexed pages to generate its report, I don't understand how could this possible. Anyone has a clue? When will the new URLs be indexed by SEOmoz and the major search engines? Thanks for your help!
Moz Pro | | Gestisoft-Qc0 -
How do I get my crawl report?
I received a message that my crawl report is complete with a link - went to the link however - when I click on the icon that has the report name and the complete check mark nothing happens looked around can't find the results. Need to bid on this job so it would be helpful to know where to get it. Thanks for all you do. Wickey
Moz Pro | | Wickey0 -
How do i get to know th pages crawled by SEOMOZ?
My SEOMOZ campaign says that "n" number of pages were crawled. How do i get access to the list of the pages crawled by SEOMOZ?
Moz Pro | | IM_Learner0
