How do i get the crawler going again?
-
The initial crawl only hit one page. Set up another campaign for another site and it crawled 260 pages.
How can I get the crawler started up again or do I really have to wait a week ?
-
It's easiest for the Moz team to respond to individual questions like this via an email to help@seomoz.org or sending in a ticket through the help hub at seomoz.org/help.
-
Hi Keri,
Yeah, I've followed these steps. htaccess forwards to www if none and my campaign is using www.
The robots.txt is available in browser no problem.
my htaccess had massive country ip blocks as well as amazonaws..
I've removed all blocks that arent china or amazon.
Still waiting on a response from the MOZ team here on how to get the real crawl going as what I found in tools is not the same as the weekly scan.
-
Also, for why we only crawled one page, check out https://seomoz.zendesk.com/entries/409821-Why-Isn-t-My-Site-Being-Crawled-You-re-Not-Crawling-All-My-Pages- for some helpful tips.
-
Whenever you have crawl issues, the SEOMoz help team is the place to go. They're the only ones who can look into your account to see if there's an issue or such. (And they are really nice and quick to respond.)
-
Research Tools > Crawl Test at bottom left of page.
Not sure why it's taken me 2 days to find that but it did.
Thanks. Sorry for the dumb question...
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Crawlers reporting upper case letter url versions although these have been 301'd to lower case !?
Hi I have a client e-com site who's dev platform is on a windows server Their product pages have been auto-named after the product title, with the first letter in each word being upper case, which has hence translated to the URL having upper cases instances too. I asked them to set up 301 redirects for all url's that had upper case instances to lower case versions, which they say they have done. However I'm still seeing url's with upper case instances showing up in webmaster tools and moz crawl reports but when I copy & paste them into a browser they do redirect to, & resolve in, the lower case version. Its also upper case versions reported in the Google cache! So how come webmaster tools & Moz etc are reporting the upper case versions, surely if redirected it should be the lower case versions All Best Dan
Moz Pro | | Dan-Lawrence0 -
I'm sure you get this all the time but....
Hi New to Moz, I'm finding it all very helpful and daunting at the same time. I've read heaps of Seo the Moz content and watched every Video I can find. It's all going well and I've managed to get A's on some of my pages, I was hoping some one cold tell me why this site (Link to screan grab http://tinypic.com/r/dnj9di/8 ) is way above me in the Google search results? It appears to be under performing me in Moz but in actual fact it is greatly out performing me in Google. Any insights are more than gratefully received! Thanks Ben
Moz Pro | | SussexChef830 -
Getting rid of duplicate content
Hi everyone, I'm a newbie and at the moment don't know very much about SEO. I have a problem with some of my campaigns where i keep getting a report with either Duplicate Page and/or Duplicate Content errors. I have no idea how to rectify this error, remove it or fix it on the relevant websites. Can anyone please help explain how to do this, maybe step by step? I really appreciate your views and opinions! Regards, Hugh
Moz Pro | | DigitalAcademyZA0 -
Do SEOmoz private questions get answered?
I submitted a question on September 30 and never received a response. I then followed up on October 6 and had an exchange with the support staff, who first told me that the question would be answered and then a couple of days later told me that they had no record of the question. By chance, I had taken a screen shot showing confirmation that the question had been sent (just something I did to remind myself to follow up if I didn't get a response!), which I sent to them along with the question again. That was four days ago, and I've heard nothing since then - not even an automated email saying that the question was received. Is it common for it to take almost a month before a response is received, and to have to follow up multiple times just to get an answer?
Moz Pro | | csmm0 -
Is there a way to get the SEO campaign crawl to run from outside the US?
We are working on a international site that uses IP redirection. I am currently trying to crawl and monitor the Australian version of the site, however the pro-campaign crawler is running from the US and only giving me US data. Is there a way to tell the crawler to run through a local proxy so we can track international versions? Thanks!
Moz Pro | | MBASydney0 -
Getting SEOMoz reports to ignore certain parameters
I want the SEOMoz reports to ignore duplicate content caused by link-specific parameters being added to URLs (same page reachable from different pages, having marker parameters regarding source page added to the URLs). I can get Google and Bing webmaster tools to ignore parameters I specify. I need to get SEOMoz tools to do it also!
Moz Pro | | SEO-Enlighten0 -
Crawler reporting incorrect URLs, resulting in false errors...
The SEOmoz crawler is showing 236 Duplicate Page Titles. When I go in to see what page titles are duplicated I see that the URLs in question are incorrect and read "/about/about/..." instead of just "/about/" The shown page duplicates are the result of the crawler is ending up on the "Page not found" page. Could it be the result of using relative links on the site? Anything I can do to remedy? Thanks for your help! -Frank
Moz Pro | | Clements1 -
Trying to understand how a website is getting higher than me with less links
Hi i am new to seo and trying to teach myself the best way to improve a site and the best way to use the tools on seomoz. The problem i have is. i am working on a page at the moment on a site. the page is called weight loss hypnotherapy http://www.clairehegarty.co.uk/weight-loss-hypnotherapy and i have around 130 links going to the page where as a site which is much higher than me in google has only around 5 links. I cannot understand with being new to seo how this can happen, can anyone please explain what i need to do to improve my ranking please. here is the site i am talking about that is higher than me www.weightlosshypnotherapy.co.uk/ any help would be great can anyone also give me a good example of a page before it has been optimised and a page after it has had this done.
Moz Pro | | ClaireH-1848860