Rogerbot did not crawl my site ! What might be the problem?
-
When I saw the new crawl for my site I wondered why there are no errors, no warning and 0 notices anymore.
Then I saw that only 1 page was crawled. There are no Error Messages or webmasters Tools also did not report anything about crawling problems.
What might be the problem?
thanks for any tips!
Holger -
I am running into this problem as well. Does anyone have an answer. I was a customer a year ago and had no problems. Stopped for a while and then came back, my site was never able to be crawled again. A few months later I did make a few changes on my site, and just tried again today, but it only gets 1 page. Once before they told it had to do with something like http 1.0/1.1 which makes no sense. Not sure why google, bing and the rest can crawl the site, but this service can't. Hopefully someone has an answer. I looked at that link that is mentioned and I do not feel that any of those are causing the issues.
-
Hi there,
Thanks for reaching out! I went ahead and started a ticket for you to do some digging. I will run a few tests on my end to see where it is causing the issue. I will get in touch you ASAP
Thanks again for reaching, hope I could talk to you soon!
Best,
-
Thanks Keri,
But its not about these common problems, because Rogerbot crawled the properly before. May be a ticked will help to find the cause.
Holger
-
Hi! Here are some common reasons why only one page is crawled. If this doesn't solve it, opening a ticket with the help desk would be the next step.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Site explorer Issue
Hello, I'm looking to see in the Site Explorer the links coming from directories such as BOW, yahoo etc. I'm listed there from almost 1 year and these links are not listed, the same with my competitors. I'm missing something? Thank you Claudio
Moz Pro | | SharewarePros0 -
Still Cant Crawl My Site
I've removed all blocks but two from our htaccess. They are for amazonaws.com to block amazon from crawling us. I did a fetch as google in our WM tools on our robots txt with success. SEOMoz crawler here hit's our site and gets a 403. I've looks in our blocked request logs and amazon is the only one in there. What is going on here?
Moz Pro | | martJ0 -
How often does seomoz crawl the site? Can you force a crawl at a specific time ?
How often does seomoz crawl the site? Can you force a crawl at a specific time ?
Moz Pro | | stewbuch18720 -
Pages Crawled: 0 ?
I've been with SEO Moz for over a month and a half. Why would this weeks crawl have Pages Crawled: 0? I've made no changes since the crawl last week that had 10k pages crawled...
Moz Pro | | mr_w1 -
Problem with our connection to your Google Analytics
It is the second time in less then a month when i get that error message. The first time I just removed the connection and re-added it. Is there any other way to fix it? is it the problem on my end or is it on SEOMOZ side?
Moz Pro | | SirMax0 -
Is there any way to view crawl errors historically?
One of the website's we monitor have been getting high duplicate page titles, as we work through the pages, we see changes and the number of duplicate page titles are decreasing. However, lately, it went up again and the duplicate page titles have increased. I wanted to ask if there's any way to view the new errors and the old errors separately or sorted in a way that can help me identify why we are getting new page crawl errors. Any advice would be great. Thanks!
Moz Pro | | TheNorthernOffice790 -
Ruling out subfolders in pro tool crawl
Is there a way to "rule out" a subfolder in the pro dashboard site crawl? We're working on a site that has 500,000+ pages in the forums, but its the CMS pages we're optimizing and don't want to spend the 10k limit on forum pages.
Moz Pro | | DeepRipples0 -
How do I find the most linked to page of a site?
I'm looking at a site for a potential link and am trying to find the most linked to page. The SEOmoz toolbar tells me the root domain (DA) is linked to by 660 root domains but the main URL (PA) is linked to by 38 root domains. I used open site explorer and got the same # of 38 root domains in the result. From the Top Pages tab, I clicked on the 2nd page down and the SEOmoz toolbar gives me 189 root domains linking to that page (PA). Then I ran a Linkscape report to see what that would say and I get 146 linking root domains. 1. Is this 2nd page down on OSE the most linked to page? 2. a. Is something off in these numbers?
Moz Pro | | Motava
b. How come OSE/Linkscape doesn't report the 660 root domains in the DA?0