Is seomoz rogerbot only crawling the subdomains by links or as well by id?
-
I´m new at seomoz and just set up a first campaign. After the first crawling i got quite a few 404 errors due to deleted (spammy) forum threads. I was sure there are no links to these deleted threads so my question is weather the seomoz rogerbot is only crawling my subdomains by links or as well by ids (the forum thread ids are serially numbered from 1 to x).
If the rogerbot crawls as well serially numbered ids do i have to be concerned by the 404 error on behalf of the googlebot as well?
-
Roger only crawls by links.
When you are using forum software or a CMS it is very common for numerous links to be created by the software without your knowledge. If you examine your crawl reports, they will show you the URL of the page which provides the link which is generating the 404 error.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How to make a Crawl Report readable?
Hi! I am trying to find out how I make my CSV report neat, so I can interprete the report. I know have a CSV report with just numbers and text all in one column. I tried the button text to columns but that doesn't work because when I do that at column A it overwrites column B in which I have the same problem! Thanks
Moz Pro | | HetCommunicatielokaal0 -
Rogerbot did not crawl my site ! What might be the problem?
When I saw the new crawl for my site I wondered why there are no errors, no warning and 0 notices anymore. Then I saw that only 1 page was crawled. There are no Error Messages or webmasters Tools also did not report anything about crawling problems. What might be the problem? thanks for any tips!
Moz Pro | | inlinear
Holger rogerbot-did-not-crawl.PNG0 -
Difference in data between http://pro.seomoz.org/tools/keyword-difficulty and http://lsapi.seomoz.com/linkscape/url-metrics/
Hi, Has any once else experienced any difference in data between http://lsapi.seomoz.com/linkscape/url-metrics/ and http://pro.seomoz.org/tools/keyword-difficulty Please look at the attached image. For "http://www.webmd.com/diet/guide/choosing-weight-loss-program" and "http://www.freedieting.com/" page authority and domain authority match exactly. But for "http://www.fitnessmagazine.com/weight-loss/plans/" data does not match. The data from "http://lsapi.seomoz.com/linkscape/url-metrics/" was retrieved brely 60 seconds latter after data from "http://pro.seomoz.org/tools/keyword-difficulty". We used our custom app for retrieve data from "http://lsapi.seomoz.com/linkscape/url-metrics/". The columns were matched against the specs given in "http://apiwiki.seomoz.org/w/page/13991153/URL-Metrics-API". We are retrieving following columns 1)ut(Title) 2)ueid(External Links) 3)uid(Links) 4)umrp(mozRank) 5)upa(Page Authority) 6)pda(Domain Authority) Any help will be greatly appreciated. zvFif.jpg
Moz Pro | | claytons0 -
Sometimes we could not download all the data of inbound links from all linking domains
Sometimes we could not download all the data of inbound links from all linking domains. (Around tithe.) Don't you have any idea?
Moz Pro | | crossfinity0 -
How long would a SEOMoz crawl usually take for a site with around 4000 pages?
We are working through optimising a site for one of our clients and the SEOMoz crawl progress says it has been running since the 8th Feburary. It's now almost a week later and it still hasn't finished. The first run took a few days, is there any way of restarting the process?
Moz Pro | | TJSSEO0 -
Why is Followed Linking Root Domains higher than External Followed Links?
Surely there must be at least one external link for each linking root domain? Some results for smaller sites give a higher number of domains linking in than incoming links - e.g. www.forbesandsawyer.co.uk Under Subdomain metrics: External Followed Links - 1 Followed Linking Root Domains - 2 Surely 2 root domains would mean AT LEAST 2 external followed links? Thanks, Andrew
Moz Pro | | Silktide0 -
Why am I still Unranked in SEOmoz?
I was very active in SEOmoz's Q&A forums today and completed my profile - raising my Moz points from 0 to 50. On my profile, however, I am still listed as Unranked. Also, I do not appear on the SEOmoz Top Members List next to other users with 50 points. I'm guessing this is something that is updated once a day or so, and the rank will update tomorrow. Is my suspicion correct?
Moz Pro | | AnthonyMangia0 -
Only Crawling 1 page?
Hi Guys, Any advice much appreciated on this! Recently set up a new campaign on my dashboard with just 5 keywords. The domain is brammer.co.uk and a quick Google site:brammer.co.uk shows a good amount of indexed pages. However - first seomoz tool crawl has only crawled 1 url!! "Last Crawl Completed: Apr. 12th, 2011 Next Crawl Starts: Apr. 17th, 2011" Any ideas what's stopping the tool crawl anymore of the site?? Cheers in advance.. J
Moz Pro | | lovealbatross0