Is seomoz rogerbot only crawling the subdomains by links or as well by id?
-
I´m new at seomoz and just set up a first campaign. After the first crawling i got quite a few 404 errors due to deleted (spammy) forum threads. I was sure there are no links to these deleted threads so my question is weather the seomoz rogerbot is only crawling my subdomains by links or as well by ids (the forum thread ids are serially numbered from 1 to x).
If the rogerbot crawls as well serially numbered ids do i have to be concerned by the 404 error on behalf of the googlebot as well?
-
Roger only crawls by links.
When you are using forum software or a CMS it is very common for numerous links to be created by the software without your knowledge. If you examine your crawl reports, they will show you the URL of the page which provides the link which is generating the 404 error.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Unsolved Can I exclude subdomain from a Campaign?
Also, I would like to track a subdomain without the root domain, is that possible?
Moz Pro | | AndreVa0 -
SEOmoz crawler not crawling my site
We set up a new campaign in SEOmoz on Friday. It is my understanding that the preliminary crawl can cover up to 250 and this has been our experience in the past. However, the preliminary crawl only went through 2 pages. This is a larger eCommerce site with many pages. Any ideas why more pages weren't crawled? We set up the campaign to track at the root domain level.
Moz Pro | | IMM0 -
How to find page with the link that returns a 404 error indicated in my crawl diagnostics?
Hi Newbie here - I am trying to understand what to do, step by step, after getting my initial reports back from seomoz. The first is regarding the 404 errors shown as high priority to fix, in crawl diagnostics. I reviewed the support info help on the crawl diagnostics page referring to 404 errors, but still did not understand exactly what I am supposed to do...same with the Q&A section when I searched how to fix 404 errors. I just could not understand exactly what anyone was talking about in relation to my 404 issues. It seems I would want to find the page that had the bad link that sent a visitor to a page not found, and then correct the problem by removing the link, or correcting and re-uploading the page being linked to. I saw some suggestions that seemed to indicate that seomoz itself will not let me find the page where the bad link is and that I would need to use some external program to do this. I would think that if seomoz found the bad page, it would also tell me what page the link(s) to the bad page exists on. A number of suggestions were to use a 301 redirect somehow as the solution, but was not clear when to do this versus, just removing the bad link, or repairing the page the link was pointing to. I think therefore my question is how do I find the links that lead to 404 page not founds, and fix the problem. Thanks Galen
Moz Pro | | Tetruss0 -
Open Site Explorer - Link Export Not Showing Same Number of Links
Noob question. I'm using Open Site Explorer to hunt down some external links for a website. In the attached image (post-pressing the filter button), it says I have 889 external links, but after I've downloaded the CSV, I only get 695 links. Why the difference in number? And if I'm doing something wrong, how do I make sure I'm getting the full amount of external links to my client's website? Thanks! GG6wURf.jpg
Moz Pro | | EEE31 -
Link reporting.
Is there a way in the Pro reporting where I can see a summary of the number of incoming links by type (blogs / news / wiki / dir / forums etc)? Even better, could the report give me an average Page Rank for each link type? Thanks,
Moz Pro | | CarlDarby0 -
Only crawling one page
Hi there, A campaign was crawling fine, but at the last crawl, for some reason, SEOmoz can only crawl one page... any ideas? If I run a custom crawl I still access all of the site's pages.
Moz Pro | | harryholmes0070 -
Does anyone know of a crawler similar to SEOmoz's RogerBot?
As you probably know SEOmoz had some hosting and server issues recently, and this came at a terrible time for me... We are in the middle of battling some duplicate content and crawl errors and need to get a fresh crawl of some sites to test things out before we are hit with the big one? Before I get a million thumbs downs- I love and will continue to use SEOmoz, just need something to get me through this week ( or until Roger is back! )!
Moz Pro | | AaronSchinke1 -
Link Acquisition Assistant
Hi Can anyone explain what all the info within the link acquisition tool is all about? I put in to look for Money Hoodies, Money being the brand name and Hoodies being the product type. This is what I get back, what do the links mean when I click them? I don't get it? money hoodies + "submit site"
Moz Pro | | WillBlackburn
money hoodies + "submit url"
money hoodies + "intitle:directory"
money hoodies + "intitle:resources"
money hoodies + "directory list"
money hoodies + "inurl:list"
money hoodies + "intitle:list"
money hoodies + "inurl:directory"
money hoodies + "inurl:resources"
money hoodies + "add url"
money hoodies + "submit url"
money hoodies + "suggest url"
money hoodies + "suggest a url"
money hoodies + "submit site"
money hoodies + "submit a site"
money hoodies + "submit your site"
money hoodies + "add a url"
money hoodies + "add a site"
money hoodies + "add listing"
money hoodies + "add your listing"
money hoodies + "submit a listing"
money hoodies + "submit listing"
money hoodies + "submit your website"0