That's exactly the problem, the redirects are disbursed randomly throughout the site. Although, and the job's still running, it now appears as though there's almost a 1-2-1 correlation between pages and redirects on the site.

I also heard from Dan Sharp via Twitter. He said "You can't, as we'd have to crawl a URL to see the status code You can right click and remove after though!"

Thanks again Michael. Your thoroughness and follow through is appreciated.

Thank you Michael.

You're right. I was on a 64 bit machine running a 32 bit verson of java. I updated it and the scan has been running for more than 24 hours now without hanging. So thank you.

If anyone else knows of a way to exclude files using status codes I'd still like to learn about it. So far the scan is showing me 20,000 redirected files which I'd just as soon not inventory.

I have a very large ecommerce site I'm trying to spider using screaming frog. Problem is I keep hanging even though I have turned off the high memory safeguard under configuration.

The site has approximately 190,000 pages according to the results of a Google site: command.

• The site architecture is almost completely flat. Limiting the search by depth is a possiblity, but it will take quite a bit of manual labor as there are literally hundreds of directories one level below the root.
• There are many, many duplicate pages. I've been able to exclude some of them from being crawled using the exclude configuration parameters.
• There are thousands of redirects. I haven't been able to exclude those from the spider b/c they don't have a distinguishing character string in their URLs.

Does anyone know how to exclude files using status codes? I know that would help.

If it helps, the site is kodylighting.com.

Thanks in advance for any guidance you can provide.

Can Moz weigh in on this please? There is a lot of really  good discussion on this particular topic across multiple threads but no clear direction. What does Moz recommend and why? The why is important and here's why.

If I have a client with an established blog that uses the blog.domain versus domain/blog structure, should I be recommending they migrate to a different structure?

Here's something else you will find helpful.

This blog post, published earlier today (4/22/13) by Phil Rosen at LocalVisibilitySystem, identifies all the well known local directories on which you can create a business listing without having to disclose your address. Places like Angie's List and the Bing business portal.