Each article's view pages are unique? Or are they all one page with uniquely propagated iframes? If it's one page, then add it to your robots.txt file as a disallowed page.

Otherwise yes you can no-index

OR you can add a rel="nofollow" to the links which might be a good idea anyway. Of course if the page is linked from anywhere else it could be indexed.

Or you could send them all to a unique directory and have that particular directory removed in the robots file. This would pertain if each article was given a unique page. Just have them go to url.com/article/view/thisarticle.html and disallow the view/ directory.

Hope this helps.

I'm confused by what exactly you're looking for here, but to make my answer totally general I'll say this:

I think that a "long-tail keyword" is one that reads more like a sentence than a phrase. For example, if I'm searching for how to change a tire on my ford focus I could type in

"changing tire ford focus"

or

"how to fix a flat tire on a ford focus"

...The first might be short-tail in this scenario whereas the second would be longer. The second won't get as many searches, sure. But I might find that this drives in an extra click or two and all I had to do was sneak this sentence into my article. E.G. "A lot of people ask me how to fix a flat tire on a Ford Focus." or something...

i think that because you said "http://lucid8.com/index.asp and http://www.lucid8.com are duplicates" your issue might actually be in the www and non-www canonicalization of your site.

Make sure that you pick one or the other (i recommend www) and in your IIS server settings redirect the non-www to the www. (or vice versa if you must)

This should solve your problem.

**update -- I just went to your site and yes this is happening. You have two sites going, http://lucid8.com and http://www.lucid8.com. The "index.asp" portion of it actually has nothing to do with this. But Google is currently seeing each version as two separate pages, resulting in duplicates of every page on that domain.

Definitely add the 301 redirect in your IIS settings to one or the other. A linkrel=canonical tag wouldn't hurt on each page either, but isn't necessary.

Good luck

links that don't exist or links to pages that don't exist?

..heck, either way i'd ignore them and focus on phase 2 of your plan. Disavow seems to be a bit overused in my opinion. It's more of a last-ditch effort for penalty recovery IMHO.

and if it's 404 errors you're trying to fix: Google will eventually stop following those after they 404 long enough. Don't even worry about it. (unless they're links you want, then put a relevant redirect in place.)

Hope this was helpful.