Thanks so much Ryan, I had a hunch that we should use common sense on this one. I will definitely try the nofollow approach with the website owners who are not cooperating, and I'll respond back in here with an update here with the results for the benefit of others.

Thanks again!

Hey Ryan,

Thanks so much for your detailed analysis, this confirms the gameplan we had to remove thousands of links to our site. We ended up contacting about 800 webmasters who owned ~2,500 domains total with spammy links to our domains, and we had about a 6.5% response rate.

About 42 site owners responded and removed their links easily. Then there were about 10 who responded that they would take down the links for a fee. We've paid 8 website owners for a total of about $135 but there are three more that own a ton of domains and they want another $410 total for about 75 links.

We are trying not to spend that much on link removal, do you think if we document this and send it to Google they will understand the high cost that we don't want to pay, or will they just tell us that we have to pay this off if we want the manual penalty removed?

Curious about this question also, it would be very useful to see a master list of all URLs on our site that are not indexed by Google so that we can take action to see what aspects of the page are lacking and what we need for it to get indexed.

Thanks for clarifying!

Do you have any suggestions on how I can avoid duplicate content that is appearing on multiple pages without noindexing the whole page (there is original content at the top, duplicate at the bottom)?

I came across this article about telling google not to crawl a portion of a webpage, but I never hear anyone in the SEO community talk about them. http://perishablepress.com/press/2009/08/23/tell-google-to-not-index-certain-parts-of-your-page/

Does anyone use these and find them to be effective? If not, how do you suggest noindexing/canonicalizing a portion of a page to avoid duplicate content that shows up on multiple pages?

Google actually has a very good overview of creating XML video sitemaps.  You can create a sitemap for your videos even if they are hosted by YouTube.

From the page: "Video content includes web pages which embed video, URLs to players for video, or the URLs of raw video content hosted on your site."

You can add video extension tags in your existing site, or create a standalone sitemaps just for videos.  Also, you can submit the sitemap through the wonderful Webmaster Tools.

Good luck!

Google actually has a very good overview of creating XML video sitemaps.  You can create a sitemap for your videos even if they are hosted by YouTube.

From the page: "Video content includes web pages which embed video, URLs to players for video, or the URLs of raw video content hosted on your site."

You can add video extension tags in your existing site, or create a standalone sitemaps just for videos.  Also, you can submit the sitemap through the wonderful Webmaster Tools.

Good luck!

I came across this article about telling google not to crawl a portion of a webpage, but I never hear anyone in the SEO community talk about them. http://perishablepress.com/press/2009/08/23/tell-google-to-not-index-certain-parts-of-your-page/

Does anyone use these and find them to be effective? If not, how do you suggest noindexing/canonicalizing a portion of a page to avoid duplicate content that shows up on multiple pages?