Welcome to the Q&A Forum

Thanks for the quick response Dana!

They are sourcing us but at the moment they're constantly showing up instead of our site. There's no reason they should outrank us, but we figured they couldn't really be hurting us since they are linking to us.

It's weird that the site is able to scrape content and still get good indexes... maybe Google just hasn't picked up on it yet.

We are hoping that we don't have to worry about scrapers once our blog posts start getting indexed like they used to.

I think we'll send them a friendly email like you suggested.

Our Google traffic has dropped significantly over the last year and now we're struggling to even get our blog posts indexed. It's been extremely discouraging and we're trying to do what ever we can to fix it.

I've included a screenshot of our Google traffic as well as Pages Indexed according to WebmasterTools.

http://i.imgur.com/Wu1D8.jpg

The Problem

• Our blog posts are frequently not getting indexed.
• Many times they are outranked by low authority scraper sites, our Twitter/FB account, etc.
• Sometimes our homepage will rank instead of the blog post.
• Sometimes we'll break a news story, get tons of quality backlinks, and still be nowhere in Google.
• Pretty much the only Google traffic we see is from existing posts.
• Still 3,200 pages indexed when we have only 1,600 posts. I guess this isn't really a problem... just waiting for the meta noindex to take effect.

More details

• We've seen no duplicate content or other warnings from WebmasterTools.
• We've been constantly acquiring quality backlinks from credible sites.
• We deleted the useless content and fixed the canonical issues that were a result of switching servers.

History

Our site is a news/entertainment blog. The traffic usually has spikes depending on what's going on in the news.

• Nov 1, 2011 - Site kept maxing out at 30k+ visits so we switched servers.
• Jan 30, 2012 - Hired a writer so we could focus on other aspects of the site.
• Apr 19, 2012 - Noticed our posts weren't getting indexed like they used to. Suspected our writer was spinning articles but couldn't find any evidence. 90% of our blog posts were nowhere to be found in Google. Scrapper sites would outrank us for our own stories... even our Twitter account was ranking ahead of us. IF our story would show up in Google it would usually be the home page instead of the blog post.
• Sep 2012 - Finally got more serious about addressing the problem. Noticed a couple potentially big problems and started making changes.

Canonical Issues

1. non-www site didn't redirect to www. It showed 2 different link profiles according to OpenSiteExplorer and 0 backlinks according to Webmaster Tools.
2. Wordpress shortlinks weren't redirecting to the actual permalink. For instance http://www.domain.com/?p=123 and http://www.domain.com/post-example were both getting indexed.

For every post there were 4 different versions that Google had to choose from.

http://domain.com/?p=123, http://www.domain.com/?p=123, http://domain.com/post-example, and http://www.domain.com/post-example

I figured the canonical issues must have happened when we switched servers which was the reason for the drop in WebmasterTools indexed pages and increase in Not Selected pages.

FIXED (Sep 15): One we fixed the canonical issue the Indexed Pages went back up however the Not Selected is still the same.

Duplicate Content

When we first created our site we wanted to have tons of images for each musician/athlete/actor/etc. so we uploaded about 5-10 for each person. We created a blog post for each image with no writing and the exact same post titles. As a result there were TONS of low-quality, similar posts, with virtually identical permalinks. e.g. http://www.domain.com/james-smith1, http://www.domain.com/james-smith2, http://www.domain.com/james-smith3, etc.

A crawl on Sep 26 showed over 550 duplicate content warnings.

FIXED (Oct 1): We deleted/301 redirected the useless pages (they weren't getting traffic anyways) and by the next crawl the number was almost to 0... which it's at now.

We also had TONS of tags (since there're constantly new names in the media) that were getting indexed so we had meta robots noindex them.

Questions:

1. Why aren't a majority of our posts getting indexed?
2. Were we penalized or just stuck because of a filter?
3. How long should it take for meta robots to noindex the tags pages? (I did it on Sep 25 but they are still there)
4. If a site is scraping our content (same title, image, excert) but linking to us, should we contact them and tell them to remove it?
5. Is there anything else we need to do start getting our blog posts indexed like they used to?
6. Should we try contacting Google to re-evaluate our site?

Sorry, that was a LOT of writing. If anyone wants the URL please let me know so I can PM it to you. Any help would be greatly appreciated!

Wu1D8.jpg

How long did it take for the tags to drop from Google's index?

I have a site with about 800 tags indexed. Even after I did no-index follow, they're still showing up after a month. Do I have to wait it out... or could it not be picking up the noindex?

Thanks for the responses! The pages just got noindexed today... it took almost exactly a week.

Is it possible that the Analytics script was altered or removed? Wouldn't it at least be showing some direct traffic?

It's showing 156 pages indexed in Google and when I searched a couple of your page titles your site showed up at the top. I would monitor the search queries in WebmasterTools and see if anything looks different.

Be careful about indexing the category and tags archives since they can cause duplicate content... I learned the hard way

I have a Wordpress site with about 3,000 pages and 1,000 of those are no value/duplicate content and drive no traffic. They are blog posts each with a single image and permalinks like example.com/post1, example.com/post2 etc.

I've started by deleting pages and 301 redirecting to relevant pages that actually have content.

• Is deleting and 301 redirecting the best route?

• Is 1,000 to many 301 redirects?

• Should I just delete the pages that aren't really relevant to anything else?

• Anything else I should know about deleting all of these pages?

Any help would be great!

I have a wordpress site with about 3,000 posts and over 1,000 tags. All of the tag archives are currently indexed in Google and I don't want them to be.

I just set the meta robots to no-index all the tag archives and was wondering how long it will take til they're out of the search engines?

Since there are close to 1,500 of these and they are duplicate content it would be nice to have them gone asap.

I noticed Webmaster Tools allows me to resubmit my site to index if my site has changed significantly... should I try that??

Any other advice would be greatly appreciated!