Welcome to the Q&A Forum

Anthony's definitely got the basics covered. How to handle any particular situation can get pretty tricky. I wrote a post about it, and that post got very long very fast:

http://www.seomoz.org/blog/duplicate-content-in-a-post-panda-world

I took a quick look at your campaign (I have Staff access), and it seems your login page is carrying an event ID - so every event is creating a different URL, but they all land on one page. That could spin out 100s of duplicates on Google, and that login page has little or no search value. A canonical tag would definitely be a good bet here.

You may have other issues going on, but clean up one at a time - getting that number down can definitely be beneficial and help boost your ranking power.

I just want to be clear - this is isn't one of those situations where I'm 60% sure from my general, anecdotal experience (I try to be up-front about that). This situation is exactly why the canonical tag was invented. Unique URLs that can be crawled are unique "pages" in Google's eyes.

Now, does that mean the URLs are necessarily causing you problems or that the canonical tag will improve things substantially? That's a much tougher question, and depends a lot on your site and any other technical SEO problems you may have. I can tell you with total confidence that I've seen it cause serious ranking problems many times. It's relatively painless and risk-free to fix, so I'd fix it 99% of the time.

If you look at Google's reference on the canonical tag:

http://support.google.com/webmasters/bin/answer.py?hl=en&answer=139394

...you can see that the first two URLs differ only by the "sort" parameter. Yet, to Google, these are different pages. In this case, the pages may even vary slightly, but they're close enough to be seen as "near duplicates". On most of the URLs you've listed, the content seems to be identical, as far as I can tell. Only the URL is changing, so that's a very textbook duplicate content problem.

The difficult part is just how best to implement a solution in WordPress and sorting out why all of these various parameters are being crawled. There are other solutions, like Google Webmaster Tools parameter handling. It's a complex problem, and I've got a mega-post (probably the longest post I've ever written by 2X) here that shows just how tricky it can get:

http://www.seomoz.org/blog/duplicate-content-in-a-post-panda-world

You've got a few variants, and it's always best to try solve the issues that caused them - often, this comes down to fixing internal links. In this case, though, I think properly implemented canonical tags would clean up a lot of the mess and help prevent future problems.

I forgot that I wrote a post about this recently:

http://www.seomoz.org/blog/which-page-is-canonical

Kyle is right, and this is a common misunderstanding. While we think of a "page" as a an actual file, Google (and pretty much every crawler) sees a "page" as a unique URL. If content is accessible from multiple URLs, then search engines view that as multiple pages. They can't see your physical files, so they don't have a good way to know what's actually a page and what isn't. In fairness, it's even more complex than that - often single code templates will drive unique content (a page like "product.php", for example, might generate 1000s of unique product pages).

So, in Google's eyes, all of these URLs are indeed duplicates. You can see multiple indexed copies by running this command on Google:

site:hawaii-aloha.com/blog/page/4

It's possible the site was even hacked in the past. There are some weird indexed URLs, like:

/?id=sbfzcjblmfjqjon

...that Google lists as "compromised". I'm not seeing malware, but it's unclear how these URLs got crawled.

You do need some sort of canonicalization solution. I'm not a WordPress expert, but there are some decent plugins. The canonical tag actually can work here - even though it's one file, every time Google crawls a different URL (landing on that same file), the canonical will consolidate those URLs. Since this seems to be paginated search, it can be a bit tricky, though. I would highly recommend addressing the problem, as all of these copies can definitely dilute your ranking ability and hurt your site's SEO.

It's a little funky, but a couple of things I'm seeing:

(1) Some of your internal links (your catalog links) have the special character code (&) for the ampersands. Although browsers can resolve it, those are definitely not appropriate for URLs in href="" values and could be disrupting internal link-juice. I can't believe it's the whole story, but it isn't helping.

(2) Having your major categories 302-redirect is definitely not a good bet. Unfortunately, I also don't think it's the entire story. You're not even ranking for very specific brand terms, and the home-page is clearly indexed. I would fix (2) ASAP, but I don't know if it's the whole story.

(3) You've got the blog off on a sub-domain with a few indexed pages, but it seems to have no content. Unless you're going to ramp that up, it's probably counter-productive.

(4) I'm seeing some weird data in OSE and found that all of your non-www pages are 301-redirecting to the home-page, instead of the appropriate URL. For example:

http://metalsforasteel.com/Catalog.aspx?Level1=01&Level2=17

...this should go to a category page, but instead goes to the home-page. If you have links to the non-www version, this is potentially a problem. Add this to (1), (2) and (4) and you're compounding the mess.

(5) You have a fair chunk of links from low-value sources. I won't say spammy in all cases, but free link directories, link exchanges, etc. I don't want to list a bunch of examples, since this is a public question. You're teetering on the edge a bit, quality-wise, I suspect.

So, can I tell you definitively which of these it is? Unfortunately, it's nearly impossible to completely separate them. I think you should tackle (1), (2) and (4) ASAP. Fix the internal problems, because then the picture will be MUCH clearer. I'd definitely take a look at your link profile and make sure there's nothing too questionable - you may have to tackle that at the same time.

Haha, yeah - I imagine it's something like "Average Position is the average of the keyword positions over time. Also, see 'Mean Position'."

Yeah, that is weird. I'm honestly not sure if average position is by unique keyword or keyword volume. In other words, maybe you gained position on a couple of head terms that produce a ton of impressions, but lost position on a bunch of long-tail terms. So, average position across unique keywords would fall, but you'd still gain traffic, because the loss was mostly on low-traffic keywords.

You'd have to dive into the keywords and/or segment them (head, long-tail, etc.) to see for sure.

Unfortunately, what Google can do and what they actually do can really vary. If you're mixing signals and doubling up the canonicals, the odds that they'll get it right are pretty low, in my experience. I think it's worth the fix, if these are traffic-generating pages.

It's a little tough to tell without seeing the URLs in question, unfortunately, but it is possible, yes. When you say "has virtually gone missing", what do you mean, precisely? It usually boils down to a continuum:

(1) Not indexed

(2) Indexed but not ranking

(3) Ranking but poorly

A bad 302 setup is more likely to cause (1) and maybe cross into (2) then (3), but it really does depend. The first step is to see how you're being indexed. The other question is if the 302 is creating duplicates or causing crawl/index problems - for example, are both versions of the URL being indexed (that could also be bad)?

If the correct URL is being indexed and it's ranking, but it's just not ranking very well, then I suspect you've got more going on than the 302s. If nothing else, fixing the problem will make the situation cleaner and easier to diagnose. If you add technical problems on top of other problems (like link profile issues), sorting out the mess can be extremely difficult.

Yeah, I'm honestly not 100% sure on the HTTP header version, but I'd bet they don't support it. It won't hurt to try it, though, and you'd at least cover Google - I think it's probably a good best practice for PDFs that have HTML equivalents.

I'm not seeing any evidence that Google is indexing multiple versions, and your canonical tags appear to be correct. Practically speaking, I think you're ok here.

I'd make sure that you're at least linking consistently (internally) to the "/" version and not both the "/" and no-slash version. If you're using both links, it could confuse crawlers (and our tools). Your search parameters should be ok, though. I don't see any evidence that those have been indexed. See this query:

site:lamangaclubresort.co.uk inurl:datefrom

My short-term goal is to provide some at-a-glance drill-down metrics (like domain diversity or EMD influence) on the Mozcast site. That way, when something happens, people can immediately  look at half a dozen metrics and see if there's a pattern across the data as a whole. No ETA on that yet, but hopefully within a few weeks.

Definitely some turbulence at the end of July, and around Panda 3.9. If you haven't seen it, keep an eye on our new project, MozCast:

http://mozcast.com/

I don't believe that Bing supports the HTTP header version of rel="canonical". They do technically support the link attribute (their comment about it being a "hint" was from 2009) - Duane confirmed that last year (I asked him point blank). Although, honestly, experiences vary and many SEOs claim that their support is inconsistent even for the link attribute.

Honestly, when it comes to canonicalization, when in doubt, try it. The worst that can happen in most scenarios (implemented properly) is that it just doesn't work.

Out of curiosity, why are your trying to use the HTTP Header version. Is it a non-HTML file (like a PDF)?

Truthfully, implementing these tags properly is way too complex, and I'm not thrilled with Google for how this solution has been structured. Stuart's absolutely right, though - rel=prev/next aren't being used properly on the current site. If you're mid-redesign than probably best to do it right on the new site, as it's not a catastrophe.

Unfortunately, we're not sure yet. We've had some strange data discrepancies in the past week. Long story short, there are two weather "stations" (A and B) - they usually run very closely in sync, but they've diverged the past few days, peaking on different days.

At first glance, it seems like Panda 3.9 rolled out over multiple days, like Panda 3.7 did, and may have masked some other updates. Oddly, the official roll-out day of Panda 3.9 was one of the lowest-flux days in that group. A similar pattern has happened three times in the past two months, and it's unclear why. Some updates, like Penguin 1.0, seem to be one-day, single-shot events. Others get released and adjusted over a few days.

We'll be trying to provide more analysis going forward on the MozCast Twitter feed (@mozcast) and possibly here on the blog for major events. Happy to field questions here as well, though.

I should also note that our tools don't handle rel=prev/next very well yet, so we're sometimes over-sensitive to duplicates. One you implement those tags, don't worry if it isn't immediately reflected in our crawl - something we're looking at going forward.

Unfortunately, I'm not a WordPress expert by a long shot, but I'm seeing what Stuart is seeing - no rel=prev/next. I think that probably is one of the safer bets these days. Your site isn't huge, but the paginated archives pages straight from the home page could dilute your index a bit and lessen the ability of your individual posts to rank well. I don't think it's a disaster, but testing out Joost's plugin probably is a good bet.

It's really tough for us to tell what those parameters do, so you have to be careful. Just because a URL has certain parameters doesn't mean it's a duplicate. If these really are duplicate or near duplicate pages, you've got a few options:

(1) Consolidate them with rel-canonical

(2) Block them with META NOINDEX (Meta Robots)

(3) Block/ignore the parameters in Google Webmaster Tools

It's a complex subject, and I have a mega-post about it here:

http://www.seomoz.org/blog/duplicate-content-in-a-post-panda-world

I'm not a Magento expert, though - Joost de Valk has some resources worth checking out:

http://yoast.com/articles/magento-seo/

Yeah, wish I could give you a simpler answer, but I'm afraid it might end up being a little tricky. Hit the biggest problems first, and at least you can manage time/money a bit. The one bright side is that the rules should be no harder to code in ColdFusion than anything else (PHP, ASP, whatever). It's just the core logic that's tricky.

Unfortunately, the rules may differ from page to page and will be entirely dependent on how your pages are generated. If it's just a matter of the "index.cfm" version vs. root ("/") versions of pages, those canonical should be straightforward. For the other parameters, though (like "i", "fs", etc.), it depends entirely on the function of those parameters.

I know ColdFusion reasonably well, and even given that, I couldn't give you a one-size-fits-all rule that would solve the problem. It really has to be guided by your site structure and code/data logic. Personally, I'd start with the pattern that generates the most problems and solve that one first. In other words, if one template (like "/press-releases") generates dozens or hundreds of duplicates, deal with that first. If you solve the top 3-4 problems, you may clean up quite a bit. That could be more effective than trying to fix everything at once.

Seems like you got the 301-redirect resolved below - if you've got that in place and fixed the canonical tag, it should be ok. It'll just take some time (usually longer than you'd like) for Google to clear out the pages, especially the deeper ones. If you see gradual de-indexation, though, you'll probably be fine.

Could you provide an approximate example that matches your real situation (a fake domain is fine, but with the same basic format)? This is a situation where fake examples that don't match the real situation probably won't help us (or you) much.

Once you have the spreadsheet, how are you going to translate that into tags? If this is a dynamic site, it would be better to be able to code rules to set the canonicals - and potentially much easier.

Technically, rel=prev/next doesn't de-duplicate the way the canonical tag does, but it should solve any problems for Google. I don't believe we currently consider rel=prev/next when determining duplicate titles. Klarke is right - you could just give those pages semi-unique titles. We're not handling rel=prev/next as well as we could be (it turns out to be a tricky tag to parse well).

Looking at your pages, your implementation appears to be correct. My gut reaction is that your probably ok here. You're doing what Google claims they want (at least what they want this week).

Could you clarify a couple of things:

(1) When you say canonical URLs, do you mean your internal links, or the actual URLs in your rel="canonical" tags?

(2) If it was just the canonical tags, is everything consistent now (tags, internal links, etc.)?

Since both version will resolve, just fixing the canonical tags (if that's the issue) should be enough - it's just going to take a little time. They should be as effective as a 301-redirect in this case. Either way, though, it can take Google a while to kick out the duplicates. I'd just monitor the index closely and make sure the top-level pages are clearing up (i.e. your home-page and major category duplicates should be disappearing). If that's happening, you're ok - you just need to wait a bit. If that's not happening, then you may have some other mixed signals in play.

Rand had a good post on this problem (and it's fairly common) a while back:

http://www.seomoz.org/blog/wrong-page-ranking-in-the-results-6-common-causes-5-solutions

I think Francisco has a great point, though - from both a search and user perspective, is the page necessary? If it's really covering all the same keywords, it may be telling you that Page B just doesn't have the value you original thought.

Generally speaking, though, your home-page is going to have more ranking power. If it's just two pages, the worst that happens is that your home-page outranks Page B, and Page B doesn't get much SEO love. If it's still good for users, that's fine. If you really want Page B to rank, then you may have to examine it's unique value and whether the home-page should be more general or target broader terms (or brand terms) instead.

I agree with Matt - as long as your primary, internal links are consistent, it's ok to use a short version for offline purposes. The canonical tag is perfectly appropriate for this.

The other option would be to use a third-party shortener that has built-in tracking, like Bit.ly. It uses a 301-redirect, but also captures the data. If you're just doing a test case, this might be easier all-around.