Welcome to the Q&A Forum

Seems like Matt and Marcus have you on the right track. With a real-estate site, duplicates and near-duplicates are very common, since you're adding and removing properties all the time and there are many search options and categories. I do agree that search-friendly URLs, long-term, where each property has a fixed URL, are definitely the best bet. In the meantime, though, a solid canonical structure helps a lot.

Ease into it - don't go sitewide in one fell swoop without a plan, unless you're having clear ranking problems. Start with your biggest problem areas, monitor/measure, and work from there. You can always check for indexed duplicates by running a Google search like:

site:daft.ie intitle:"176 Rathgar Road"

In this case, I'm not seeing any index issues, although I think Matt's concerns are valid.

I'd also consider rel=prev/next for search results pages, as that can help focus Google, too. Again, take it one step at a time and start with the biggest problems. It'll mitigate your risk all around.

While I think keyword cannibalization is a valid concern, it's natural for there to be some overlap when a site is clearly focused on a topic. I hate to over-generalize, but my gut reaction is that the home-page and a single category page aren't a big problem. Sure, they'll compete a little, but we're only talking about two pages, and the home-page is generally already stronger. I think the canonical might be artificial here (these aren't true duplicates, and it could look a bit odd to Google).

I'd think more about how the two pages can co-exist - are their certains aspects of the keyword that one more naturally fits than the other? Could you reflect those in the on-page content/tags (TITLE tag, especially)? If the topic merits an entire site, there almost definitely enough facets to it that two pages can live together perfectly well.

Again, don't to over-generalize, but I think you may be worrying too much about this one. If the pages compete in an undesirable way later, you can always add a canonical, but two pages shouldn't be a big issue.

Unless I'm misunderstanding, do NOT 301-redirect. That's going to kill your category page, as 301s impact human visitors as well as search bots.

Dupe-content-wise, you should be fine. iFrames just make me itchy these days, and I've never thought they were good for users, but it shouldn't be a disaster for SEO. The biggest problem is probably just that you're not really getting any SEO value - it's really just direct traffic via a referring site. Granted, it's better than nothing, and I know from painful experience that sometimes you have to take what you can get in these situations.

When you say "embed", do you know what they have in mind, specifically (that word means a couple of specific things depending on the context). If they're just looking to copy the content, then it's important that the link back to you and probably even use cross-domain canonical tags. Otherwise, they'll be competing with you for your own content. It's not just a matter of traffic, but Google could filter out your version of the page or even (at large scale) devalue your entire site. In other words, they could mistake you for the one copying the content, especially if the other sites are more authoritative.

If you're talking about old-school embedding, like wrapping up your content in an iFrame or something like that, I'd avoid it entirely. Those "solutions" are outdated and more trouble than they're worth.

It is common to "embed" some content, like infographics, but those embeds usually have a link back or some clear attribution. If you're just talking about using the content, then I think you're much better off just asking people to use snippets (like a paragraph or two) and then linking to the source.

If you've got a specific example of what someone has in mind, I'd be happy to dig deeper.

Are those URLs (or URLs like them - I realize they're just examples) actually being used in internal links, or are you just saying that they all resolve? The case-sensitivity thing isn't a huge issue, and the canonical tag would work well for that. Otherwise, you'd have to 301-redirect every possibly version (and 98% of them will never be used).

I'd really focus on fixing the internal links first, and then 301 or canonical the versions you used internally (or that have inbound/external links). For the "Default.aspx" version, I think 301s are a little better, but ASPX can be a bit persistent about that, so it's a bit hard to advise. Sometimes, you are constrained by the platform.

The biggest difference is that a 301-redirect will also redirect people, so they'll be more likely to link to the proper version. The canonical tag only impacts Google. Both work reasonably well, though, and do pass on most link-juice if used properly.

I prefer to think of it as "index control", since PR sculpting has a history of being abused, but you've covered the big ones. Obviously, good site architecture is the first step. If they tag exists in 2012, I pretty much covered it in this article:

http://www.seomoz.org/blog/duplicate-content-in-a-post-panda-world

Sorry, not sure what you mean. Site-wide tags, or tags that perform canonicalization?

Hooray! Usually, I just give my advice and then run away, so it's always nice to hear I was actually right about something Seriously, glad you got it sorted out.

Agreed - rel=prev/next is probably more appropriate here. Google prefers that you not rel-canonical a paginated series back to page 1. You are allowed to canonical to a "View All" version, but then they prefer you have a "View All" link for users as well. It depends a bit on how many pages we're talking about and if there are any options that complicate the URLs, like sorts and filters.

I'd generally agree with (and thumbed up) Adam - it's harmless and can sometimes help sweep up any stray URLs. I find it especially useful for the home-page, which naturally has a lot of variants.

I'd only add that you often see this in place not so much because it's strategic but because it's easier to implement, especially in a CMS. Telling the system to add a canonical to every version but the canonical URL is a lot more of a pain, so most people don't do it. Originally, Google and Bing suggested this was their preferred method, but it was so immediately obvious that it's easier to put the tag on all versions that I think they completely reversed that.

I've never seen it cause any harm, and I've seen it help a bit more than once.

I'm going to disagree a bit. While I do think Google understands navigation links and generally views them a bit differently from contextual (on-page) links, there's still a fundamental problem of dilution. If you have 200 navigation links, you split your authority ("link juice") 200 ways, and you're treating the main pages, sub-pages, and sub-sub-pages as if they're all essentially equal from an SEO standpoint. If you prioritize everything, you prioritize nothing.

What this ultimately means is that you drive a bit more ranking power to your very long-tail pages but a lot less to your top-level pages. It's a balancing act and there's not one right answer, but generally this isn't going to be a good fit to your business goals.

I dig into it more in a post from last year:

http://www.seomoz.org/blog/how-many-links-is-too-many

Again, it's not a right-or-wrong thing, but a matter of prioritization. From what you're describing, I'm worried that this could expand to hundreds and hundreds of links, and on a new site that could spread your ranking power pretty thin.

With the canonical tag in place, I'm guessing that extra link would basically be ignored. It's probably harmless, but I'm not sure it will do anything. You could create an HTML "sitemap" (or even an XML sitemap) with the canonical URLs. It's not my first choice, but it at least would give Google an extra push.

I assume you mean an XML sitemap (in Google Webmaster Tools)? The only information that really contains is the URLs on the site. So, if, for some reason, you don't want Google to know about the pages on your site, then I guess a sitemap is a bad idea. If you want your pages to be indexed, though, you're shooting yourself in the foot.

I will say that I think it's important to focus your sitemap and definitely don't use non-canonical URLs or pages you don't want indexed. It should be a "clean" list, so to speak.

Some people don't think you should have a GWT account at all, but I find that to be pure paranoia in 99% of cases. If you're going to rank on Google, they need to know about your site. Even if you're pursuing black-hat tactics, I don't see what information the root account or sitemap would contain that would harm you.

I suspect you're ok, then. I'd watch those GWT numbers, but unless you're seeing problems with indexation and ranking, then I'd just consider that a notice. I think you're handling it by the book, at least as well as currently possible with Google's changing and somewhat mixed signals on the subject.

Sorry, saw the follow-up, but I think the overall thread has you covered. The only real issue with CMS URLs is that you can sometimes have multiple versions pointing to the same page, and this creates duplicate content. There are plug-ins for WordPress that can help with that.

The only exception would be something like an AJAX-style URL, where the page content could change without the URL ever changing (Flash has the same issue, for example). You'll rarely see that in a standard CMS, though, and definitely not in WordPress.

Endorsing Jared for the full thread/follow-up. Unfortunately, when it comes to indexing all of these pages, you can't really have your cake and eat it too in 2012. These pages do look thin to Google - honestly, when the results don't change (and I get that that's just because the filters don't always impact the search), then it starts to look like you're just spinning out duplicates to target new keywords in the header. At high volume, that could get you into trouble (and is the kind of thing Panda has targeted).

You're right, though, if you canonical these pages, they won't get indexed and ranked. These days, my gut reaction is that the trade-off is worth it. If you focus your ranking power, the core category/neighborhood/etc. pages will get more authority, you'll reduce the risks of thin content, and you'll land search users on core pages that they can use to navigate to the options they want.

There's no solution that doesn't involve a trade-off, but I think focusing your index would be a positive trade-off. Keep in mind, too, that Google isn't really that fond of search pages - ultimately, you want them indexing the core property listings. The key is to have clear paths to those listings and to index and ranking prominent category pages. If you try to rank for every variations of ever search/sort/etc., you'll just end up diluting your ranking ability in most cases.

Unfortunately, it can be really tough to tell if Google is honoring the rel=prev/next tags, but I've had gradually better luck with those tags this year. I honestly the GWT issue is a mistake on Google's part, and probably isn't a big deal. They do technically index all of the pages in the series, but the rel=prev/next tags should mitigate any ranking issues that could occur from near-duplicate content. You could add the page # to the title, but I doubt it would have any noticeable impact (other than possibly killing the GWT warning).

I would not canonical to the top page - that's specifically not recommended by Google and has fallen in disfavor over the past couple of years. Technically, you can canonical to a "View All" page, but that has its own issues (practically speaking - such as speed and usability).

Do you have any search/sort filters that may be spinning out other copies, beyond just the paginated series? That could be clouding the issue, and these things do get complicated.

I've had luck in the past with using META NOINDEX, FOLLOW on pages 2+ of pagination, but I've gradually switched to rel=prev/next. Google seems to be getting pickier about NOINDEX, and doesn't always follow the cues consistently. Unfortunately, this is true for all of the cues/tags these days.

Sorry, that's a very long way of saying that I suspect you're ok in this case, as long as the tags are properly implemented. You could tell GWT to ignore the page= parameter in parameter handling, but I'm honestly not sure what impact that has in conjunction with rel=prev/next. It might kill the warning, but the warning's just a warning.

Yes, to set up rel-canonical properly, every page that could conceivably be tagged with a csref= parameter should have a self-referencing canonical. The tags are easy to set up, in theory, but once you get into a large site and/or CMS, setting them up on dozens or hundreds of pages can be tricky. Ultimately, it's a more effective approach that has some other benefits (like scooping up stray duplicates that may have been created by other URL parameters), but it really depends on your development resources and how complex your site is.

It really depends on the platform you're on and the way the page level redirects are set up, but if you list all the rules for the existing pages, you can always add a redirect at the very end. If implemented properly, anything left over should just that rule.

The alternative is to build a custom 404 handler that actually implements a 301-redirect to the new site.

I'd agree with this post, though - if the content really is dead, in some cases, it's better to let it 404 -

http://www.seroundtable.com/archives/022739.html

If you're really starting over, and for pages that aren't very active (no links, very little traffic), it can make more sense to clean things up. There's no one-sized-fits-all answer - it depends a lot on the scope of the site and the  nature of the change.

The good news is that you only seem to have about 8 of these pages in the Google index. You can use this query on Google to see them:

site:www.tryg.dk inurl:csref

Ideally, I'd use the canonical tag on those pages to strip out the parameter and de-index any duplicates, but across the site that can be tricky. You could also tell Google Webmaster Tools to ignore the csref parameter via parameter handling - it's not quite as robust a solution, but it's a lot easier to implement.

Endorsing George for the full thread. Technically, Google does allow relative URLs, but I've heard of some odd issues, so I think it's better to use full URLs. Your home-page version isn't really either an absolute or relative URL - you really should have the "http://" (protocol) in that URL. If you're being nitpicky, that's an improper URL, and Google could end up interpreting it as something like:

http://www.ccisolutions.com/www.ccisolutions.com

Now, odds are, they won't, but with these tags it's really best to do it by the book.

The AJAX URLs are used by the site, though, right (for visitors)? If you 404 them, you may be breaking the functionality and not just impacting Google.

Another problem is that, if these pages are no longer crawlable, and you add a page-level directive (whether it's a 404, 301, canonical, NOINDEX, etc.), Google won't process those new instructions. So, they could get stuck in the index. If that's the case, ti may actually be more effective to block the "ajax=" parameter with parameter handling in Google Webmaster Tools (there's a similar option in Bing).

If you know the path is cut and this isn't a recurrent problem, that could be the fastest short-term solution. You do need to monitor, though, as they can re-enter the index later.

Apologies, but I'm not familiar with the csref parameter - could you tell me what information it passes or give me a sample URL (you can make it generic and mask your domain info)?

It sounds like some kind of tracking code, in which case it can definitely start to create duplicate content issues. You could probably use the rel=canonical tag to make Google "collapse" those pages, or you could tell Google to ignore the parameter in Google Webmaster Tools. Neither should impact your tracking.

I agree with Ben - sitewide links are not inherently bad - low-value sitewide links (like footers loaded with junk) or excessive sitewide links that dilute your link value are bad. If you have a handful (10 is a good number) of popular posts in the sidebar, it'll be seen more like navigation, and it can definitely be beneficial for those posts. It's also good for visitors. This is a common practice and definitely not one to get you penalized.

William's note could be applicable, depending on your application. If these are non-standard links (like JavaScript-based), then it's a moot point.

Is there a crawl path to them currently? One issue I see a lot is that a bunch of pages get indexed, the path is found and cut off, NOINDEX (canonical, 301, etc.) is added, but then the pages never get re-crawled. Since they don't get recrawled, the page-level directive never gets honored.

If there's a URL parameter involved, you could use parameter-handling in GWT - it's not a perfect solution, but it sometimes seems to work without a re-crawl.

The other option would be to create a new XML sitemap with all of the bad/indexed URLs. This may push Google to re-crawl them and then see the tags to deindex. It's a bit safer than re-opening the crawl paths.

If they are being crawled and Google is just ignoring the NOINDEX for some reason, I'd try to 301 or canonical those pages to a primary search page, if that's feasible (probably canonical, since you don't want the users to 301). Sometimes, if a signal isn't working for that long, you just have to shake Google and try a different signal. Even following their exact recommendations, it rarely works as planned at large scale.

Especially post-Panda, duplicates can create a real mess. At best, it's a matter of dilution. The more pages in you have in Google's index that are "thin", the more thinly your internal link-juice (authority, basically) is spread. So, each page just gets less of it. In extreme cases, though, the entire site can suffer.

Canonicalization is tricky, and it's tough to be 100% sure from sample URLs, but my gut reacionts:

(1) Yes, I think you could safely use rel=canonical here. It's slightly odd, since these search pages are actually showing different lists of products, but your only real choices are rel=canonical or blocking the "prices=" parameter in Google Webmaster Tools. You could NOINDEX anything with "prices=" in it as well. I think canonical will work, though.

(2) This is definitely a case where you should use rel=canonical. There are true duplicates. Actually, the best case here is not to create these URLs, but I realize that's not always an option.

You could use GWT for #1, if development is an issue, but to solve (2) you're going to need some kind of page-level directive (like rel=canonical). There's no good way to get around the coding.

It's hard to gauge the impact, but I've definitely seen cases where the consequences of large scale duplicates were severe, and where large ranking/traffic improvements (as much as 3X, although it's not usually that dramatic) have occurred when the problem was fixed. To be aware that it's not instantaneous. It can take a few weeks to really see the impact.