Welcome to the Q&A Forum

Sorry - I didn't mean to sound like I think you're lazy or creating low-quality content. It's just that I see this stage as a natural evolution in a site's success, and a critical decision point. When the low-hanging fruit is gone, you have to start climbing, and that's going to take some more effort.

I just want to say that every big blog started off as a small blog. Granted, if I post on SEOmoz, I get a lot more attention than other places, but SEOmoz started as Rand's blog and built up over years. My own company blog has about 1,000 subscribers (not SEOmoz's 100K), but I've had some major content marketing success over the years, and some of those still generate traffic/links 2-3 years after they were written.

In most cases, those successes not only took an investment in the content (I estimate about 20-30 hours for a really good piece of content), but it took pounding the pavement. I had to build relationships online, find my audience, and generally promote myself and my content. That goes well beyond just posting on FB/Twitter. You have to seek out the people who are the right audience for that content, even if it's one by one at first. It's tough, but once you've built that base of relationships, they'll keep paying dividends.

I like David's approach. I'd just add that you probably will lose some impact of the exact-match domain, but, on the other hand, if you get 10 visitors/day, that page-1 ranking on the micro-niche site really isn't all that valuable. All else being equal, I'm a fan of consolidation these days.

One tricky point - you don't necessarily want to fix the duplicate URLs before you 301-redirect and clear out the index. This is counter-intuitive and throws many people off. If you cut the crawl paths to the bad URLs, then Google will never crawl them and process the 301-redirects (since those exist on the page level). Same is try for canonical tags. Clear out the duplicates first, THEN clean up the paths. I know it sounds weird, but it's important.

For malformed URLs and usability, you could still dynamically 301-redirect. In most cases, those bad URLs shouldn't get indexed, because they have no crawl path in your site. Someone would have to link to them. Google will never mis-type, in other words.

I'd some point you have to start going for the harder links, and that really means content marketing, pushing social forward, and building industry relationships. The lists and competitive analysis will get you most of the low-hanging fruit, but then you do hit that wall.

Personally, I find I never run out of sources with content marketing. There are always new people to discover your work. The great thing about content marketing and building strong social profiles is that they're self-perpetuating. I've got posts that are 2-3 years old and still get new links, without me doing any additional work. It's more work up front, but it pays off by 10X-100X in the long run, IMO.

I DO NOT believe in letting Google sort it out - they don't do it well, and, since Panda (and really even before), they basically penalize sites for their inability to sort out duplicates. I think it's very important to manage your index.

Unfortunately, how to do that can be very complex and depends a lot on the situation. Highland's covered the big ones, but the details can get messy. I wrote a mega-post about it:

http://www.seomoz.org/blog/duplicate-content-in-a-post-panda-world

Without giving URLs, can you give us a sense of what kind of duplicates they are (or maybe some generic URL examples)?

I think Highland covered the basics - it will be seen as a sitewide URL change, so you need to make sure those 301s work. You could lose some small amount of link-juice, and expect rankings bounce in the short-term (although it should recover 98%). Also, it can add a lot of overhead to a site, so make sure your servers are up to it.

Technically, Google says it's fine. They don't treat secure pages any differently than "regular" pages. Practically, I'm not a big fan, to be honest - it's adding a layer of complexity. Still, there are big sites that are secure and rank fine.

I think Harald and James covered the bases here, but a couple of comments on Harald's reply:

(1) Definitely check this. A common cause of indexed https: pages is that a secure section of your site is being crawled (like a shopping cart), and you're using relative navigation links (like ) - when a crawler or visitor hits the nav link from a secure page, the relative link grabs the https: In most cases, you may want to NOINDEX secure pages. Shopping carts and checkout pages have no business in the search index, IMO.

[(2)-(5) I believe this does work, but it's very tricky, so please be careful. If anyone has linked to the https: pages, you'll lose the link-juice this way (you'll just cut those pages off). I honestly don't think it's a good choice for most sites.

(8) I actually believe the 301-redirect is simpler in most cases.

As James said, sitewide canonical tags (or on the affect pages, if they're isolated) will also work.](/contact.php)

Sorry, by "bigger problems" I just meant the potential link-farm.

The nofollow will remove the SEO risk - you'll still lose a little link-juice to those links, but you won't get penalized down the road for having them. Of course, you won't gain any SEO value from the cross-linking either. At this point, though, I think that's inevitable. The risk is greater than the reward from cross-linking this many domains.

Any other ways to block the links are going to look more suspicious to Google than nofollow (including iFrames). Any I can think of would be best avoided in this scenario.

Any way you can contextually cross-link would create less SEO risk and potentially let you get some ranking value out of the connections. That's why I suggested links at the job listing level. I think that might benefit users a bit more, too. Even then, you don't want to go overboard.

Honestly, with only 235 indexed pages, it's pretty doubtful that duplicate content caused you an outright penalty (such as being hit with Panda). Given your industry, it's much more likely you've got a link-based penalty or link quality issue in play.

You do have a chunk of spammy blog comments and some low-value article marketing, for example:

http://undertheinfluence.nationaljournal.com/2010/02/summit-attendees.php

http://www.selfgrowth.com/articles/der-bestebesterhoechstermaximalerneueroptimaler-party-poker-bonus-code-im-internet

A bit of that is fine (and happens in your industry a lot), but when it's too much of your link profile too soon, you could be getting yourself into penalty territory.

I'm not seeing that Google is currently indexing either of these pages, so they may be too deep or duplicated in other ways. Pagination is a tough issue, but in general pages 2+ have little or no search value (and, post-Panda, can actually harm you).

I would strongly recommend NOT using a canonical tag to page 1 - Google generally advises against this. You can use rel=prev/next, although it's a bit tough to implement and isn't honored by Bing. Generally, I'd advise one of two things:

(1) META NOINDEX, FOLLOW pages 2, 3, etc. - they really have no SEO value.

(2) If you have a View All page, link to it and rel-canonical to view all. This seems to be accepted by Google, but then the larger page will rank.

Generally, I find (1) easier and pretty effective.

Sorry, just saw Nakul's comment, and didn't realize you already have canonical tags in place. While it's not preferred solution, since it's already there and seems to be keeping these pages out of the index, I'd probably leave it alone. It doesn't look like Google is indexing these pages at all right now, though, which you may need to explore in more depth.

It's a complicated issue, but adding 50K variations to 27K product pages can definitely be dangerous, especially post-Panda. At best, you're diluting your index and your ranking ability. At worst, Google could actually start de-indexing or at least devaluing core pages. Personally, I don't think the long-tail gains are worth the risk - these kinds of pages were behind the "May Day" update in 2010, and Panda continued that core philosophy. Google considers it a low-value tactic in 2012 - of that, I have no doubt at all.

Of course, it does depend on how you use them. To have custom landers for PPC and not index them is perfectly fine, for example. If you're tripling your indexed page count with thin content just to target SEO keywords, though, you're taking a very real risk, IMO.

Be careful changing your sitewide URL structure, but if you're going to show users the "url-with-extension" version, make that canonical. It's dangerous to use one version for users and another for search, and people will start linking to the one they see.

I'd also 301-redirect the rewrite (add "[R=301]" to the end of the rewrite rule) and link to it internally. If you have one "canonical" version for bots and another for visitors, you'll end up with a mess.

First off, a slight word of warning. When you spin out the custom landing pages, make sure they have unique content and don't do it in large numbers. It used to be that these kind of long-tail, keyword-targeted pages could help SEO (or, at worst, not hurt it). Since Panda and Google's attack on thin content over the past couple of years, these pages can actually cause you SEO harm. It depends a lot on the quantity and quality, of course. If you spin out 500 pages on a 50 page site just to target a bunch of keywords, and those pages only differ by a sentence or a few words, you're going to do more harm than good.

I doubt the two URLs you list would be much different. Theoretically, the shorter URL will focus more keyword power on "silver fish", but URL keywords are just one, relatively weak ranking factor, and you're talking about 4 characters.

You could use a hash-tag style URL, like:

www.domain.com/silver-fish.html#clp

I think those characers would be ignored by Google. Unfortunately, you'd have to modify your analytics to read them (as they'll be ignored by most analytics packages, too). Here's an article on how to do it in GA:

http://www.searchenginepeople.com/blog/how-to-track-clicks-on-anchors-in-google-analytics.html

That's a pretty technical feat for something that I doubt would have much impact, though.

You've got bigger problems here than 100 links/page (that's just a guideline) - you're cross-connecting enough sites that it looks like a link farm. Having them all in the footer only adds to the problem, and makes the tactic look lower-quality. I'd definitely no-follow these, as you could potentially be penalized.

The nofollow won't really help with the 180 links - it'll still burn up link-juice. It'll just keep these links from getting you into trouble. Realistically, these links are probably already being devalued by the algorithm.

Practically, being in the footer, these links may not have a ton of value for visitors (if you click-mapped the page, I'm betting the CTR is very low). I wonder if there's a way to integrate them. For example, when someone clicks through to a job listing in Kent, having something like "See more jobs in Kent" on that page and link it to: http://www.kentjobsonline.net/.

From an SEO standpoint, these geo-targeted microsites have lost a lot of value over the past couple of years. I've even seen the strategy run into Panda issues. You may want to re-evaluate down the road and consider consolidating.

Ideally, you'd fix the crawl path, but that may be tricky (unless they've patched the CMS). You could add the canonical to just the "page=1" version, but admittedly that's a bit code-intensive.

An alternate idea - that is fairly Google-friendly. You could add a "View All" version and then point the canonical on all search pages to that version. Especially since all is only 2 pages, that could work well in your case and you wouldn't have to worry about all the variants or search results not getting crawled.

I think our system has roughly a 90-95% threshold for duplicate content. The pages I'm seeing in your campaign don't look that high, so something is up - I'm checking with support.

For now, use the "Duplicate Page Title" section - that'll tend to give you exact duplicates. The duplicate content detection also covers thin content and near duplicates.

I want to expand on what I think Istvan was trying to say. First, the canonical on "products.php" will consolidate all of the affiliate IDs. That should be a perfectly valid solution here.

The only warning is whether you have other IDs on that page that drive different product views. If it's just one single product page, then the canonical is great here. If, however, you have something like:

http://www.mysite.com/products.php?prod=1&ref=12345

http://www.mysite.com/products.php?prod=2&ref=12345

...where "prod=" (or something like it) represents separate products, then a canonical tag to "/products.php" would collapse ALL of your product pages into one. That's certainly not what you want. So, it does depend a lot on the details. In that case, the "?prod=1", etc. version would actually be the canonical version (you'd have to set the tag dynamically).

(2) is a calculated risk. Yes, you'll knock the deeper pages out of contention for ranking, but they're low-value pages (in Google's eyes) and near-duplicates. So, you should boost the rnaking ability of the main Armani brand page. Net-net, I think you'll gain branded searches.

Truthfully, though, 15 brands X 8 pages isn't a ton. You may be having ranking issues beyond this. It is common for these pages to do poorly, as they're search results and tend to use copy that's used across the web (since many people sell the same brands). There's a lot of competition.

I doubt it would harm you, but it is a bit unorthodox. Why not make the table header a header tag?

I'd argue with the "take up space unnecessarily" point a little. Headers aren't just for SEO (their SEO impact is probably pretty small these days) - they're for visitors. If these pages get direct visits (including from search), a prominent header can really help people know their on the right path. Breadcrumbs are great for people who are already on your site and have a sense of it, but they're too small and complex for that 5-second test of "Am I in the right place?"

I notice you're a PRO Member - have you submitted this to Private Q&A? We could at least take a look at the site and see how bad the problem looks. Fixing it may still require a WordPress consultant, but diagnosing the severity of the problem is within the scope of Private Q&A.

It looks like you're 301-redirecting you're root site to the "/index.php" version of the page. The problem is that you're linking to the root "/" in the navigation and most of your inbound links are probably to the root. So, you're sending a mixed message about which version is actually canonical. I would stick to the root, personally.

The non-www vs. www issue is also in play, but I think it's a secondary problem.

While it's true that big brands get away with more than other sites, I think it's a lot more complex than that. Big brands also have massive link profiles and high authority, which cushions a lot of other problems. Some big brands actually were hit by Panda, although to varying degrees.

Driving brand searches alone isn't going to work for most sites. Google is taking more offline factors into account for what constitutes a brand, I'm willing to bet. They're also looking at overall volume, authority, social activity, etc. It's not going to be easy to make yourself look like a big brand online if you aren't in "real" life.

If you want to build your brand, that's fine, and I think it's certainly good to do things like use some branded terms in anchor text (just for diversity). It's also, of course, a business decision as much (or more) than an SEO one. There's value to brand-building. I just wouldn't treat it as a cure for SEO problems. There are plenty of easier and more effective ways to prevent algo-related issues. A lot of Panda comes down to content quality, aggressive advertising, and site architecture.

Unfortunately, these things are really tough to diagnose from generalities (you have to be able to dig in and see what's happening on the actual site). Going from the non-www to "www" version should be ok, but any full-scale change can cause short-term rankings bounce. A couple of questions:

(1) Did you only change from non-www to www or did you make other URL changes? Make sure you've mapped every major URL to a new URL - it's easy to miss some.

(2) Are the 301s working and are they single-hop? Test some individual pages with a header checker, like this one:

http://tools.seobook.com/server-header-checker/

I think our Firefox toolbar will do it, too.

(3) Are the pages we show as 404s returning 404s (again check the headers)? Should they be? If you have legit pages returning 404s and 50Xs, you've got a problem. Unfortunately, why you're getting 404s is almost impossible to tell from the outside - it's specific to your server/platform.

Interesting - I've very rarely had issues with GWO, but if a new URL was created and someone linked to it, I can see where you might have a problem.

(1) None of these things are absolute, I'm afraid, but typically, yes - a rel=canonical to a different page should keep the first page out of the index.

(2) Usually, but it depends. The problem here may be that Google just isn't crawling the test variant very often, so they may not be processing the rel=canonical yet.

If it's just a couple of pages, I'd give it time - it's probably not an emergency situation. Again, you could just tell Google to remove them in GWT. I think you're doing the right thing with the canonical tags, but it can take Google time to process them the way you want to, in practice.