Welcome to the Q&A Forum

It can take a while. I disagree very slightly with Alan and EGOL on one point - while 301s are traditionally more appropriate here, I often find that canonicals are pretty strong (and more than a hint). Both suffer the same problem, though - the signal has to be crawled and processed, and that doesn't always take right away. I haven't seen any reports on it taking 2, 3, etc. times to happen, but I've definitely seen a page re-cache without the indexation signals beign honored.

Are these true duplicates or did something change in the interim a bit? If the duplicates don't seem like true duplicates or you put 1000s of them out there all at once, Google could choose to ignore the canonicals.

If these really seem stuck, though, switching to 301s is harmless, and for a permanent URL change, it is probably the better way to go. I wouldn't expect that to kick in instantly either, though.

I strongly doubt that checklist is enough to tip you over the edge - it could just be that the page is code-heavy relative to any unique content (we including the template text in our duplicate calculations, so sometimes it can be over-sensitive).

If there's no unique information at all, then I'm with @MyHolidayMarketing - the time to add unique content to 30-40 products is time very well spent, for both SEO and conversion. If the problem is that you're spinning 30-40 products out into 1000s of URL variations, then you may want to consider not indexing all of those or canonicalizing them. Having every color and size of every product in your Google index can start to devalue your core pages after a while. That really depends on the scope.

It's the same link as Public Q&A, but you can request a private question under the "Privacy" section at the bottom. PRO members get a monthly allotment:

http://www.seomoz.org/q/ask

Unfortunately, we can't "move" a question from Public to Private.

It can be really tough to gauge the impact - it depends on how suddenly the 404s popped up, how many you're seeing (webmaster tools, for Google and Bing, is probably the best place to check) and how that number compares to your overall index. In most cases, it's a temporary problem and the engines will sort it out and de-index the 404'ed pages.

I'd just make sure that all of these 404s are intentional and none are valuable pages or occurring because of issues with the new CMS itself. It's easy to overlook something when you're talking about 100K pages, and it could be more than just a big chunk of 404s.

I'm seeing a chunk of product pages with an identical, generic title tag and no canonical in place (as far as I can tell). I don't want to publish the exact URL here, since it's a public question and you. It could be that we're not recognizing all of the canonical tags properly, but I'm seeing pages where they aren't in place and there are near duplicates (same title, not copy, small product thumbnail).

I've honestly had mixed luck with using Robots.txt to block pages that have already been indexed. It tends to be unreliable at a large scale (good for prevention, poor for cures). I endorsed @Optimize, though, because if Robots.txt is your only option, it can help "stop the bleeding". Sometimes, you use the best you have.

It's a bit trickier with 404s ("Not Found"). Technically, there's nothing wrong with having 404s (and it's a very valid signal for SEO), but if you create 100,000 all at once, that can sometimes give raise red flags with Google. Some kind of mass-removal may prevent problems from Google crawling thousands of not founds all at once.

If these pages are isolated in a folder, then you can use Google Webmaster Tools to remove the entire folder (after you block it). This is MUCH faster than Robots.txt alone, but you need to make sure everything in the folder can be dumped out of the index.

Thumbed up Matt and Moosa - I think I'd make sure first that the PA/DA is telling the whole story. PA, for example, doesn't take into account possible Google devaluations or penalties (which we can't really measure). That PA of 80 may be over-estimated for a lot of reasons. It may be that you have a solid link profile, but your on-page targeting is lacking. There are a lot of ways to target without switching to a partial-match domain.

Now, would a partial-match domain help a bit? Probably, for now, but Google seems to be dialing that down slowly (exact-match domains really are too powerful, and are still being abused like crazy). Keep in mind, though, that the 301-redirect does carry risk, and may not transfer all of the link-juice. Also, if you 301-redirect just this one powerful page, it could impact your main site's Domain Authority and potentially harm your other rankings on that site (if that page has a huge concentration of the site's links).

Did something change recently? I'm currently seeing your non-www pages 301-redirect to the "www." version, at least in a handful of cases. There are a couple of oddities where a page double-301s, but that looks isolated. I'm not seeing any clear signs of a problem.

You don't need both canonicals and 301s for this particular issue, although a canonical tag can still have value on other parts of the site (including the home-page).

I think you have to separate out a couple of issues here...

First off, 250 links distributed across a site or 50 across 10 blog posts may have very little additional value than a few links on a site or 10 links on 10 blog posts. Linking root domains generally have a lot more impact than total links. If you've got 50 links on 10 posts to the same site, it's very likely some of those links are low quality (even not having any idea what the anchor text is). I'm assuming you mean inbound links all to the same site (and even the same page), though. It's really tough to talk in generalities.

It depends a lot on the targeted text. If your exact-match anchor text is your brand name, or if it's product-oriented but matches your domain name, you'll get more leeway. If Apple has 10,000,000 links to "Apple, Inc.", Google won't care. If Bob's plumbing has 1,000 links and 90% of them are to "buy cheap viagra", that's going to look bad really fast. So, "excessive" can be very situational.

If you're targeting one phrase over and over, most of your links are built/paid (not "natural") and that phrase is keyword-loaded without reflecting your brand, you're probably pushing too hard. Diversification is very important. I think targeting is important, to a point, but it's really easy to over-think it.

It's tough - I do think it boils down to the numbers. If you were talking about 1,700 pages and the rest of your site had 50K indexed pages, I'd probably say not to worry about it (unless, as @Dejan said, you experienced a traffic drop or other problems). When the rest of your site is 500 pages, though, I'd start to worry, especially with Panda updates hitting sites with too much copied content (even if legitimately syndicated).

The fact that your linking back does help (you're not trying to claim these are your articles), but if these pages, which are more than 75% of your index, only represent 4% of visits, I'd really start to question the usefulness and whether it's worth the potential SEO risk.

There might be a partial solution - you could NOINDEX a large chunk of the pages, but leave 50-100 of the articles, if those account for 90%+ of the traffic you're getting. Of course, that's going to take some analysis and is a bit trickier to implement, but it could let you keep most of that 3.9%.

I'd also see where that traffic is coming from - if it's 3.9% of total traffic, but only 0.9% of search traffic (mostly direct visits, bookmarks, etc.), then you've got even less to worry about if you de-index the whole subdirectory.

It can be really tough to separate the Google "dance" from "bounce" from daily fluctuations. The "Google dance" was originally a reference to monthly algorithm/data updates - that actual phenomenon has been gone for years now. Data is updated all the time, and the algorithm can change daily at this point. If you suspect an algorithm change has impacted your site (for example - a Panda update), you typically have to look at the dates of the updates and how the factors in that update are represented on your site. We keep an ongoing history here:

http://www.seomoz.org/google-algorithm-change

When I think of "bounce", I tend to think of sites that are moving up and down a lot for a sustained period of time (while other sites in the same rankings don't move). That can, in some cases, be a sign of an impending penalty or Google re-evaluating something about your site (your indexed pages, links, etc.).

Unfortunately, it can be really tough to separate "bounce" from daily fluctuations. I'm working on a project to track daily changes in rankings, and one thing I can tell you is that Google is far from stable. The average change in any given Top 10 day-to-day is surprisingly high. Anecdotally, the changes in deeper pages of the SERPs (2, 3, etc.) are even larger.

So, this could be perfectly normal. I'd keep an eye on a few things:

(1) Are you tracking using a 3rd-party tool, with personalization, localization, etc. off? It's really easy to see different SERPs these days, just by changing computers, being logged in, etc.

(2) Are you bouncing up and down daily, or is it a gradual change?

(3) Are other people in the Top 20 bouncing as well?

(4) Is this bounce having any real impact? This gets into @stubby's reply - if the bounce isn't impacting search traffic, the number of keywords you get traffic from, the pages that get traffic, etc., it may not be worth worrying about. Dig into Google Webmaster Tools and your analytics and find out what impact these ranking changes are really having on visitors to your site.

Ultimately, you have to pick something and just get started. I can't full audit a site in Q&A, but a few starting points:

(1) Your home-page title is over 200 characters long. It's just too much, and it may look keyword stuffed. I have the feeling you're trying to target everything, and that just doesn't work. Pick a focus and target a couple of keyphrases with each major page at most.

(2) Many of your online store pages have the same title tag, and all your product pages start with the same long phrase. The "Online Store" in the title is almost useless, and I'd move the brand to the end. Put the most unique keywords up front - it's better for SEO AND users. You probably want to control or even canonicalize some of these pages, but that's a complicated topic.

(3) You seem to have a bunch of pages doing a 302 to a 404 page, but the 404 returns a 200 code. In other words, your 404 isn't actually 404'ing. So, Google is never clearing out these old pages. If that's a common occurrence on the site, you need to fix it, or you're just filling the index with junk and hurting your ranking ability.

Obviously, that's not everything, but hopefully a few useful starting points.

The new pagination advice is really tough to navigate. I have mixed feelings about rel=prev/next (hard to implement, doesn't work on Bing, etc.) but it seems generally reliable. If you have pagination AND parameters that impact pagination (like sorts), then you need to use prev/next and canonical tags. See the post Alan cited.

I actually do think NOINDEX works fine in many cases, if the paginated search (pages 2+) have little or no search value. It really depends on the situation and the scope, though. This can range from no big deal at all to a huge problem, depending on the site in question, so it's tough to give general advice.

I'm not having great luck with GWT parameter handling lately (as Alan said), especially on big sites. It just doesn't seem to work in certain situations, and I have no idea why Google ignores some settings and honors others. That one's driving me crazy, actually. It's easy to set up and you can try it, but I wouldn't count on it working.

When you say "duplicate blog content", do you mean that the paths are creating duplicate URLs for the post themselves, or for that the tags, categories, etc. have overlapping search results?

If it's the latter (more common), I'd generally NOINDEX those. They aren't really duplicates, and canonical isn't appropriate in most of those cases. Agreed with @Boomajoom, though, it does depend on how integral those paths are for search. Some people use tags as major category navigation and build links to them. For others, the tags are just secondary navigation.

If your paths are all creating different URLs for the individual blog post, definitely use rel-canonical on those. That's a problem, and they are true duplicates. I'd even advise seeing if you can not do that - it's just messy long-term, and the perceived usability benefits are very, very small in my experience. It's almost never worth having multiple URLs for the same final page.

Yeah, I'm with Kevin - there's a way to do a 301-redirect, and you need to find it. Otherwise, you'll lose the value of most of the current links to your site, and you'll create a ton of 404s from visitors and links with old URLs trying to find the old content. That's an SEO nightmare waiting to happen, and it's bad for usability, too.

Wow, that's a tricky one. I haven't seen a lot of good data on rel="alternate" hreflang="x" yet, other than general suggestions that it's worth trying and seems to sometimes work. Technically, Google would say that the green/red/yellow versions aren't true duplicates, but practically, I think the canonical here is a good approach.

My gut feeling is that the canonical will overpower the hreflang - adding the alternate language versions to the color variants won't hurt you, but it probably won't do anything. If it were me, I think I'd go with your first approach - just add the hreflang tag to the canonical version and leave it off the variants. Monitor that - see how it impacts your international rankings, and go from there.

Hard to say without details, but I'd definitely get those 301s in place fast, if they fell through the cracks. Possibly the new site introduced a couple of home-page duplicates as well? (like "/" vs. "/index.html"). Might want to check the internal links and add rel-canonical to the home-page.

I'd just keep the general ROBOTS variant and drop the ID:

The id="" shouldn't hurt, but it may be messing with our crawlers (Google should be ok). The additional GOOGLEBOT directive is repetitive.

When you say that you're not getting benefits, what do you mean, exactly? If you're not suffering from any particular indexation problems or something like Panda, you probably won't see much difference.

It's possible that the canonical timing was just a coincidence and something deeper is going on, but I look at it this way - if it's easy to fix, fix it, and then you'll know for sure. It can be really tough to separate technical indexation problems from penalties.

I think your follow-up comment is a good bet - either use rel-canonical to a "View All" version or use the rel=prev/next tags. A canonical back to page 1 of comments is technically not endorsed by Google (although, honestly, it would still probably work in most cases). If you don't feel the comments add any SEO value, you could also just META NOINDEX pages 2, 3, etc.

The W3C standards might allow for no protocol, but you would never just put "//" - that's part of the protocol ("http://", "https://", "ftp://", etc.). This usage is technically incorrect. It could cause minor issues on some browsers (although probably not on newer ones).

Does it matter for SEO? Well, that's a bit trickier. Google tend to ignore base href unless there are ambiguous relative URLs, like canonical tags that have no base URL and are unclear. Practically speaking, it's probably not a huge problem, but it is possible for it to cause issues down the road.

Either way, if it's on a sitewide template, it's a 5-minute job, and what they have is wrong. I'm not one to knock devs (I've been a dev and I've managed devs), but they need to stop arguing and just fix it.

Marisa's right - we're seeing evidence that trusted outbound links can actually help a site, not harm it. Nofollow really came out of problems where sites had a lot of links they couldn't vouch for. For example, if you have a blog that allows comments and 100s of people comment, you can't tell if all of those links to their sites are legit, relevant, etc. So, you'd add nofollow to say "I can't vouch for this link" (that was the original intent, anyway). When you've chosen the resource and it's relevant to your site, let Google follow it.

I agree with @activitysuper - it would work like a 301-redirect and Domain B would inherit the ranking power. Actually, I've seen that in action - some people even use a cross-domain canonical in place of 301s (it can be faster, but that's a tricky choice).

Even if Domain B had no relevance to Domain A, you might see some short-term rankings. In that case, though, the rankings would probably fall off quickly once Google re-evaluated Domain B.

Generally agree with the comments - links between two of your own sites shouldn't be a huge problem. Google could devalue them a bit, if they see the connection, but devalued isn't penalized. As Philipp said, if you also put contextual links to the webshop in the blog, it could be that the banners keep those links from counting - Google tends to disregard the 2nd, 3rd, etc. link to Page B from any given Page A. If the contextual links are specific, deep pages, though, you should still get value.

Sorry, I could've sworn they recommended not using relative paths somewhere, but now I can't find that reference. I'd just make doubly sure they're resolving correctly. Given that these pages disappeared completely from the index, it's hard to believe the canonical tag addition was just an accident. You always have to start with what you know, and you know this changed.

I have to admit I have mixed feelings about Google's recent advice, because it's very complex (and they've oversimplified it), and it doesn't work well for all scenarios. If you're using this as prevention and don't have any major problems (like a Panda penalty), then I think rel=prev/next is a good bet here.

As Alan said, you should be able to remove the Meta Robots (NOINDEX), and that's probably sending a mixed signal to the crawlers.

For the sorts and other additional parameters, Google recommends you use rel-canonical to the root page. So, a URL like:

http://www.vistastores.com/patio-umbrellas?dir=asc&order=name&p=2

...would have the following tags on it...

In other words, canonicalize to page 2 (with no additional parameters) but then rel=prev/next should reflect the sorts and other parameters of the current URL.

This is the main problem I have with the approach - it's extremely complicated.

Meta Robots (NOINDEX) is very effective for keeping the search pages out of the index and avoiding duplication problems, and it's much easier to implement. The advantage of rel=prev/next is that your other pages (2, 3, etc.) could potentially rank if they're a better fit. For internal search, like product search, I find that's almost never a big issue. It's much more important for article pagination (Google doesn't make this distinction very well in any of their recent statements).

Also, as Alan said, it's approved to just canonical to a "View All" version, if you have one and it's linked/available for users. That can create a huge page, though, so you have to take usability and load times into account.

Sorry, it's very complex - I need to do a write-up on this, as I'm frustrated with Google on the subject. Honestly, I still tell some folks to use NOINDEX, because it's just simpler and it's very effective and preventing duplication problems. Rel=prev/next is more subtle, but it does seem to work, if you can implement it properly.

In terms of the current algorithm change, we really don't know yet. I suspect that Donnie's right - it's really the same sort of stuff that we've referred to as over-optimization all along:

• Keyword stuff copy, titles, URLs
• Aggressive exact-match anchor text
• SEO-targeted copy (like 1000s of thin pages that only differ by a keyword)
• Suspicious code-order changes
• Hidden or partially hidden content

I suspect that, unlike the past, Google may start treating over-optimization more like Panda. In other words, dozens of over-optimization variables will get fed into a model and spit out one unified factor. I also suspect the threshold will be pretty high - you're going to have to be knowingly manipulating your site in an aggressive way. I don't think anyone is going to get penalized for an extra keyword repetition or a TITLE that's a bit too long.

I think that canonicalizing the colors/variations back up to the "root" product is a good bet - while those color variations are technically unique, they can look like thin content to Google, especially at a large scale. A couple of suggestions, though:

(1) I wouldn't use the canonical tag AND Meta Robots (noindex) - it could confuse the crawlers. In this case, since there are separate URLs for the colors/variations and people might link to those, I'd just keep the canonical and drop the Meta Robots.

(2) I think our crawler might be tripping up on the id="" reference in the Meta Robots tag, but I'm not 100% sure. That shouldn't be an issue for Google, although I try to keep those tags free of ids and other extra attributes.

(3) In general, you don't need a Meta Robots tag for all bots and Googlebot separately (especially if the behaviors are the same). I don't think it's a problem here, but it's not necessary.

I haven't specifically tested the impact of relative URLs, but to the best of my knowledge, all canonical tags should be absolute URLs (including "http://"). I would've figured Google would just ignore the incomplete tags, at worst, but it's certainly possible they're attributing them incorrectly.

Since you know you made the change and that they pages have de-indexed, I'd definitely fix the issue, even if it's on a few test pages (not sure how difficult the implementation is).

One note - this is probably just a typo in your question, but it's href="", not src="" in the canonical tag. Google's reference page on the tag is actually pretty good:

http://googlewebmastercentral.blogspot.com/2009/02/specify-your-canonical.html

Yeah, if the .com is blocked now, there's really no point in putting 301s or canonicals over there, because they won't do anything (theoretically, at least). You could put self-referencing canonicals on the .co.uk site. It would at least be a nudge to Google to ignore the old canonicals (to the .com). Other than that, you may have to wait and see.

As Alan said, you could 301-redirect the .com and then stop blocking it. Properly redirected, no visitors should be able to view the old pages. In some ways, that's even more reliable than blocking.

Update: Sorry, realized that was a bit confusing, as I sort of told you that a 301 was pointless but then to 301 What I'm saying is that you could stop blocking the .com and THEN 301-redirect it. If it really is fully blocked, 301-ing it probably won't have any impact (although it won't hurt anything).

Oh, sorry - didn't realize those two pages were different. Can Google crawl the members page? I'm not completely clear where the ref= version is coming from.

Actually, I missed something. You have both the root ("/") version and the "/home" version being indexed, but the root version is cached as your other site. This could indicate that Google is connecting them. This is the cache of CollegeXpress, and it's the Careers and Colleges page:

http://webcache.googleusercontent.com/search?q=cache:w8YxyZPf5cgJ:www.collegexpress.com/+&cd=1&hl=en&ct=clnk&gl=us

I thought I had ruled this issue out, but apparently not. I'm not clear if Google thinks they're duplicated (which I'm not really seeing to be the case) or if there are some weird redirects or cross-linking causing the problem. I've seen this happen once or twice, but usually when sites were heavily duplicated.

I have to echo what others are saying, just to be honest with you - even most decent consultants I know have minimums of about $1,500/month for retainer work. Most agencies won't deal with anything below $2,500/month (here in the US).

You could work with a freelancer (domestic or overseas) and get a decent chunk of hours for $500/month, but you'd be talking about a lower level of experience. If you can segment what you want and really lay out projects, I think that's fine. If you need someone to tell you what you should be doing and guide the process, it's going to be a lot tougher.

We do have a recently updated recommended list, but many of the firms are higher-priced:

http://www.seomoz.org/article/recommended

I think that Portent Interactive out of Seattle does reasonably priced audits (starting in the $1,500 range) - that might be a good starting point, as @Nakul said. Spend your first 3 months of budget on a solid site audit, figure out where your gaps are, and then go from there. You can do some of the initial work yourself and then hire a freelancer/contractor to do some small-scale projects based on what the audit found.

Google claimed Panda 3.4 was rolling out on 3/23, but they didn't really provide much in the way of details, and "chatter" has been pretty light. I didn't see any major movements in monitored keywords or major changes in any clients, honestly.

They started their link-network crackdown in late March, too, although the timeline of that is unclear (it seems to be ongoing, as far as I can tell). If you're not engaging in that kind of back-link building, then it's unlikely you'd be affected. That ones a lot more black-and-white than Panda, in most cases.

Beyond aggressive advertising, major Panda factors still seem to be:

• Large-scale "thin" content
• Massive duplicate content (internal or cross-site)
• Bad user signals (dwell time and CTR, probably)

Most Panda recovery cases I've seen have involved major culling of indexed content - getting rid of anything thin or duplicated and pulling back to the most valuable content. Finding that and dealing with it effectively is a tough job, though, and usually requires adjustments along the way.

I'm not seeing any suggestions that the Panda updates have fundamentally changed the basic Panda "philosophy". I think Google is just tweaking the parameters as they go.

While we don't know the exact math, I think the existence of two PR measures (internal and Toolbar) is pretty much established fact at this point. TBPR is a simplification of a much more continuous PR scale and is almost definitely non-linear. The difference between TBPR 6 vs. 7 is MUCH greater than the difference between TBPR 1 vs. 2, for example.

People polarize quickly on this one, but I think Charles' middle ground is probably closest to the truth - PR is still a part of the algorithm and is a raw measure of link popularity. The trick is that:

• It's just one ranking factor of 100s
• Toolbar PR approximates Google's internal PR
• Toolbar PR could be 1-3+ months out of date

Keep in mind that TBPR is probably on a logarithmic scale, so the distance between low TBPR's is much less than the distance between high TBPRs. In other words, the difference in "real" PR between TBPR 1 and 2 is MUCH smaller than the difference between 6 and 7.

I tend to agree with EGOL - it's a rule of thumb. See my post from last year:

http://www.seomoz.org/blog/how-many-links-is-too-many

If this was a sitewide issue, like your navigation, and your site had relatively low authority, then it would be worth worrying about. If you're just talking about a couple of pages, and one of them is your HTML sitemap, then I'd probably just ignore it. Some pages naturally require a lot of links. You could split that sitemap up if you think it would have value for users, but from a purely SEO perspective, I don't think it's an issue.