Welcome to the Q&A Forum

I am experiencing the same problem. Pages that are easily checked with this tool in the past are now giving us the same error as the OP.

Is the tool down for maintenance or am I missing something?

We recently had Google crawl a version of the site we that we had thought we had disallowed already. We have corrected the issue of them crawling the site, but pages from that version are still appearing in the search results (the version we want them to not index and serve up is our .us domain which should have been blocked to them).

My question is this: How long should I expect that domain (the .us we don't want to appear) to stay in their index after disallowing their bot? Is this a matter of days, weeks, or months?

We have a .us and a .com version of our site that we direct customers to based on location to servers. This is not changing for the foreseeable future.

We had restricted Google from crawling the .us version of the site and all was fine until I started to see the https version of the .us appearing in the SERPs for certain keywords we keep an eye on.

The .com still exists and is sometimes directly above or under the .us. It is occasionally a different page on the site with similar content to the query, or sometimes it just returns the exact same page for both the .com and the .us results. This has me worried about duplicate content issues.

The question(s): Should I just get the https version of the .us to not be crawled/indexed and leave it at that or should I work to get a rel=canonical set up for the entire .us to .com (making the .com the canonical version)? Are there any major pitfalls I should be aware of in regards to the rel=canonical across the entire domain (both the .us and .com are identical and these newly crawled/indexed .us pages rank pretty nicely sometimes)? Am I better off just correcting it so the .us is no longer crawled and indexed and leaving it at that?

Side question: Have any ecommerce guys noticed that Googlebot has started to crawl/index and serve up https version of your URLs in the SERPs even if the only way to get into those versions of the pages are to either append the https:// yourself to the URL or to go through a sign in or check out page? Is Google, in the wake of their https everywhere and potentially making it a ranking signal, forcing the check for the https of any given URL and choosing to index that?

I just can't figure out how it is even finding those URLs to index if it isn't seeing http://www.example.com and then adding the https:// itself and checking...

Help/insight on either point would be appreciated.

I didn't know that marking a Good Answer would mark the thread as Answered.

Still looking for responses; hope the 'Answered' tag doesn't cause too many people to pass the topic over.

Yes, the code isn't posted here 'as-is'.

I appreciate the quick response and the explination. I will wait for some other voices to chime in with their thoughts but I suspect they will mirror yours (although I am very interested in conflicting opinions if they are out there).

Tabbing seems to have scrunched the code, maybe this is easier to read

<div id='<a rel="nofollow" target="_blank">navBoxContainer</a>' class="<a rel="nofollow" target="_blank">textClass</a>">

<div id="<a rel="nofollow" target="_blank">boxTitle</a>" onclick="<a rel="nofollow" target="_blank">location.href='bla</a>h.example.com">

<div class="<a rel="nofollow" target="_blank">boxTitleContent</a>" title="<a rel="nofollow" target="_blank">Text Here</a>"><a href<a rel="nofollow" target="_blank">Text Here</a>"><a ="blah.exam.cpleom">Text Herea>div>

Hope that helps.

I have been seeing conflicting opinions about how Google would treat links using 'onclick'.

For the example provided below: Would Google follow this link and pass the appropriate linking metrics(it is internal and points to a deeper level in our visnav)?

=-=-=-=-=-=-=

<div id='<a class="attribute-value">navBoxContainer</a>' class="<a class="attribute-value">textClass</a>">

<div id="<a class="attribute-value">boxTitle</a>" onclick="<a class="attribute-value">location.href='bla</a>h.example.com">

<div class="<a class="attribute-value">boxTitleContent</a>" title="<a class="attribute-value">Text Here</a>"><a href<a class="attribute-value">Text Here</a>"><a ="blah.exam.cpleom">Text Herea>div>
```

 =-=-=-=-=-=-=

An simple yes/no would be alright, but any detail/explination you could provide would be helpful and very much appreciated.

Thank you all for your time and responses.

The best answer I have seen is from a Whiteboard Friday that Rand put out a few weeks ago.

http://www.seomoz.org/blog/prediction-anchor-text-is-dying-and-will-be-replaced-by-cocitation-whiteboard-friday

Take a listen, it is interesting and if I read your question correctly, it is the answer you vare looking for -or at least a good starting point.

I think it would depend on how well the site(s) is ranking already. If it is ranking well and you are only filling in the extra characters (that is how I am reading it), then I can't see it hurting the sites SERP rankings.

However, if you are removing terms from the title to add the number, then I think you may see a dip in the results.

My bottom line: It seems to me that you are simply adding the number and not replacing anything with it. If that is the case, I really can't see it hurting the results. You shouldn't act on your clients site with just my opinion though; hopefully some others will populate this thread with some additional insight.

I am going to have to disagree, sort of, with Horizon on the usefulness of the number within the title tag -with some reservations of course:

If the client is a local service business (I am only assuming it is based on the description of the client as a 'carpet cleaner') then I can see some usefulness to filling the space you haven't already used with a phone number. I see what Horizon is saying about using that valuable real estate to help draw visitors into the site, but at the same time I would assume that a converting lead is most important to the client so I am inclined to say that your best option is to test them both.

I know, sort of a lackluster answer.

The only reason I am not fully supporting the answer supplied by Horizon (it is a good answer regardless) is that I am not positive about how someone searching for a service, like the one supplied by your client, would like to come across and contact a vendor. Traditionally it was the phone book, word of mouth and hoping for a good result, but with all the local business sites populating the web now, I find myself taking the quality of the site as an indication of the quality of the business. However, if I was needing a carpet cleaned in a hurry I might be inclined to call the number listed in the SERPs if the rest of the blurb made it clear that they were in my area and did, in fact, clean carpets.

If it was up to me I would make a note of how the site is currently performing in terms of bringing in organic traffic/converting leads, and then I would test the other two options over a period of time. When benchmarking the current performance I would establish a set of key performance metrics to track throughout (the most important being how many converting leads are generated). Maybe test with the phone number and then test with some additional information and no number. I would wait, and use the data available to me, to find a historically stable period of time in which the testing can be as controlled as possible. Remember: if you are testing the phone number at the end you should be sure that the rest of the title supplies enough information for the potential customer to feel comfortable contacting your client.

So after all of that typing I am still where I started:

Is it better to have a small sell line or a phone number?

Is supplying a number in the SERPs when the competition does not a good way to gain a little bit of ground over them?

Are those ten or so characters enough to include anything substantial besides the phone number?

Is the possible decrease in organic traffic worth the possible increase in the 'oh my god, the cat just knocked over an entire bottle of wine and the inlaws are visiting this weekend!' busniess that may or may not be created by offering a phone number in the SERPs?

Is the rest of the title descriptive enough to convince someone searching for your clients service is what they need and that they should skip clicking through to the site in favor of calling the number?

I would be interested in reading what some of the other users have to say. I am sure there are differing opinions and a few hundred minds are better than one.

Without knowing the existing site architecture it is a little difficult to give a specific answer, but my two cents:

Are the 'like products' on the same page? For instance are...

Motorcycle pants R2000, Motorcycle jacket R2000, and Motorcycle kit R2000 on Page A

...and...

Motorcycle pants R4000, Motorcycle jacket R4000, and Motorcycle kit R4000 on Page B

...that is the image I am getting from your description.

Would it work with your site architecture to have a guide page for each category and then link to the product pages from there? The pants guide could talk about how amazing your motorcycle pants are, the relevant specs and about how wonderful your butt would look in a pair. The link could land on a product page that is a collection of all the pants you offer, it could be a link to the R2000 'set' page (where you sell all the products under one page), it could theoretically land on whatever you think is most user-friendly and would increase your ROI.

Ideally, and in my humble opinion, you would optimize your first page -however you choose to lay out the internal linking- for SEO and to show in relevant SERPs. Give some great original content; make that page have personality/establish your brand and brand persona (fun, serious, edgy, whatever); and something people would feel good about sharing with their buddies on facebook. Your awesome page on pants, for example, could be the canonical page and some appropriate usage of the 'rel=canonical' element could ensure that, if your user lands on the buy page (the one where all the size selections, etc... take place), that the linking metrics find their way to the page you want to rank, and have optimized for ranking, while the user happily shops and buys. This should avoid eating your own tail when it comes to talking about pants on subsequent pages -let's be honest, you can't sell pants without talking about pants.

I hope that this was clear and offered some sort of insight, but please take it only as a consideration which should be examined critically and with other options in mind. I am sure there are some other great ideas to be put forth and I would love to see some others post their thoughts!

My two cents: Like most things SEO the answer is to stop and think; collect all the statements/opinions that you can and then apply what you have learned to your current situation. In this case you are going to be taking both the EMD devaluation and the intent of the Panda (thin content)/Penguin (spammy links) updates and apply them to your situation. Without knowing your site or the other sites being linked from. I will be responding in broad hypotheticals and hopefully, if I understood your question correctly, it will provide you with something to consider.

First things first: Ryan is basically dead-on with his assessment of the 'I heard...' issue. So, for the sake of clarity, I will once more state that what follows is my opinion on the vague question presented.

Your question is a good example for the 'use your head' school of thought. The question appears to be about the spammy-ness of buying and redirecting EMD sites to a main website in an attempt to benefit from their existing site traffic/linking metrics. The confusion seems to be a common one: "Does changing X on site Y accomplish/trigger Z? If yes, then does this apply to everything?" Coupling individual situations and fixes to something as scattershot as 'in all cases' is writing yourself a check for future struggle.

I think that your main source of confusion is all the misconceptions that float around about the EMD update, so for the sake of clarity:

If you own an EMD site that sells baby clothes (ex: babyclothes.com) then the EMD update didn't penalize your site, it simply took away the algo advantage that having an EMD used to grant. Think of that EMD advantage/'penalty' as a short person standing in a crowd and on a stool; the short person appears taller than they actually are and the stool may be large enough to help that person stand out from the crowd more easily than they otherwise would. If the stool were to be removed then that short person would lose the boost in visibility the stool provided. This wouldn't be making the person shorter, but simply removing the vertical boost the stool provided.

With this understanding of the EMD devaluation in mind, your situation/question clears up and becomes less muddy than it may have initially seemed:

If you own a site that sells baby clothes (branded, EMD, etc...) and you acquire related EMDs (babyclothes.com; babyclothes.net; infantclothes.com; etc...) so they can be redirected then you should be just fine. The key word in that last sentence was 'related' and it applies to any type of domain, not just EMDs (they have lost their boosted value so should be treated like any other site). This is all assuming you are buying established/existing sites that have content because link schemes (like Ryan correctly pointed out) are penalized regardless of the site being an EMD or not. A doorway is a doorway regardless of its domain name.

Oh, and one final thing: Make sure that if you buy an existing site you give the user base of that site a few days or weeks notice of the coming change. Users are the most important part of any decision made and nothing will earn you the ire of your users more than sudden changes with no warnings (once again, I am assuming you are buying existing sites and not just setting up a bunch of doorways pointing to your site).

In short:

The EMD update has made the benefit of having such a domain moot, however, existing EMDs can still ride the wave of link and user base metrics that having an EMD used to be helpful in acquiring.

Redirecting any existing sites (EMD is an irrelevant variable for this case in my opinion) which you have purchased into your primary domain is not a problem assuming you aren't simply setting up a bunch of doorways and that the purchased/redirected sites are relevant.

I would love to hear some of the other opinions on this.

Vikas_Rana, did this help at all? I hope so.

Brian and IPRO both suggest that you use the disavow links tool that Google recently rolled out. That may wind up being the answer in the long run but Matt Cutts, in a recent GoogleWebmasterHelp video, seems to stress the fact that this tool should be used after exhausting other link-removal attempts.

Barry, over at seroundtable, has a straight forward write up about this on his blog and even includes a sample (quoted below) of what it is he thinks Google is looking for in a disavow action.

Here is a link to the specific article I am speaking of: http://www.seroundtable.com/google-disavow-link-tool-15848.html

I know you want a quick and relatively painless fix, but Google tends to be vague in that regard and I doubt, unfortunately, that such a fix exists in a situation like this.

*Contact those who run the sites where the links are coming from, and keep a record of your interaction with them.

*E-mail all relevant parties until you get some sort of answer, positive or negative. If you get no response make a note of that.

*Contact Google with a spam report (probably won't get a non-automated response quickly or at all), and make a note of your report submissions/

*If none of these, or other methods I am sure I must be leaving out, solve the issue, format and submit a detailed disavow file.

WIsh I could offer the silver bullet but, as far as I am aware, that bullet has yet to exist.

I am going to have to disagree, sort of, with Horizon on the usefulness of the number within the title tag -with some reservations of course:

If the client is a local service business (I am only assuming it is based on the description of the client as a 'carpet cleaner') then I can see some usefulness to filling the space you haven't already used with a phone number. I see what Horizon is saying about using that valuable real estate to help draw visitors into the site, but at the same time I would assume that a converting lead is most important to the client so I am inclined to say that your best option is to test them both.

I know, sort of a lackluster answer.

The only reason I am not fully supporting the answer supplied by Horizon (it is a good answer regardless) is that I am not positive about how someone searching for a service, like the one supplied by your client, would like to come across and contact a vendor. Traditionally it was the phone book, word of mouth and hoping for a good result, but with all the local business sites populating the web now, I find myself taking the quality of the site as an indication of the quality of the business. However, if I was needing a carpet cleaned in a hurry I might be inclined to call the number listed in the SERPs if the rest of the blurb made it clear that they were in my area and did, in fact, clean carpets.

If it was up to me I would make a note of how the site is currently performing in terms of bringing in organic traffic/converting leads, and then I would test the other two options over a period of time. When benchmarking the current performance I would establish a set of key performance metrics to track throughout (the most important being how many converting leads are generated). Maybe test with the phone number and then test with some additional information and no number. I would wait, and use the data available to me, to find a historically stable period of time in which the testing can be as controlled as possible. Remember: if you are testing the phone number at the end you should be sure that the rest of the title supplies enough information for the potential customer to feel comfortable contacting your client.

So after all of that typing I am still where I started:

Is it better to have a small sell line or a phone number?

Is supplying a number in the SERPs when the competition does not a good way to gain a little bit of ground over them?

Are those ten or so characters enough to include anything substantial besides the phone number?

Is the possible decrease in organic traffic worth the possible increase in the 'oh my god, the cat just knocked over an entire bottle of wine and the inlaws are visiting this weekend!' busniess that may or may not be created by offering a phone number in the SERPs?

Is the rest of the title descriptive enough to convince someone searching for your clients service is what they need and that they should skip clicking through to the site in favor of calling the number?

I would be interested in reading what some of the other users have to say. I am sure there are differing opinions and a few hundred minds are better than one.

I agree with Mark Scully on this one, but would like to add some thoughts:

If you are looking to clean out your backlink profile you should go about it in a very methodical fashion. I would recommend exporting the links to an Excel file and then, in a new sheet, start skimming and categorizing them -needs more research; relevant; potentially harmful; show stopper. It will be time consuming but once you have a basic categorization set you can start reaching out.

There is a real possibility that many of the directory links are from neglected and orphaned directories and that the contact e-mail may not be in operation anymore. When you find this to be the case, note it on your categorized Excel sheet. Note the date you sent the link removal request and note the response; if there is no response, note that as well. Be realistic concerning the expected reply time (this is a big deal to you; it is probably not a big deal to those hosting the directories) and send out second and third requests.

If it was me, I would concentrate on the two most harmful categories and give them a real thorough going through. After a few weeks (I know, it's a long-ish project) you should have a nice detailed actions-taken report and should feel comfortable utilizing the disavow links tool if needed.

Note: This tool, from what I understand, is not a click-and-fix and you will need to have a file of the links you would like disavowed to upload to Google for review. Barry Schwartz, over at seroundtable.com, has a nice post concerning this and he supplies an example of what a disavow report might look like:

Watch the video by Matt Cutts explaining the tool and use it with caution and only as a last resort; don't spam them with reports.

One final note: Some of these links may not be harming you as of now. Use your best judgement and ask yourself this question: "if I knew another penguin update was coming tomorrow, would having this link cause me to worry?" It isn't always a straightforward answer, but if you find yourself stretching and searching for a rational to view the link as relevant or user-centric, then it probably isn't.

I am sure there is plenty more to say on the topic, and I hope some others chime in with their thoughts. It's time to earn that paycheck.

Keep us posted, and happy digging.