Welcome to the Q&A Forum

Hey Moosa,

Since the value of footer links is very low anyway, the likelihood that they may be seen as manipulative would be of far more concern to me - no following them is unlikely to make any sort of negative dent in search visibility. It will, however ensure that the site does not become an easy target for Penguin.

Given the business your client is in, I imagine the most important thing for them is bringing new qualified traffic to their site. The potential for referral traffic from a nofollowed link with anchor text properly crafted to reinforce the brand and entice the click should be the prize they have their eye on.

Hope that helps,

Sha

Hi Zippy-Bungle,

To understand first why the 803 error was reported:

When a page is called, the web server sends header details of what's to be displayed. You can see a complete list of these HTTP header fields here.

One of the headers sent by the web server is Content-length, which indicates how many bytes the rest of the page is going to send. So let's say for example that content length is 100 bytes but the server only sends 74 bytes (it may be valid HTML, but the length does not match the content length indicated)

Since the web server only sent 74 bytes and the crawler expected 100 bytes the crawler sees a TCP close port error because it is trying to read the number of bytes that the webserver said it was going to send. So you get an 803 error.

Now browsers don't care when a mismatch like this happens because Content-length is an outdated component for modern browsers, but Roger Mozbot (the Moz crawler, identified in your logs as RogerBot) is on a mission to show you any errors that might be occurring. So Roger is configured to detect and report such errors.

The degree to which an 803 error will adversely affect crawl efficiency for search engine bots such as Googlebot, Bingbot and others will vary, but the fundamental problem with all 8xx errors is that they result from violations of the underlying HTTP or HTTPS protocol. The crawler expects all responses it receives to conform to the HTTP protocol and will typically throw an exception when encountering a protocol-violating response.

In the same way that 1xx and 2xx  errors generally indicate a badly-misconfigured site, fixing them should be a priority to ensure that the site can be crawled effectively. It is worth noting here that bingbot is well known for being highly sensitive to technical errors.

So what makes the mismatch happen?

The problem could be originating from the website itself (page code), the server, or the web server. There are two broad sources:

1. Crappy code
2. Buggy server

I'm afraid you will need to get a tech who understands this type of problem to work through each of these possibilities to isolate and resolve the root cause.

The Moz Resource Guide on HTTP Errors in Crawl Reports is also worth a read in case Roger encounters any other infrequently seen errors.

Hope that helps,

Sha

Hi fryguy,

A couple of Quick things to check in Google Webmaster Tools:

Index Status: Google Index > Index Status should show you visually whether the site has ever been indexed or not.

Robots.txt tester to check that your robots.txt file is correctly configured.

Fetch as Googlebot Tool to see whether Googlebot is able to successfully crawl your site. You can also use Fetch & Render to see what Google sees on your pages.

Once you get a handle on what might be happening it will be way easier to find a solution

Hope that helps,

Sha

Hi Healeyman,

Tom has pretty much nailed the answer above.

Just one thing to add. If the blogs in question meet all of the criteria Tom has mentioned in his first paragraph and are likely to bring you referral traffic, perfect. If you are just thinking about the possibility that they might influence search engine rankings, your time would probably be better spent looking for opportunities that have a potential to bring people to your site.

Sha

Sorry Marie,

Should have included you in my comments, but could not see that you had commented since I was dealing with a "disappearing answer" catastrophe by composing elsewhere and pasting in.

Why is it only the long ones that do that?

Sha

Hi Elchanan,

Well Eyepaq is batting a thousand today!

Eyepaq is quite correct. The only way to "transfer" the bulk of the link equity is to redirect the domain which would inevitably result in a transfer of the manual action as well. In fact, it is worse than that. In recent times a number of domains have been dealt a manual action by association without a redirect even being in place. These manual actions have been applied because the Webspam team believes that the sites are related and are part of a larger scale manipulative effort, or indeed an effort to get out from under a penalty. Matt Cutts talked about this at SMX West, stating that people should not be able to "move down the road" to avoid a manual action.

If you genuinely needed to build a new site before the manual action and have not put your life savings and years of effort into building a brand, there could be a business case for starting again with a brand new domain, BUT remember you will be starting with nothing - even less than you will have if you successfully clean up the existing domain. This has to be a careful business decision and any new site would need to be completely unique and without any connection to the penalized site. Personally, starting over would be my last resort unless the site was fundamentally broken (and the domain name was a poor choice) to start with.

There are generally five broad reasons why a reconsideration request may fail:

Insufficient data - maybe links in the backlink profile have not been surfaced in the data gathering stage. Incomplete data is common and is best remedied by using as many data sources as possible and in some cases by pulling multiple samples over a period of days or weeks.
(Remember that the links returned by the Webspam team when a reconsideration request fails are "examples". They are intended to point you toward other links in the backlink profile which follow the same patterns or use the same unnatural linking tactics.)

Mistakes in Analysis - If links have been misclassified as natural and are kept, the reconsideration request will fail. Sometimes this happens because people rely solely on algorithmic analysis tools to determine which links to keep or remove and results are not 100% accurate. I would always argue that a real human should be the primary tool when doing analysis because I believe there is no room for mistakes in a job that your livelihood depends upon!
Sometimes human analysis can go wrong too - most often because people forget that this is about "unnatural" linking. That means links that were created rather than earned.
Another mistake that people make at this point is to try to just remove the worst of the unnatural links to preserve some of the benefits that were gained from unnatural linking. Omitting unnatural links from the cleanup effort because you think they are not so bad is a big mistake for two reasons:

1. It will mean leaving unnatural links in the backlink profile - on their own they could cause the manual action to be upheld, but even in the rare instance that this might be allowed to scrape past on reconsideration, retaining them leaves the site vulnerable to the Penguin algorithm
2. It immediately shows the Webspam team reviewer that the site owner's manipulative mindset has not changed. Making a case for reconsideration requires that they are able to trust the site owner will never employ those kinds of tactics again.

Incomplete or ineffective Disavow submissions - As mentioned above, it is always best to disavow at the domain level to ensure that any links you are unaware of do not remain in play and sabotage your efforts. The only exceptions to this rule are unnatural links on high value domains you might reasonably love to have "natural" links from. In these extremely rare cases you should disavow the specific URLs to ensure that any natural links are preserved and any natural links you might earn from that domain in the future will be accorded their rightful value. Also - a red light went on for me when I saw "Google ignored some domains in the two disavow files we submitted". This causes me to wonder whether you have uploaded two completely separate files to the Disavow Links Tool? If so, then this could be the problem. The Disavow Links Tool submission is an overwrite, not an update. This means that you need to combine any existing disavow list with the new list before uploading. If you don't do this then you are effectively re-avowing all of the domains or links that were in the existing file.
If you need to update an existing disavow file with a new list, you can use this free tool to make it easy. Once you have created a free account you can upload your existing list, then upload any new list in the future to create an updated disavow file. When you upload a new list the tool will combine the data, remove duplicates and add date notations so that you can keep track of when domains were added. The tool also ensures that your new disavow file is within the 2Mb file size limit and generates it in the correct text format, ready for submission.

Insufficient effort in the cleanup - Sometimes this is actually just that there is insufficient evidence provided that the work has been done. Most common mistakes here:

1. Omitting domains completely from the cleanup effort because a WhoIs email address is not available
2. Including domains that do not have a WhoIs email address, but not bothering to look any further to find a method of contact. If there are email addresses or contact forms available on the site, a "good faith effort" will require that you have used them to attempt to contact the domain owner.
3. Being unhelpful when requesting that links are removed. The more that can be done to help the domain owner easily locate and remove the links, the better the success rate for the entire link removal campaign.The Webspam team needs to see that a "significant proportion" of the links have been removed. The better the cleanup rate, the smoother the path to getting a manual action revoked.
4. The Bullying approach. Link removal requests should always be written with three things in mind a) You are asking someone to do you a favor b) threats or demands are unlikely to make someone feel that they want to be helpful c) the Brand is at stake here - whatever impression is created by the request will reflect heavily on the Brand. When people get this incredibly wrong, flow-on results can be catastrophic.
5. Not recording and providing evidence of link removal efforts for domains that have not been successfully cleaned up. Keep good records. Provide evidence where domain owners have refused to help or requested payment.Provide  evidence where on-site forms do not function. Make it easy for the Webspam team to make an assessment by providing good documentation.

Not making a case for reconsideration - Site owners need to demonstrate that they understand where they went wrong and will not repeat the same mistakes. In addition to this they need to convince the Webspam team that they have made a "good faith effort" to remove the links. Also, if there are links that are known to be natural, but may look suspicious, address them. Give a reasonable explanation as to why links have been retained (as long as there IS a reasonable explanation). You can use this checklist to make sure you have covered the most important things in your reconsideration request.

This Slide Deck provides an overview that might be helpful.

Any or all of these things can be playing a part in a failed reconsideration effort. It is not uncommon for it to take multiple attempts to have a penalty revoked, but the more of these potential problems we can eliminate by following best practice from start to finish, the more predictable the results.

Best of luck with resolving the manual action and getting things back on track.

Hope that helps,

Sha

Well Hello Mr Fisher

So good to get back into Q&A and to catch up with you too!

Hope it's all good with you and your team,

Sha

Hey William!

I should say thanks for leaving that open for me

I have so been hanging out for the opportunity to get back in and get a Q&A fix lately, but the support phone always takes precedence. It was so good when I made it back in to find some questions that I could contribute to.

Glad you love rmoov!

That started my day off with a big grin

Sha

Hi Kate,

Good call

The nofollow tag will eliminate any unnatural linking issue and in your case the potential to bring qualified traffic to your site by far outweighs any value the links would ever bring in terms of visibility.

Sha

Hi Jampaper,

Just to preface, I spend my days wading through the unnatural links sewer looking at the mess people have gotten themselves into because they thought they were smarter than Google or had that "how would Google ever know" thought in their heads.

EGOL is spot on with his response.

The criteria for undesirable links is not "how would Google ever know it's unnatural?", but "is it unnatural?"

On the "How", here are some things to consider:

1. Google's reach and ability to mine and interpret data (accurately or not) is so far outside our comprehension that it is probably better we don't even think about it.

2. Reviewers have a habit of unitentionally sharing information or creating patterns in the way they do things that are a clear red flag for orchestrated reviews

3. "These reviews always point to inner pages" ...Ooops! There's a pattern

4. "We're obviously targeting authoritative sites which do do reviews" ...Ooops! another pattern

5. Unnatural links on "Authoritative sites" would be more likely to enrage me if I were a member of the Webspam team than those on less influential sites. Let's face it, nobody ever sent me an email suggesting they could sell me links on a crap site

6. (and this you should take as very tongue in cheek, but perhaps give some thought to implications)
   This site has upwards of 400,000 community members. One of them is a guy who is currently on leave from his job at G, but occasionally comments on Moz blog posts that interest him (that's the tongue in cheek part as while it is possible, I seriously doubt he or any of the other Googlers who might be members spend time combing through this site looking for extra work!)

   However, it doesn't take much imagination to think there may be other people out there who could be made aware and if they were a certain kind of person might be likely to look into a backlink profile and perhaps lodge a report. Once the manual review process comes into play, the cleverness of the algorithm is irrelevant.

When you have a great product your customers will always be your best sales force! Do things that make THEM want to tell people how THEY feel about you. If you do that enough, even those Authoritative sites will be checking you out for themselves and gifting you natural links

Hope that helps,

Sha

Hi HashtagHustler,

Great advice from WilliamKammer

No single tool will give you a complete look at your link profile - especially not Google Webmaster Tools!

If you are serious about auditing and dealing with unnatural and/or undesirable links in your backlink profile you should be gathering link data from as many sources as possible. While the data returned from different tools will overlap, there will always be links surfaced in every tool that you do not see in others.

Every tool that provides link data is crawling with different primary aims - some crawl more broadly, some more deeply, each with different preferences and limitations - the result being that the more tools you use to gather link data the more complete look you get at the profile.

As for Google Webmaster tools data, there are some specific limitations to be aware of:

1. Google provides only a "sample" of the links they are aware of in downloaded link data
2. Each link data download from your Google Webmaster Tools account is a unique sample. This means that you can actually get different data in the list if you download at different times!

In addition to the tool William mentioned above, these are some of the most commonly used tools for gathering backlink data:

Bing Webmaster Tools (they actually give you all of the link data they have for your site, although they generally see less than others.)
Majestic SEOahrefs
SEMrush

To get better data from Google Webmaster Tools you can use Tony McCreath's SWAT Google Webmaster Tools Exporter tool.

If you are doing pre-emptive disavow work and likely to need to update your disavow list from time to time, you can use our free tool disdit. This enables you to upload your first disavow list and then just update new lists to have them appended to the existing list. Download the updated list ready to upload to Google.
Why? Because the Disavow Links Tool upload is an overwrite, not an update. This means that if you just upload a new list to Google without appending to the existing list, you will have reavowed all of the domains/links that were previously disavowed!

Hope that helps,

Sha

Hi,

The quickest, easiest and most accurate way is to crawl your site with the Screaming Frog SEO Spider tool.

This will give you the information you are looking for and a whole lot more besides and depending on the size of your site, you could have it in seconds.

If you have your site loaded as a campaign in Moz then you would be able to see the information current to the time of the last crawl.

Hope that helps,

Sha

Hi,

As everyone else has mentioned, nofollowed links are not passing PageRank and therefore should not be influencing search visibility as assessed by the Google Algorithm. If there is no manual action in place then there should be no reason to disavow them.

However, I look very differently at nofollowed links in the backlink profile of a site that has a manual action in place. In that case I ask two questions:

1. Is the site one that is in any way undesirable?

2. Are the nofollowed links part of an obvious large-scale manipulative effort?

If the answer to either or both of these questions is yes, then I choose to both attempt to get the links removed and also disavow. The reason for this: of course I don't want links from dubious sites which even though not passing PageRank could be doing damage to my brand and reputation  and; when asking for reconsideration (judged by a real human being conducting a manual review) I believe that ignoring links that were obviously placed with the intention of manipulating search rankings simply because it didn't work, hardly shows a commitment to the "good faith effort" expected by the Webspam team. So, I go to the extra effort to try to remove and also disavow.

Marie is quite correct in suggesting that there are cases where links on some pages are followed and on other pages nofollowed - in fact we see instances at rmoov where URLs that contain multiple links to a site are actually a mix of followed and nofollowed links as well. As suggested ...you have no control over those links at all, so once you have decided which ones really need to be dealt with disavowing is the safest option available unless you can actually get them removed.

Again, I think the point has been fairly well made above ...Local directories are not all bad. Check for compliance with the Webmaster Quality Guidelines (which for paid directories includes a clear statement in their terms of service which indicates that you are paying for a site review and not for inclusion). The most relevant Guideline is the one about what Google views as "Link Schemes"

Hope that helps,

Sha

"A sitemap should only contain links to active pages."

Hi shawn81

Alex is absolutely correct there.

In fact, Duane Forrester has said repeatedly that Bing absolutely does not like to find such pages in a sitemap and that you should make sure there are never 3XX, 4XX or 5XX status pages included because it will stop Bingbot from crawling your site.

While Googlebot is not so sensitive, the reality is that all search engines allocate a certain amount of crawl capacity for your site...if your sitemaps include a load of pages that are not likely to be indexed, the result is twofold:

1. you are wasting capacity on useless pages and the crawler may never get to the stuff you really want indexed 
2. if the crawler encounters a lot of non-active pages when it crawls, future crawl capacity (not to mention trust) is likely to be reduced  

Replace the old URLs with the new and give the bots a little thrill of adventure 

Hope that helps,

Sha

Hi shubham1234,

If you are concerned about spelling and grammar in text you could add grammarly.com to your list

Hope that helps,

Sha

Hi Dario,

Just one thing to remember - links/domains submitted via the disavow tool will not disappear from your GWMT list as long as they still exist. Google simply discounts them when ranking your site, so you shouldn't spend time checking to see if they have dropped out of the list.

If they have absolutely been removed, then yes it is quite possible that googlebot has been unable to crawl the site of origin (but if there are hundreds of sites then they would all have to have the same roadblock in place) for none to be crawled.

Dana is quite right in saying that Webmaster tools is notorious for updating slowly (and incorrectly at times) and she is absolutely on the money in advising that you need to try to nail down the cause of the rankings drop so you know what you are dealing with.

Hope that helps,

Sha

Hi luwhosjack,

That's about the best you can do if the issue is as I described above - if every 404 on the site generates a new version of that one page then it is generally happening because of poor backend configuration.

Where it gets really messy is in the case where new URLs are being generated by hacking. I've seen several instances where every attempt at injection by a hacker generates a new copy of the page and if the anchor text happens to be exact match, the result is a disaster for the linked site

Sha

Hi mim12,

Afraid I am a lot more risk averse than most when it comes to unnatural links (and I have seen similar situations that emanated from nasty goings on at a couple of sites), so I have a couple of questions to clarify:

First, my read of your description is that there is a single page on that site which contains a link to your site and since you are saying there are now 50,000 links that would mean that there are 50,000 different URLs all originating from that one site and pointing to your site?

So, are you saying that these 50,000 individual URLs are actually identical copies of that one page which contains the link? If so, can you test whether new copies of the page are being generated from 404's by typing a new non-existent path and hitting enter? For example, if the URL of the page in question is http://www.netnewspublisher .com/strangepage.php, what happens if you change the address to http://www.netnewspublisher .com/anotherstrangepage.php and try to access the page? Do you simply get a not found error, or does it generate a new copy of the page with the new URL?

Alternatively, can you give us the actual URL where the original link was placed so we can take a look at it?

Sha

WooHoo!!!!

8D

Q&A mail received and read in 1 beautiful second!

Cookies are on me at Mozcon!

Thanks Mozzers.

Sha

Hey Dana!

See you in Seattle

Sha

Hi Charles,

I would say

1. Keep an eye on #Mozcon and connect with some people before you get to Seattle
2. Stay over and take a Tour of the Mozplex
3. Spend some real time checking out the Speaker photos on this page so you know who they are and what they look like...there is a real chance you will find yourself standing next to them in the lunch queue, sitting down to breakfast or lining up to sing Karaoke with them!
4. Be at the conference early every day and try to sit with new people as well as catch up with friends you have already made.
5. Read this post from Zeph , hit him up on Twitterand get him to hook you up for a spot at the informal Meetup on the night before the conference
6. Make a list of awesome people that you would like to meet while there and keep it close handy
7. Make a donation to the Myrtle Point Library in honor of Michael Cottam's parents at the conference and help send love to Michael and his family from everyone in the Moz "family"
8. Get some tips on where to find the Best coffee in Seattle from Jonathon Coleman
9. Arm yourself with Rand's Restaurant and Bar Guide to Seattle and the The Not-so-Short Shortlist of Moz's Top Seattle Restaurants, Bars, and Activities for MozCon 2013
10. Bring business cards and make sure you make notes on the cards people give you so you can easily put names to faces and shared experiences when you get home.

Oh...and you might consider joining a PosSEO, becoming a PosSEM or something like that, but whatever you do, say G'day!

Sha

Hi Greg,

That is an impossible question to answer I am afraid

OSE results are an indicator of how your site stands according to the metrics that influence PageRank.

Your actual rankings may be influenced by any number of things which are known only to Google:

• Algorithmic filter effects such as Penguin, Panda, Exact Match Domain etc;

• Manual Actions (penalties) applied for any number of issues, from link spamming to cloaking or even abusing rich snippets markup. Check Slides 6 and 7 of this deck for breaches of the quality guidelines that will lead to a manual action ;

• Disavowed links (any links that you have disavowed and are disregarded by Google remain live links which inform the OSE results because Moz is not privy to the list);

If your rankings have tanked recently, I would be using OSE to see if there are any obvious issues with anchor text ratios, links originating from places that are undesirable, changes in authority of sites linking to yours etc, rather than trying to correlate the numbers with your results.

Sha

Hi Marc,

The fact that you did not receive an unnatural links warning does not mean that you don't have a manual penalty.

Despite comments from Matt Cutts and John Mueller that suggest otherwise, I have personally dealt with almost 100 site owners through rmoov support who never received any communication from Google until we suggested they lodge a reconsideration request because the effect in the SERPS looked like more than algorithmic adjustment to us - sure enough, in each case back came notification of a manual penalty.

I would do as Mark suggested: proceed as though you are dealing with a manual penalty and continue to document all of your link removal activities meticulously, then lodge a reconsideration request. After all, that is the only way you will know for sure whether there is a manual action in effect or not.

Sha

Hi Peter,

I don't really see how this can be related to images as I have always had outlook set up not to download images.

After your response that it was related to images I edited the settings to ensure that there is absolutely no way outlook will ever try to download images, but emails from the blogs and Q&A still freeze Outlook for between 1 and 2 minutes before they resolve to show the actual text of the message...it is as if the actual text of the post itself is being called and queued.

As mentioned by Christopher above the problem does not occur with ranking reports, new crawl notifications, the Moz Top 10, Monthly Newsletter or general email from Mozzers.

At this point I can do nothing but avoid allowing those emails to open and can't even delete them without the same issue unless I am lucky enough to have them sandwiched between two other emails that I want to delete

I know it probably seems like a trivial issue if it isn't happening to everyone, but honestly, for those of us who are fairly active on the blogs and in Q&A, 1 - 2 minutes of having your mail program disabled for every email that comes from there gets out of control very quickly!

I'm sure I'm not alone when I say, Please HELP!

Sha

P.S. A heartfelt apology to all of you who just had your mailbox freeze up again because I added this post   hugs

Hi Eric,

A quick look in Majestic SEO shows nearly 34,895 external backlinks from 260 linking root domains with large numbers of links being added during March and April.

As mentioned by Bradley and Chris, there is a lot of suspect anchor text (including cialis, adipex, tramadol etc) in there and many of the referring sites are not places would really want to be getting your links from regardless of the anchors.

It seems your site has attracted some unwanted attention which has dramatically lowered Google's trust. While the obvious conclusion is that Penguin might be responsible, it is clear from the screenshot you provided that the dropoff actually happened around 5/6 May (Dr Pete has talked about "something" happening with the algorithm around 5 May and that would seem to correlate with your problems). Of course, it is possible that this was a test rollout of Penguin and your site was captured as a result, or it may be something completely different (which Google has given no information about).

Either way, it is easy to see that the linking behavior is harming your site's trustworthiness and unfortunately, this is likely to also affect your visibility in Bing and other Search engines as well.

Dealing with the backlink problem is the most critical place to start ... site updates will come later I'm afraid.

Hope that helps,

Sha

Hi Christopher,

Yes, I'm having the same issue with emailed comments from the blog.

Three guesses as to what might be causing it:

1. Images
2. javascript objects
3. the email tracking software being used

The best thing to do is go to the Help hub and lodge a ticket with as much explanatory detail as you can give. I know Mozzers are already working on it, but the more specific information they have, the easier it will be to fix it.

In the meantime, I hope it helps to know you're definitely not alone

Sha

Those rich snippets abuse penalties are already here.

Sha

Hi Sean,

It really depends what type of service you are looking for.

The two you mentioned are quite different services.

Remove'em uses algorithmic analysis to highlight links it thinks should be removed if you use the Self Service option. They also provide a full service option which would mean that the analysts at Virante (the company that builds the tool) would manage every element of the campaign for you. As far as I am aware there is no trial available, but there is a video tutorial which shows you how the Self Service version works - Starts with Self Service at $249 per domain.

To be honest, Link Delete doesn't seem to actually give any details as to how or what they do apart from generally helping to clean up your backlinks. Starts at $97 per month for 1 domain with a max of 400 linking URLs (3 different Plans)

Another service which provides algorithmic analysis as part of their service is Link Detox, which is part of the Link Research Tools suite. You can get started with an initial report by paying with a tweet, get Daypass access (72 hours) for 30 euros, or 2 Link Detox credits that last for a month at a cost of 50 euros.

rmoov is a link removal outreach tool which does NOT provide algorithmic analysis (because we believe good analysis = eyes on sites), but helps to automate much of the outreach process that Marie described once you have a list of the URLs you want links removed from. It has been specifically built to help those dealing with a manual spam action to manage link removal outreach at scale and to provide a level of reporting detail that will meet the Webspam team's expectations when considering a reconsideration request. rmoov allows the user to choose the level of automation or manual work they wish to do, so analysts like myself and Marie can run it as an aid to a predominantly manual process, while those who prefer can make use of automated link checking etc.
rmoov has a free Basic account which allows 1 campaign at a time with a max of 25 linking root domains and/or 250 linking URLs. This free account does not provide automated link checking or pulling of domain contact info, but does not expire. Paid subscriptions start at $49 per month for 5 active campaigns at a time with a max of 100 linking root domains and 1000 linking URLs. (4 different subscription plans). In the interest of full disclosure: I work for the company that develops rmoov

There is another service called Delete Backlinks which provides a pay per link service. This appears to be offered by a company that owns, controls or has agreements with a large number of directory sites. With this service you enter your domain and they provide a list of the links they have access to which will be removed. Pricing starts from 99 cents per link for the first 5 links with the cost per link reducing on a sliding scale according to the number of links you want removed.

Hope that helps,

Sha

Hi Bian,

The domain is indexed (it appears in the Google search results), but since the domain is expired and parked, there is nothing else to see...just a parked page.

Sha

Hi Niners52,

Rather than rely on an external tool to have found all of the URLs in a crawl and report them accurately, I would run a direct crawl using Screaming Frog.

There is a post on their blog which provides step by step instructions on how to use the tool to crawl a specific list of URLs.

You can only do this with the licensed version of Screaming Frog, but if you are not already using it, the cost is only 99 British pounds per year, and it is crazy useful for a thousand other things. A little more value for money than just subscribing to a more specific tool for just one month.

This way you will actually have a crawl result, not just a read from another tool.

Hope that helps,

Sha

Hey Dr Pete,

The Googledocs reference comes out of Matt Cutts' specific advice not to provide external links to the Webspam Team as they have security protocols which mean they will be unlikely to look at them.

It is included in the Webmaster video embedded on this page: http://www.rmoov.com/google-reconsideration-request-checklist.php

Unfortunately, the little advice that comes from Googlers varies wildly, so in the end, site owners need to apply a good measure of careful thought and common sense.

See you at Mozcon

Sha

Hi BeytzNet,

The answer to that question really depends on another question:

Are you looking for a short-term solution that may or may not get your current penalty lifted, or are you genuinely interested in dealing with links that really shouldn't be there?

If you're after the band-aid solution then you can try going with the arrogant suggestion from some Googlers that only links which offend Google at this point in time need to be dealt with. (Given Google's current propensity for adding to its list of what is "unnatural", their attitude borders on sadistic.)

If you really want to get some control over your backlink profile and future proof your site in the face of changing spam targets, impending Penguin updates and whatever else may be coming down the line, you might find it useful to try this little exercise:

Download backlink data from as many of the following as possible (free download limits for the tools you don't subscribe to will give you enough of a sample)

• Google Webmaster Tools
• Bing Webmaster Tools
• Open Site Explorer
• Majestic SEO
• ahrefs
• Raven Tools (pulls in data from Open Site Explorer & Majestic SEO)

Open each csv, select all and change text color so that the data for each list is a different color.

Copy and paste the content of each into one Excel spreadsheet so that all of the URLs are in one list.

Deduplicate the list.

Check out the different colored URLs left in your list...the takeaway is that every tool will bring you different link data. If you want a true picture of your backlink profile, you are now much closer to having it.

Incidentally, Google is not the only search engine to apply manual penalties. Others just don't talk about it as much as Google does. You might also find it helpful to read this post from Ryan Kent about identifying the source of your link penalty.

Hope that helps,

Sha

Hi,

I'm not referring to the type of data, but whether or not you have a complete view of your backlink profile. If you check the quite lengthy answer I left on your other thread you should see what I mean 

Sha

Hi Matthew,

As Mark suggested, create a spreadsheet in Google Drive and list all of the domains/URLs in the spreadsheet.

Add a single URL in the first field of the Spam Report form (the required field), then in the third field, provide a link to the spreadsheet (300 character max)

Something like: "I believe the sites listed in the googledoc at URL are using manipulative linking practices to influence search engine rankings"

Do be confident though, as Mark warned, that your client's site is able to withstand any scrutiny that might come with a review of sites in the niche.

Hope that helps,

Sha

Hi lucidsoftech,

The site is definitely indexed, but without knowing the specific URLs you are waiting on, it is a little difficult.

I ran a quick crawl with Screaming Frog and there are actually quite a few pages on the site which have the noindex meta tag in place - have you checked that the pages you are wanting indexed are not excluded with a tag?

Just a thought,

Sha

Hi sbrault,

I think your problem is more about whether you started with sufficient data.

Check my response on your other thread ... short answer is that the best tool is your brain (in conjunction with your eyes).

Be aware when using algorithmic assessment tools that it is quite common to get both false positives and false negatives!

Sha

Hi sbrault74,

No, links that have been disavowed will not disappear from your GWMT most recent links list.

Google has made it clear that a disavow instruction is treated as a suggestion only and since the links have not actually been removed from the web, they will remain visible in the list.

Googler John Mueller made this clear in a public Hangout a few months ago.

While it is difficult to make judgements about your process, whether you have provided sufficient detail in your reconsideration request etc, there is one area to consider that jumps out at me from your post.

It seems the only data source you have used to identify troublesome links is the GWMT list. Google provides only a sample of backlinks in this list, and we know from experience that every tool which provides backlink data will return a different list. While there will be some overlap (which means you will need to deduplicate your list), there is always a number of links found by each which do not appear elsewhere.

You should also make sure that you include links from any old SEO reports and other lists that may have been provided by link builders etc.

There is information in these slide decks that will give you some tips (and there are some almost invisible links in the slides) for data mining and email outreach:

Move or Remove, The Where, Why and How of Dealing with Penguins & Penalties

10 Link Removal Mistakes You Never Want To Make

and if you haven't read this post from Ryan Kent yet, you should do that before going any further.

If you are looking for tools and resources that might be helpful, you will find a short list of both, including a reconsideration request checklist in the sidebar of rmoov.com under helpful tools and resources.

Hope that helps,

Sha

Hi Jesse,

Without a little help from a googler in the forums there is only one sure way to determine whether you have a manual penalty or not - a reconsideration request.

BUT with the knowledge that you have of some issues and the absence of an obvious effect, I would not recommend the reconsideration route, as it is likely it may actually GET you a manual penalty which at this stage I am doubtful you have.

I would continue to do good work for your site and at the same time, put some effort into eliminating the most troublesome of the links you know about. Keep a close eye out for any effect (especially when news of the next penguin refresh appears).

also...good on you for taking a proactive approach to this and having the guts to make it an issue that gets dealt with.

Sha

Hi Lee,

Some good information here from Marie.

You might find it this Reconsideration Request Checklist helpful too: http://www.rmoov.com/google-reconsideration-request-checklist.php

Hope that helps,

Sha

Hi Jesse,

Since you have Basic with rmoov, it might be worth taking a look at the level of detail in the campaign summary report for your campaign (click the Excel icon in the campaign list to download). I think that will help you to get an idea of the type of information that might be useful to include.

I really think it is imperative that you try to get an indication as to whether the messaging you received is just for information or confirmation of a penalty. Given that you are not seeing visible effects, I am leaning toward the former, but as I mentioned before, that really means it's an opportunity to get things in order before disaster strikes

Sha

Hi im rubbish,

I would generally agree with Tom on the need to be completely honest with yourself when assessing links.

It is also a good idea to make sure that you have carefully read the finer detail of Google's quality guidelines. Make sure you know what qualifies as a link scheme in Google's eyes. This page provides some specific examples that might be eye opening if you have not read them recently.

If you think that the ongoing issue is caused by the appearance of more links, then there are a couple of things you should do:

1. Make sure that you disavow entire domains as Tom suggested, so that any other links in existence at that domain will also be disavowed.
2. Carefully monitor and identify new links as they appear. If it seems that unnatural links are still accumulating without explanation, highlight the issue in a reconsideration request and make sure that you provide scrupulous detail for the Webspam team to follow up. Be very clear - "these links have not been initiated by us".

Since you're in the UK it might be useful to know there are a couple of Search conferences coming up in the next couple of weeks that include sessions and workshops on link removal and penalty recovery.

BrightonSEO in the South - 11 & 12 April. I believe individual conference tickets are all gone, but there are still places in the Link Removal workshop with Tim Grice (which comes with a full day conference ticket included)

ionSearch in the North (Leeds) - 18 & 19 April. The conference includes both an Expert Panel on Link Removal and workshop sessions. Tim Grice will also be speaking here, as well as Christoph Cemper (owner of the company that develops Link Detox) and myself (rmoov.com).

A final note on starting again with a new domain: This absolutely has to be purely a business decision. With the obvious amount of time, money and effort put into your site's branding to date, even stripping back the entire link profile and "rebooting" your domain is likely to be a better business proposition than having to recreate an entire brand.

Hope that helps,

Sha

Hi lucidsoftech,

The following is from the Crawl Diagnostics section of the SEOmoz Help Hub and might help identify the issue:

Why didn’t you crawl all my pages? I only got a one page crawl. Looks like you missed a bunch!
If you suspect you didn’t get a full crawl, or Rogerbot missed some of your pages, there could be several reasons why this happens.

• We only crawl a maximum of 400 links per page. If several pages of your site all have the same 400 links on each page, we may not discover all the pages on your site. Try optimizing your navigation to reduce the number of links.
• Does your navigation rely on JavaScript? Can visitors navigate your site with JavaScript disabled? SEOmoz doesn’t crawl JavaScript, so make sure your links work in all browsing environments.
• Does your site consist of multiple subdomains? Crawls are restricted to the subdomain you set your campaign up on. This means that in general, we don't crawl multiple subdomains. You can solve this by specifying a “Root Domain” crawl in the setup process. (This requires starting a new campaign.)

I have also run across a couple of single page crawl issues in the past which related to secured pages and an edge case. This was identified and sorted out very quickly by the SEOmoz Help Team.

If the suggestions above don't identify the issue for you, then best to go to the Help Hub and click the Contact Our Help Team button to get in touch with them direct.

Hope that helps,

Sha

Hi Jesse,

First and foremost, you need to determine exactly what the messaging means for your site. Is it a warning which indicates that there may be a manual action applied to your site, or is it actually just a notification intended to tell you that Google is discounting unnatural links that have been detected.

The most efficient way to get an answer on this is to go the Webmaster Central forum and ask the question "Does this mean that my site is under a manual penalty?". (I would not discuss the links you already know are a problem, just the need for clarity in the messaging).

I have seen a number of instances where Google's John Mueller has taken a look at the specific site and advised webmasters that the message is actually just an advice that they have detected unnatural links and are discounting them at what they refer to as a "more granular level". If this is the case you do not need to lodge a reconsideration request, but you should take it on notice that your backlink profile needs some careful scrutiny and any troublesome links should be dealt with as soon as possible. Basically, it's a "heads up" and you should take it as such.

Second, over 10 months of providing phone support for rmoov.com I have spoken personally with hundreds of site owners who are confused by the messaging they have received or not received from Google.

From my discussions with these people I know that there are a large number of sites which have a manual action in place and have never received a message from Google of ANY kind. Clearly, not everyone who has a manual action applied is sent a message, despite assertions to the contrary.

On the more helpful messages, out of the hundreds of people I have spoken with, barely a handful have ever received one of these more helpful messages.

While I am sure that Matt sincerely believes the information he provides in his answers is accurate, the fact is that what is actually happening within the webspam team does not match his information. In my view this is the fundamental issue preventing most people from dealing effectively with their situation.

On the subject of removing the links, you have a bit of an advantage in knowing where the paid links have come from

First deal with these paid linking arrangements.

Since you mention that they have been acquired from sources that are likely to bring with them their own problems, I would not recommend trying to remediate those links by applying a nofollow tag. So, first job is to ensure that every payment arrangement that still exists has been cancelled. Record evidence of these cancellations by keeping screenshots and/or email receipt text.

Add any where there is no payment arrangement to your list to contact.

With the low response rate you indicated, the next thing to focus on is increasing the effectiveness of your outreach:

• Make sure the text of your email is friendly and personal. Remember that your email could well be one of hundreds already received by the webmaster that day. Show some understanding for their situation, explain briefly why you are making the request and ensure that you provide details of the URLs where the link(s) can be found in their site. Ask nicely and thank them for their help.

• Help the webmaster to tick their due diligence boxes (ensure that your outreach email originates from the domain you are trying to clean up, use a real name and provide contact information so that they can match them easily to information that appears on the site).

• maximize accuracy by obtaining as many different contact methods as possible (email addresses from the site and the WhoIs record, contact forms where you can submit a request, twitter handles, google+, facebook & linkedin profiles, telephone numbers, physical addresses). Use whatever works for you.

• send polite reminders at reasonable time intervals. At rmoov mail is sent every 3 days for the life of a campaign. Don't assume because a webmaster did not immediately remove your link that they aren't willing to do so. Again, your request could be one of hundreds ... sometimes a couple of gentle reminders will help.

• Pay particular attention to undeliverable notifications returned from your outreach. Some privacy protected domains do not accept mail, but will provide a contact URL that you can use to contact the webmaster.

• Document everything. Keep copies of responses and make notes on how you will handle those domains. e.g. "Webmaster advised a cost of USD $500 is required to remove each link. We believe this to be an unreasonable demand, so have chosen to disavow all links from the site at the domain level".

Finally, disavow only after all attempts to remove links have been exhausted and disavow at the domain level unless you have a very good reason not to.

Well, this is now way too long, but hope it helps.

Sha

Hey James,

Welcome to SEOmoz

It's a wonderful community and quite addictive too!

Here's a post that I made a while back that might help you find your way around ... a couple of extra resources in the comments too.

Look forward to seeing you around in the future.

Sha

Hi James,

No worries

To give googlebot a little help, you can use the "Fetch as Googlebot" feature in webmaster tools to introduce the page to the crawler. Once the URL has been fetched, click the "submit to index" button to give it a head start.

Sha

Hi Mirco,

The SEOmoz Help Hub is the place you need to visit

Check the menu at the left of the page to find the Getting Started Guides for all of the different features and tools available.

You will also find some great posts in both the SEOmoz Daily Blog and Youmoz User Blog that provide explanations and real examples of how to use specific tools.

Here are a few that are great value:

The Best Kept Secret in the SEOmoz Toolset - Rand wrote this great post when the tool was first developed, but has updated it as features have been added and refined.

Announcing Fresh Web Explorer - This post from Matthew Brown explains SEOmoz newest tool and it's an awesome tool!

Using Followerwonk to Grow Your Twitter Account - This one is a Whiteboard Friday with Rand and makes it easy to understand how valuable this tool can be.

...and for more Whiteboard Fridays try this list

Also, don't be afraid to ask a question here if something seems difficult to understand ... there are some pretty awesome people here in Q&A who hail from Italy, so I'm sure help will always be at hand.

Hope that helps,

Sha

Hi Bob,

The term you referenced "the 4 step belief change" appears nowhere else but in the Title.

The shortened term "4 step belief change" also appears in the meta description.

Neither term appears anywhere within the text on that page.  So the primary issue seems to be that there is nothing on the page to signal relevance for that term to search engines

Hope that helps,

Sha