How do you evaluate algorithm changes?

[RobertFisher]

First a request: Please answer this only if you own/manage more than 5 to 10 sites. Also, please understand I am trying to go beyond speculation. With the current "pigeon" update (as coined by SEL), we are seeing a lot of changes, good and not good. An example is within the real estate vertical. There is literally no "pack" whatsoever. Page one for every major real estate search in Houston: houston real estate, homes for sale, townhomes for sale, etc. is now populated by a major RE vertical player. All of these companies use the real estate rules, hire a broker, then scrape RE data from other MLS brokers.

So, there is not a single Houston real estate company showing on Page one for any of these searches. Zillow, Trulia, HAR, etc. dominate the entire organic page. Within other verticals it is a mixed bag of results. A home service provider shows up with their outlying offices for searches on that service and Houston. Etc.

Just curious how those of you who handle multiple sites analytically approach learning what is going on. I know what we do, but am very interested in others methodologies. 
Best

EDIT for Clarification: I am speaking to how do you evaluate the changes (the effects of) over a fixed period in order to draw a conclusion as to what has changed, how that affects your clients,  and what you will do now given the change and the knowledge you have gained.

[RobertFisher]

Thanks EGOL. I shall read it.

Robert

[EGOL]

Dr. Pete has posted about this on the Moz Blog.  Four of these changes in June/July alone..

http://moz.com/blog/the-month-google-shook-the-serps

[EGOL]

Good point, Doug.

I have one site, built about ten years ago, that I have not edited in several years.   It is a Panda barometer.   It gets hit, comes back as strong as ever for several months, drops again, comes back. It has been in and out several times.  (currently it is out and on the first page of google for its main keywords)

This is a site that receives a lot of traffic in a rather uncommercial niche - so it is not a priority for me to fix.  I have work that is more enjoyable.  Even when hit it gets xxx,xxx visits per month.  It is interesting to see that Pandas can not make up their mind about what to do with some sites.

So, my own advice, on my own sites, is sometimes to.  Do nothing.   Doing something costs money.  Do something if the potential gain is worth the work or the money.

[DougRoberts]

Oh and it'seems important not to panic. We've seen updates get reversed and updates that get scaled back when their impact is just far too drastic.

I'very been watching one site that got hit for about 30 days before search traffic was restored only got the site to get hit again a month later.

[EGOL]

I have seen Panda problems on my own sites.   I know a lot about Panda and know my sites well, so I can look at the sites with an eye that can pick out the content that could be causing the problem.

Some of these sites have pages that were made over a decade ago.   They were great pages in that day but if I was to make them today, knowing what I know, they would be a lot better.  So when Panda hits, I have a good idea of what might be causing it.  It has always been legacy content.  The first two times Panda arrived I picked some pages to axe, picked some to improve, and picked some to noindex.  Panda went away a few weeks later.  Today I have another encounter with Panda, and I have taken the ax, again, to some of this legacy content and will see if Panda goes away again.

These are all sites that I have been involved with building every page and I know them very well.  Still, I am lucky to know a couple of people who are experienced with algo problems and know my site very well.   I see what they have to say about my proposed changes before I make a final decision on amputations or improvements.  That's what Panda usually requires.

However, if I was an SEO and a client brought these sites to me, I would get familiar with the site, see the legacy content and it would send up a red flag.  So, another thing that I would do if I was an SEO receiving these sites would be to  give that new client a list of content that might not work well with Panda.  Then, if they see a traffic drop on a Panda date they will know what I am going to tell them to cut or fix or noindex.  Will not be a surprise to them.

And, if they are smart and want to mitigate they can decide to amputate or improve now.  My own decision has been to wait until I find out if it is a problem because I think that I can take care of it quickly. But I would be more nervous about doing this with a client, so would inform them of my concerns right away.  It is possible that the things that I see as concerns have already hit their site, so a long-time-line traffic review - back to about 1/1/2011 - would be done to look for historic hits.  We might be able fix the problem and get them a traffic improvement that they were not expecting.

[RobertFisher]

Doug,

Great to hear from you and I think you make a massive statement when you say, "Another thing to consider is how an algorithm update might change the search terms that people use to find your clients." This is truly an experiment worth taking the time to develop.

All the best,

Robert

[RobertFisher]

EGOL,

This is truly well put and worthy of multiple thumbs up. There is never a qualifier that would keep you from answering a question of mine and this is the reason. What I did not want was speculation.

You are so right about Google's original model vs. today and about how these changes are slowly cutting out sites and affecting wallets. Your paragraph:

OK... Now you ask about what to do for your client.  First.  Explain that he has a little algo risk... and that he has a little competitor risk... but also explain that even if you achieve the miracle of getting his website to one of the top ten positions of google that he will still be subject to a much greater risk for which you have absolutely no control.  That is SERP format risk.

is exactly what we do with every client. We have tons of data before we take a client on and we are very clear about writing checks with our mouth that our ass cannot cash (we will get you on page one, etc.). We just state we will never make that kind of promise and elucidate the reasons. The beauty is we refocus them on actual business results as opposed to rankings. That, we can affect.

I have to say this latest change is really interesting in good and bad ways around some service pages (good) and then real estate (horrific). It will be interesting to see where we go from here.

Thanks and triple thumbs up,

Robert

[DougRoberts]

Hi Robert.

Sounds like you're having plenty of fun!

From my experience, there is only one thing that you can really do to establish if the changes are algorithmic or not and that is to benchmark your clients sites against their competitors.

If you know how very similar sites are being affected by an algorithm update you can always tell if it's just you or all the similar sites in your niche.

It's not enough to pick a few random sites in that are ranking in serous for your main target terms. You need to dig a little deeper than that.

Who are your clients real competitor.  Consider their target audience (demographics as well as geographical locaton). What about the size of the business and their suspected online budget. What about the approach they'be taken on their site, blogging frequency etc?

You'll want to monitor the performance of sites/businesses that are very similar,  somewhat similar and dissimilar but related.

It can be reassuring to a client to be told that even though there'should been a massive shake up in the serps - they are still out raking their competitors.

Another thing to consider is how an algorithm update might change the search terms that people use to find your clients.

If the serps are now being dominated by large players, does this mean there is a knock on effect as searchers start using more specific keywords to find the little guys/in dependants?

Or don't they give a stuff and are just happy to go with whoever appears at the top of the search results.

My tips:

Don't just rely on rank trackers - make sure you take screen shots or capture the serps so you can determine the winners and loses following an update. You can then use this info to find out where the top 10 sites are ranking following an update. (Hard to do if you'be not tracked who they are)

Keep an eye on webmaster tools and track the impressions as well as clicks. Impressions may show the impact of an algorithm change more obviously than rank tracking or looking at your organic search numbers.

Make sure you keep a fantastically good record of every change that happens to the site and make sure your client understands the need for a robust change control process.

When trying to determine if it's an algorithm change or just a cockpit ... I'd be more likely to put my money on "cockup".

[EGOL]

I might not meet the stated qualifications for replying... and I am not going to talk about algos... but I will say that I have very carefully watched a small number of sites for over a decade and have experienced several huge losses in income caused by something that Google had done.  So, from the standpoint of wallet smoking, I feel that I have a good reply.  I have also been a student of Google - and shareholders.

What I am going to explain here is a long-term trend for Google.  Google used to be a very crappy webmaster.  They gave all of their traffic away.  Google used to be a revolving door.  Traffic came in and went out.  Google kept little of it.  Google monitized only a fraction of the searchers.

Over the past few years google has become a better webmaster and has become better at keeping traffic on google.com or giving a higher percentage of that traffic to people who are paying them.

Shareholders want revenue growth.  Google has owned almost all of the USA search volume for over a decade. So they are not going to get growth from claiming more marketshare.  Instead they are going to get growth from being a better webmaster.  They are going to get growth out of making more paid clicks.  That means your clients and their organic or local traffic are in the crosshairs.  For the past several years they have been the targets of google's revenue growth.  That is not going to change.   Your client's current profits are google's future growth.

OK... Where has made my wallet smoke?  One day I wake up in the morning and find one of these.  These traffic losses are not temporary, they are not recoverable, they are permanent.  BAM!

When my income has taken the greatest hits.. the hit was usually NOT caused by an algo, it was caused by a change in the format of the SERPs... and since my competition is usually confined to organic listings I have been clobbered by these..

-- In the early days of Adwords, the ads were only displayed on the right side of the SERPs.  When Google got the smarts to display them above the SERPs my retail income was hammered.

-- One day google decided to start displaying a sample of the image search results near the top of the SERPs.  I have a couple sites that are heavily based upon images and those sites lost an enormous amount of traffic.  Tens of thousands of visitors per day disappeared overnight.  More wallet smoke.

-- Then google added "Product Search Results" to the top of the SERPs.  My retail sales for pages ranking in the top three postions of the organic got hammered.  I decided to jump in to product search. Did that and got some traffic and sales back.  Now product search is pay-to-play.  Google grabs some of my profits if I want to play or some of my total revenue if I ain't going to play.

-- Then google decided to change the image SERPs.  Instead of the visitor clicking on my image and being delivered to my site they started to show my site in a frame.  Lost a little income that day.  They they decided to display my image on a page of google.com and only have a tiny link to my site.

-- One day I wake up and google is showing a big map at the top of the SERPs.  Sometimes it has pushpins, sometimes it doesn't.  Familiar smell of currency and leather smoldering in my back pocket.

-- One day google started showing carousel at the top of the SERPs.  Now, instead of the visitor clicking straight to my site it clicks straight to more SERPs.

-- This year Google started adding knowledge box to the top of the SERPs.   (Just my weak minded observation... when they did this they demoted a large number of informational pages.  Go search for "jade" the green gem.  Not long ago that SERP was filled with oriental history pages, museum pages and pages of vendors selling the green gem.  Now the organic SERPs on the left side of the page are very different.  Wikipedia is there but the museums, sellers and oriential culture pages are gone.  Instead they have a knowledge box that answers light queries, If you are lucky your site might appear in the knowledge box.  But they have obvious links to other Google queries in large BLUE font the knowledge box but links in microscopic black font to your website.

OK... Now you ask about what to do for your client.  First.  Explain that he has a little algo risk... and that he has a little competitor risk... but also explain that even if you achieve the miracle of getting his website to one of the top ten positions of google that he will still be subject to a much greater risk for which you have absolutely no control.  That is SERP format risk.

So, I would take a screenshot of his target SERPs on the day that he signs your contract... and when he calls you about his wallet smoking you will be able to show him that google has changed the playing field by changing the format of the SERPs.   Your clients profits are the field that google is plowing to get the growth that their shareholders demand.

[GPainter]

Tough one, had some similar troubles most people seem to hire SEO people without knowing (or caring) what we do only that they "should" have some SEO. So normally you have to explain what you've done or what has happened. So you might find out where the organic traffic has some from ( is it a keyword metric they were not expected, a post some where that's perhaps driving traffic etc.)

Now assuming this is update related and you've (and there has been a period of consistency) it comes down to the fun time of explaining it all.

First off that you're not an internet Wizard

It was (if applicable) caused by a update what the update is intended to do why it did it o your site etc. etc.

Now combine with the above it can be what you've done that could of helped this, e.g We updated your local (or created) listings and a new update has helped this promote and thus given you a boost or visa versa this update has come out and you've seen a drop but here is what we're doing to fix it. Now I'm sure about here you're thinking "yeah, yeah I know all this pretty standard"

Well it really comes down to how you want to play it, you could say "yeah: We're internet wizards-high five" or maybe you do some heavy digging and find out that a popular twitter users found some content you wrote tweeted about it and some people have gone looking for your client, maybe its none of the them and you just say "there was an update we've done this for you, but there seems to be more to it.."

To summarize it really comes down to each client, some just don't care some what lots of numbers others just want a quick summary. Most of the time it involves some detective work and looking through page and pages of boring stuff to figure it out and sometime after doing that there is still some head scratching.  If organic is up but rankings are down is direct visits etc. its trying to find the good old fun "not provided" link!

Hope my reply was helpful (sorry if it wasn't on track). I'm sure everyone has their own way of dealing with it but I recommend you get one of these and most of the time explaining to the client (depending on their size) like you would a normal person (and that you're not an internet Wizard!)

Thanks

[RobertFisher]

I guess I could have posed the question more clearly. We track pages daily. With large sites and a multitude of sites we are dealing with more than keyword tracking. There is no way to win as an agency trying to track 50 to 100 terms daily for every client. You are concerned more with performance over time. 
With a large home services client, they have fewer page one rankings than when they came on board with us but their business is up by 50% over 14 months that is trackable to organic methods. With them and others there we are tracking key pages, etc. but I am more interested in how you draw a conclusion about the change in the algorithm and as the result, how will your methodologies change, what will you tell your clients.

Thanks again.

Robert

[GPainter]

Hah know the feeling, keep seeing one particular company spam there way to top. They hang about for couple of weeks then vanish then a month or two later they pop again with a new site having redirected their old site. Crazy the amount of brokers too!

I tend to bulk track my keywords which is updates a few times a day so if I come in one morning and see a large movement across a few sites I tend to take a look a the likes of mozcast, algoroo etc. and go and take a look at my twitter or some seo forums etc. see if other people are chatting then its the fun game of trying to work out what changed..

I also sometimes find there can be "fake" results when I say this I mean there might be what looks like an update but its normally just Google testing or similar so at least give it a few days to see if it would settle.