Farmer Update Case Study. Please question my logic here. (Very long!)
-
Hi SEOmoz community!
I would like to try to give a small (well...) case study of a Farmer victim and some logical conclusions of mine that you are more then welcome to shred to pieces.
So, I run MANY sites ranging from low to super quality and actually have a few that have been hit by farmer but this particular site had me scratching my head as to why it was torched.
Quick background: Sitei s in a very competetive niche, been around since 2004 initially as a forum site but from 2005 also a content driven site. Site is an affiliate site and has been ranking top 5 for many high-value commercial KW's and has a big long-tail of informational kw's. Limk profile is a mix between natural, good links and purchased links from various qualilty sources.
Content is high quality written articles, how-to's, blog posts etc. by in-house pro writers plus UGC from a semi active forum (20-30 posts a day).
Farmer: After Farmer, this site's vertical is pretty much same as before with the biggest exception being my site. I quickly discounted low-quality content (spider-food) and focused instead on technical reasons. I took this approach since this site isn't the most well kept site I have and I figured the crappy CMS + PHPBB might have caused isseus.
I didn't want to waste my time crawling the site myself so I quickly downloaded all the URLs that Majestic had crawled. Too my surprise the result of Majestic's crawler was over 3 million URLs when the real number would likley be 30-40k and Google has about 20k indexed.
After scanning through the file with URLs I knew I had issues. Massive amounts of auto-generated dupe pages from the forum and so on. By adding around 20 new lines to robots.txt I was able to block millions of pages from being crawled again.
My logic: Ok, so now I think I've found what caused the drop. Milllions of dupe pages and empty pages could have tripped the Farmer algo update to think the site is low quality or dupe or just trying to feed the spiders with uselessness.
My WEAK point in this logic is that I can't prove that Google even knew about (or smart enough to ignore them). Google WMT tells me they've crawled an average of around 10k pages the last 90 days. Given this I'm doubting my logic and if I've found the issue or not.
My next step is to see if this gets resolved algorithmically or not, if not i feel I have a legitimate case to submit a reinclusion request but i'm not sure?
Since I haven't been a contributing member to this community I'm not looking to get direct help with my site, but hopefully this could spark some discussion about Farmer and maybe some flaming of my logic regarding the update
So, would any of you have drawn similar conclusions as I did? (Sweet blog bro!)
-
Good to hear that all your rankings have recovered. How have things gone the past couple of weeks, do you have anything you can share here? Or maybe even for a YOUmoz post?
-
The site is back, all rankings recovered.
-
Hi Barry,
Thanks for bringing up some great points!
I probably should have talked about the effects the update had on the site in my op Well, it was a drastic drop in rankings for pretty much everything the site ranked for, at least 20-40 positions drop acroos the board EXCEPT for the most commercial/highest volume KW's that only resulted in a 3-6 position drop (still top 10 for the most part.
The landing pages, that dropped the least, for the commercial queries is obv. where all the paid links go to (mostly from sites in my niche that are still doing fine). The informational pages that got hit the worst only ranks because of natural links from great/good sites that were not affected by farmer and domain authority.
I really see your and Dan's point about Google not caring about the millions of urls but i can't shake the feeling that it might have tripped the wire somehow
FTR, I'm not complaining about my situation, just generally surprised about this site getting hit when I have a ton of sites that deserved to be torched when I feel this site is actually "clean". I think this is why my logic seems strange to you
Anyway, another great reply, thanks!
-
What's actually changed for you though? Have you looked in the analytics to see what pages are no longer bringing traffic, what keywords are no longer bringing you traffic, that kind of thing?
Is it across all pages and all keywords or is it just a few high traffic keywords and pages?
Just because your niche has many link buyers doesn't mean that you're not getting penalised for it
I don't think Google will have known or cared about those millions of pages, I assume none of them have shown up as a landing page for visitors so I would guess they were effectively invisible to Google.
Paid links (and indeed normal links), you may not be getting penalised for them, but if some of your highest value links are themselves being punished or in some way devalued you may be losing out there as well.
I asume no other significant change occured at this time?
-
Hey Dan,
Thanks for replying! I figured the purchased links would come up but I have pretty much discounted that since the niche is crowded with link buyers and no-one got hit. I'm also active in other verticals where lik buyers prosper and I haven't seen any impact on just about any of them.
In comparison, my sites link profile is pretty vanilla compared to many other sites. That said, I know I can't discount the links 100% as being the reason here since I've been paying for them.
Really appreciate you taking the time to reply!
-
My first knee jerk response to this post is to target the part about purchased links. If you have paid links, then I would remedy that problem before I look at anything else.
Do you have a lot of adsense on this site? I've been hearing left and right that a lot of sites that got hit hard were those with 5 or 6 adsense units on one page. Excessive ads drive users crazy, so Google could be torching you for that.
As for the auto generate dupe content pages, Google may or may not have found them. Were there links to these pages? Do you know how Majestic found them?
If you never linked to these pages it is unlikely Google ever found them. Google tends to only crawl content with links or found in a sitemap. If you never had links to those dupe pages and they weren't in your sitemap, I doubt it is causing the problem. Plus if that were the issue, you probably would have been torched long before this algo update.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
What is left ethical? What is working for offpage SEO? Very long write up in here and my take on things.
Hello,
White Hat / Black Hat SEO | | MarketingOfAmerica
Please ignore misspells and grammar, this was typed quickly as I am spending my time researching not writing a perfect book on it. My goal is to find ethical very hard to get links unlike guest posts which are now dead according to Matt Cutt's blog here http://www.mattcutts.com/blog/guest-blogging/. My journey started with a quick message to Rand Fishkin, he responded the following "Hi Matthew - thankfully, there's literally hundreds of link building methodologies that are still completely legit. Check out http://moz.com/blog/category/link-building and you'll find tons and tons of them. The key is that none are easy, none are particularly scalable, and all of them require doing work that will add value for searchers, for your brand, and for your overall marketing - which is exactly what Google wants to count. Wish you all the best," Thanks Rand Fishkin! So I started my search looking for links that are hard to get other than those that are directories, forum links that are dead and spammy, blog comments which are overused, guest posts, or any type of black hat link. I figured I would start to check what other popular SEO companies were doing and that have been at the top through many of the updates. After running an analysis on the term SEO services I found the following Test 1. I analyzed Main Street Host to start with. If you type in SEO services in Google you can see they are rank 1 for it. After a quick analysis it's easy to see that they have 100's of footer links on clients that they have, some with exact match anchors and some without. My question is, is why is this a viable tactic? Lets take for example the following. If you pull up their http://www.opensiteexplorer.org/ stats and look at the inbound links you will come to an exact match anchor right away that says SEO Marketing Company. I went to the weebly link that they have and found that they have put their name at the bottom of this page. Issue 1 - Why is it ok for this type of link, but it's not ok for a template link? Aren't these links suppose to be penalized? Issue 2 - Nothing on the page is even relevant to their link at all. As we have read before, you need to have links surrounding relevant text. Take a look at their backlinks you and you will find almost all of their high quality links are exact match anchors coming from their clients surrounded by irrelevant text. Why is this working? How is this different than a network? What stops someone from just starting a network and dedicating 1 footer link to a full site and putting up dummy info... Anyone can go to Godaddy and purchase a DA 40+ site or so and throw up $20 of content and a footer link. As I dove deeper into finding what is ethical and working I discovered many of the top SEO companies use this. Not just one, but over 20 of them use this same method. Lets use another example. So I started to look at what they did for their clients. How did I know who they worked for? Simple I assume that since they have their link at the bottom of the page and claim that they do SEO for them, they are indeed working for them. So I analyzed the site we talked about a while ago on the Weebly that they had their link on. It's the Valley Art Weebly link if your checking yourself. I quickly found that they are using a network to rank up some of their clients as well. For example http://firesidebookshop.com/index.html Take a look at the link on this page leading to the art place. At first glance the site doesn't look spammy, but try to buy a book, or even order one. Who has an online book store, but doesn't sell books lol? Who also puts interesting links on their home page? This screams network to me. I am willing to bet the following will happen - Matt Cutts and his spam team will ad something like the following to the algorithm or whatever you would like to call it "ignore link if total outbound dofollow links on full site = x amount or higher" = internal Google disavow tool = bye to guest blogging. So what is everyone going to do? Okay it's time to figure out what that number is right? Lets do some tests and lets say that magic number is 5 to 10 links on a whole site. What does this do? This drives the price of quick SEO up again evening the playing field for others using ethical SEO like myself. How do I figure this? Lets face it black hat SEO will never end as long as someone is able to do it. Now since guest posts are gone, the quick link on quality sites surrounded by enough text to count is gone. This means that it will cost extra money, because everyone will be forced to put a max of x amount of links to be safe and for the links to get noticed on a website. So now they have to purchase an established domain that is high enough quality to pass the correct link juice through to a clients site that they want to rank up. Lets figure a few dollars for a unique IP, another few for the hosting, $40 to $100 for the domain if your lucky on Godaddy auctions, and then $40 for the content to make it look realistic if your getting it for $0.01 a word. Plus the time it takes to setup your site. This price of that $30 Odesk guest post backlink just went up to a min of $100 or so. Diving deeper into what's working and moving past the networks, because I feel this will only work temporarily as well if you are brave enough to use this and I know I am not. It doesn't seem to ethical to me at the end of the day even though some may argue, you are just creating more relevant websites which can maximize your traffic streams. The problem is I have stopped here and am stuck. Sure I have looked at http://moz.com/blog/category/link-building and read the most recent post where it talks about 31 types of links. Most of those links don't apply or are outdated and you shouldn't use them. Some of them talk about forum links,directories, bookmarks.. Those have been tactics for years and sure you may find 1 out of 1000 that are good, but the rest are just spam. I have been over to search engine land, and a handful of other sites. I have talked to many other SEO's as well. They are emailing me asking what they should do after guest posts, because they are unsure. The question is, what is ethical? Let say you have a plumber, or a roofer, .gov links are nearly impossible for them and quite frankly that seems spammy to me to even post them on one. I know what many are going to say, build links as if your not worried about Google and you will grow.. Where are you going to build the links to if everything is unethical? As we know clients will walk if they don't see improvements quickly. What's quickly? I would say around the 3 to 6 month period using ethical SEO. Sure there is onpage, a great blog, etc., but what is there left truly ethical for offpage SEO besides some good press releases, some social profile links like a pinterst, and the normal? I must be missing something! I am not looking for the easy way, I am not afraid to get my hands dirty and work hard. If anyone can show me a quick example of a truly ethical link I would be grateful to see this. I can't seem to wrap my head around something that I can do that will last at this point. If you don't want to share it to the world, please PM me. [edited for formatting by Keri Morgret]0 -
Will blank category pages automatically get updated
Hello, We've got old category pages that are blank like domain/shoes.html (blank white page not in menu anymore) domain/newshoesurl.html (working URL with link in menu) Will the blank pages be automatically deindexed and updated by Google?
White Hat / Black Hat SEO | | BobGW0 -
The purpose of these Algo updates: To more harshly push eCommerce sites toward PPC and enable normal blogs/forums toward reclaiming organic search positions?
Hi everyone, This is my first post here, and absolutely loving the site and the services. Just a quick background, I have dabbled in SEO in the past, and have been reading up over the last few months and am amazed at the speed at which things are changing. I currently have a few clients that I am doing some SEO work for 2 of them, and have had an ecommerce site enquire about SEO services. They are a medium sized oak furniture ecommerce site. From all the major changes..the devaluing of spam links, link networks, penalization of overuse of exact match anchor text and the overall encouraging of earned links (often via content marketing) over built links, adding to this the (not provided) section in Google Analytics, and the increasing screen real estate that PPC is getting over organic search...all points to me thinking on major thing..... That the search engine is trying to push eCommerce sites and sites that sell stuff harder toward using PPC and paid advertising and allowing the blogs/forums and informational sites to more easily reclaim the organic part of the search results again. The above is elaborated on a bit more below.. POINT 1 Firstly as built links (article submission, press releases, info graphic submission, web 2.0 link building ect) rapidly lose their effectiveness, and as Google starts to place more emphasis on sites earning links instead - by producing amazing interesting and unique content that people want to link to. The fact remains that surely Google is aware that it is much harder for eCommerce sites to produce a constant stream of interesting link worthy content around their niche (especially if its a niche that not an awful lot could be written about). Although earning links is not impossible for eCommerce sites, for a lot of them it is more difficult because creating link worthy content is not what eCommerce sites were originally intended for. Whereas standard blogs and forums were built for that exact purpose. Therefore the search engines must know that it is a lot easier for normal blogs/forums to "earn" links through content, therefore leading to them reclaiming more of the organic search ranking for transaction and non transaction terms, and therefore forcing the eCommerce sites to adopt PPC more heavily. POINT 2 If we add to the mix the fact that for the terms most relevant to eCommerce sites, the search engine results page has a larger allocation of PPC ads than organic results (above the fold), and that Google has limited the amount of data that sites can see in terms of which keywords people are using to arrive on their sites, which effects eCommerce sites more - as it makes it harder for them to see which keywords are resulting in sales. Then this provides further evidence that Google is trying to back eCommerce sites into a corner by making it more difficult for them to make sense of and track sales from organic results in comparison to with PPC, where data is still plentiful. Conclusion Are the above just over exaggerations? Can most eCommerce sites still keep achieving a good percentage of sales from organic search despite the above? if so, what do the more niche eCommerce sites do to "earn" links when content topics are thin and unique outreach destinations can be exhausted quickly. Do they accept the fact that the are in the business of selling things, so should be paying for their traffic as opposed to normal blogs/forums which are not. Or is there still a place for them to get even more creative with content and acquire earned links..? And finally, is the concentration on earned links more overplayed than it actually is? Id really appreciate your thoughts on this..
White Hat / Black Hat SEO | | sanj50500 -
Question about local SEO when you serve many more cities than you have brick and mortar locations
My URL is: http://www.mollysmusic.org for the record.I run a music school that serves in-home lessons to a whole slew of cities. Since I only have 3 brick-and-mortar locations, I can't make google local profiles for all the cities served, but I want to get seen by those people searching in their own cities. Right now, our biggest competitor, takelessons.com, is top ranked for every single city you can think of, because they have individual web pages for every city served. Their content is repetitive and scrapey, and to me, that says "doorway page" which supposedly can get you de-indexed. I'm reluctant to do that because I'm afraid I'll get banned, but I have to compete. I also want a strategy that can scale when we move into new areas. Is there something that makes TakeLessons's content NOT a doorway page? What's the best practice for getting ranked in multiple individual cities if you run a service? Thanks in advance.
White Hat / Black Hat SEO | | mollysmusic0 -
Duplicate Content due to Panda update!
I can see that a lot of you are worrying about this new Panda update just as I am! I have such a headache trying to figure this one out, can any of you help me? I have thousands of pages that are "duplicate content" which I just can't for the life of me see how... take these two for example: http://www.eteach.com/Employer.aspx?EmpNo=18753 http://www.eteach.com/Employer.aspx?EmpNo=31241 My campaign crawler is telling me these are duplicate content pages because of the same title (which that I can see) and because of the content (which I can't see). Can anyone see how Google is interpreting these two pages as duplicate content?? Stupid Panda!
White Hat / Black Hat SEO | | Eteach_Marketing0 -
Yet another Negative SEO attack question.
I need help reconciling two points of view on spammy links. On one hand, Google seems to say, "Don't build spammy links to your website - it will hurt your ranking." Of course, we've seen the consequences of this from the Penguin update, of those who built bad links got whacked. From the Penguin update, there was then lots of speculation of Negative SEO attacks. From this, Google is saying, "We're smart enough to detect a negative SEO attack.", i.e: http://youtu.be/HWJUU-g5U_I So, its seems like Google is saying, "Build spammy links to your website in an attempt to game rank, and you'll be penalized; build spammy links to a competitors website, and we'll detect it and not let it hurt them." Well, to me, it doesn't seem like Google can have it both ways, can they? Really, I don't understand why Competitor A doesn't just go to Fiverr and buy a boatload of crappy exact match anchor links to Competitor B in an attempt to hurt Competitor B. Sure, Competitor B can disavow those links, but that still takes time and effort. Furthermore, the analysis needed for an unsophisticated webmaster could be daunting. Your thoughts here? Can Google have their cake and eat it too?
White Hat / Black Hat SEO | | ExploreConsulting0 -
Landing Page or Doorway ?- that is the question!
Hi Guys, So, I'm looking at a project to build a series of landing pages that cross map cities with Suname. E.g. Sydney + Smyth, New York + Fitzpatrick. On those pages I'll pull in from our directory relevant name based listings and try and display some other tailored / information. The page itself is the end goal - it is definitely not a doorway in the classic sense of encouraging someone to then go on the main site. I want the user to fill out a form on this page because they realise they've landed on a valuable service. I'm looking at potentially 500 names against 2000 locations, creating 1,000,000 landing pages. Although some of the content will be repetitive I genuinely believe someone doing the appropriate search and finding our page will derive value from our page as our whole business is designed to answer their needs. However I'm worried that Google may classify these pages as doorway pages. Could anyone please shine the light of experience on this for me? Thanks!
White Hat / Black Hat SEO | | flow_seo0 -
Is it possible that since the Google Farmer's Update, that people practicing Google Bowling can negatively affect your site?
We have hundreds of random bad links that have been added to our sites across the board that nobody in our company paid for. Two of our domains have been penalized and three of our sites have pages that have been penalized. Our sites are established with quality content. One was built in 2007, the other in 2008. We pay writers to contribute quality and unique content. We just can't figure out a) Why the sites were pulled out of Google indexing suddenly after operating well for years b) Where the spike in links came from. Thanks
White Hat / Black Hat SEO | | dahnyogaworks0