Farmer Update Case Study. Please question my logic here. (Very long!)
-
Hi SEOmoz community!
I would like to try to give a small (well...) case study of a Farmer victim and some logical conclusions of mine that you are more then welcome to shred to pieces.
So, I run MANY sites ranging from low to super quality and actually have a few that have been hit by farmer but this particular site had me scratching my head as to why it was torched.
Quick background: Sitei s in a very competetive niche, been around since 2004 initially as a forum site but from 2005 also a content driven site. Site is an affiliate site and has been ranking top 5 for many high-value commercial KW's and has a big long-tail of informational kw's. Limk profile is a mix between natural, good links and purchased links from various qualilty sources.
Content is high quality written articles, how-to's, blog posts etc. by in-house pro writers plus UGC from a semi active forum (20-30 posts a day).
Farmer: After Farmer, this site's vertical is pretty much same as before with the biggest exception being my site. I quickly discounted low-quality content (spider-food) and focused instead on technical reasons. I took this approach since this site isn't the most well kept site I have and I figured the crappy CMS + PHPBB might have caused isseus.
I didn't want to waste my time crawling the site myself so I quickly downloaded all the URLs that Majestic had crawled. Too my surprise the result of Majestic's crawler was over 3 million URLs when the real number would likley be 30-40k and Google has about 20k indexed.
After scanning through the file with URLs I knew I had issues. Massive amounts of auto-generated dupe pages from the forum and so on. By adding around 20 new lines to robots.txt I was able to block millions of pages from being crawled again.
My logic: Ok, so now I think I've found what caused the drop. Milllions of dupe pages and empty pages could have tripped the Farmer algo update to think the site is low quality or dupe or just trying to feed the spiders with uselessness.
My WEAK point in this logic is that I can't prove that Google even knew about (or smart enough to ignore them). Google WMT tells me they've crawled an average of around 10k pages the last 90 days. Given this I'm doubting my logic and if I've found the issue or not.
My next step is to see if this gets resolved algorithmically or not, if not i feel I have a legitimate case to submit a reinclusion request but i'm not sure?
Since I haven't been a contributing member to this community I'm not looking to get direct help with my site, but hopefully this could spark some discussion about Farmer and maybe some flaming of my logic regarding the update
So, would any of you have drawn similar conclusions as I did? (Sweet blog bro!)
-
Good to hear that all your rankings have recovered. How have things gone the past couple of weeks, do you have anything you can share here? Or maybe even for a YOUmoz post?
-
The site is back, all rankings recovered.
-
Hi Barry,
Thanks for bringing up some great points!
I probably should have talked about the effects the update had on the site in my op
Well, it was a drastic drop in rankings for pretty much everything the site ranked for, at least 20-40 positions drop acroos the board EXCEPT for the most commercial/highest volume KW's that only resulted in a 3-6 position drop (still top 10 for the most part.The landing pages, that dropped the least, for the commercial queries is obv. where all the paid links go to (mostly from sites in my niche that are still doing fine). The informational pages that got hit the worst only ranks because of natural links from great/good sites that were not affected by farmer and domain authority.
I really see your and Dan's point about Google not caring about the millions of urls but i can't shake the feeling that it might have tripped the wire somehow
FTR, I'm not complaining about my situation, just generally surprised about this site getting hit when I have a ton of sites that deserved to be torched when I feel this site is actually "clean". I think this is why my logic seems strange to you
Anyway, another great reply, thanks!
-
What's actually changed for you though? Have you looked in the analytics to see what pages are no longer bringing traffic, what keywords are no longer bringing you traffic, that kind of thing?
Is it across all pages and all keywords or is it just a few high traffic keywords and pages?
Just because your niche has many link buyers doesn't mean that you're not getting penalised for it
I don't think Google will have known or cared about those millions of pages, I assume none of them have shown up as a landing page for visitors so I would guess they were effectively invisible to Google.
Paid links (and indeed normal links), you may not be getting penalised for them, but if some of your highest value links are themselves being punished or in some way devalued you may be losing out there as well.
I asume no other significant change occured at this time?
-
Hey Dan,
Thanks for replying! I figured the purchased links would come up but I have pretty much discounted that since the niche is crowded with link buyers and no-one got hit. I'm also active in other verticals where lik buyers prosper and I haven't seen any impact on just about any of them.
In comparison, my sites link profile is pretty vanilla compared to many other sites. That said, I know I can't discount the links 100% as being the reason here since I've been paying for them.
Really appreciate you taking the time to reply!
-
My first knee jerk response to this post is to target the part about purchased links. If you have paid links, then I would remedy that problem before I look at anything else.
Do you have a lot of adsense on this site? I've been hearing left and right that a lot of sites that got hit hard were those with 5 or 6 adsense units on one page. Excessive ads drive users crazy, so Google could be torching you for that.
As for the auto generate dupe content pages, Google may or may not have found them. Were there links to these pages? Do you know how Majestic found them?
If you never linked to these pages it is unlikely Google ever found them. Google tends to only crawl content with links or found in a sitemap. If you never had links to those dupe pages and they weren't in your sitemap, I doubt it is causing the problem. Plus if that were the issue, you probably would have been torched long before this algo update.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Backlink Question
I have a backlink on a very popular sports news site PR5 this is a sitewide link. The link is near the top on righthand side. As you scroll down, the page keeps loading more and more information. According to my Google webmaster tools the link is on the site over 21,400 pages. As the stories being submitted are new most of the pages have no PR however there are around 30-40 categories that have PR ranging from 0-4. According to my AHREFS account on a daily basis it picks up between 100-200 new links with on average around 10-20 being lost as the stories are being removed. Would anyone advise I just ask for my link to be on the category pages only or should I leave it as it is? Many Thanks in Advance for any Feedback.
White Hat / Black Hat SEO | | Palmbourne0 -
Pleasing the Google Gods & Not DeIndexing my site.
Hey Mozzers, So plenty of you who follow these threads have come across my posts and have read bits and pieces of the strange dark dark gray hat webspace that I have found myself in. So I'm currently doing some research and I wanted all of your opinion too. Will Google always notify you before they stop indexing your website? Will Google always allow you back if you do get pulled? Does Google give a grace period where they say "fix in 30 days?"? What is every bodies experience with all of this?
White Hat / Black Hat SEO | | HashtagHustler0 -
Re-Post: Unanswered - Loss of rankings due to hack. No manual penalty. Please advise.
Sorry for reposting, but i must have accidentally marked this as answered. I am still seeking advice/solutions. I have a client who's site was hacked. The hack added a fake directory to the site, and generated thousands of links to a page that no longer exists. We fixed the hack and the site is fully protected. We disavowed all the malicious/fake links, but the rankings fell off a cliff (they lost top 50 Google rankings for most of their targeted terms). There is no manual penalty set, but it has been 6 weeks and their rankings have not returned. In webmaster tools, their priority #1 "Not found" page is the fake page that no longer exists. Is there anything else we can do? We are out of answers and the rankings haven't even come back at all. Any advise would be helpful. Thanks!
White Hat / Black Hat SEO | | digitalimpulse0 -
The purpose of these Algo updates: To more harshly push eCommerce sites toward PPC and enable normal blogs/forums toward reclaiming organic search positions?
Hi everyone, This is my first post here, and absolutely loving the site and the services. Just a quick background, I have dabbled in SEO in the past, and have been reading up over the last few months and am amazed at the speed at which things are changing. I currently have a few clients that I am doing some SEO work for 2 of them, and have had an ecommerce site enquire about SEO services. They are a medium sized oak furniture ecommerce site. From all the major changes..the devaluing of spam links, link networks, penalization of overuse of exact match anchor text and the overall encouraging of earned links (often via content marketing) over built links, adding to this the (not provided) section in Google Analytics, and the increasing screen real estate that PPC is getting over organic search...all points to me thinking on major thing..... That the search engine is trying to push eCommerce sites and sites that sell stuff harder toward using PPC and paid advertising and allowing the blogs/forums and informational sites to more easily reclaim the organic part of the search results again. The above is elaborated on a bit more below.. POINT 1 Firstly as built links (article submission, press releases, info graphic submission, web 2.0 link building ect) rapidly lose their effectiveness, and as Google starts to place more emphasis on sites earning links instead - by producing amazing interesting and unique content that people want to link to. The fact remains that surely Google is aware that it is much harder for eCommerce sites to produce a constant stream of interesting link worthy content around their niche (especially if its a niche that not an awful lot could be written about). Although earning links is not impossible for eCommerce sites, for a lot of them it is more difficult because creating link worthy content is not what eCommerce sites were originally intended for. Whereas standard blogs and forums were built for that exact purpose. Therefore the search engines must know that it is a lot easier for normal blogs/forums to "earn" links through content, therefore leading to them reclaiming more of the organic search ranking for transaction and non transaction terms, and therefore forcing the eCommerce sites to adopt PPC more heavily. POINT 2 If we add to the mix the fact that for the terms most relevant to eCommerce sites, the search engine results page has a larger allocation of PPC ads than organic results (above the fold), and that Google has limited the amount of data that sites can see in terms of which keywords people are using to arrive on their sites, which effects eCommerce sites more - as it makes it harder for them to see which keywords are resulting in sales. Then this provides further evidence that Google is trying to back eCommerce sites into a corner by making it more difficult for them to make sense of and track sales from organic results in comparison to with PPC, where data is still plentiful. Conclusion Are the above just over exaggerations? Can most eCommerce sites still keep achieving a good percentage of sales from organic search despite the above? if so, what do the more niche eCommerce sites do to "earn" links when content topics are thin and unique outreach destinations can be exhausted quickly. Do they accept the fact that the are in the business of selling things, so should be paying for their traffic as opposed to normal blogs/forums which are not. Or is there still a place for them to get even more creative with content and acquire earned links..? And finally, is the concentration on earned links more overplayed than it actually is? Id really appreciate your thoughts on this..
White Hat / Black Hat SEO | | sanj50500 -
Google Disavow and Penalty lifted please help?
We disavowed 80% of our backlink profile due to our last SEO building cheap nasty links and filed a reconsideration requested (we had the Google Webmaster Tools notice of detected unnatural links to http://www.xxx.co.uk penalty for a year from the 24<sup>th</sup> march 2012 but thought it best to clean up before round 2 – even though we had no real penalty and we dd some decent link building that moved us up). We then received a successful penalty lifted note (on the 22<sup>nd</sup> of May 2013) but our rankings dropped (due to the crap links propping us up) since then we have built a fair few high quality links but our rankings do not seem to be moving much if at all (7 weeks clear now). has anyone had any experience with the above (are we in a sandbox type situation). Thank you for your time Thanks Bob
White Hat / Black Hat SEO | | BobAnderson0 -
Keyword Density Question
Here's my hypothetical. I'm working on a car dealer site. And it's a Chrysler Jeep Dodge Ram dealer. Would "Chrysler Jeep Dodge Ram dealer," count as four keywords rather than one? My goal is to make the website show up for either Chrysler Dealer, Jeep Dealer, et cetera. Thanks!
White Hat / Black Hat SEO | | OOMDODigital0 -
Unnatural Link Notification - Third Go Round, specific questions
Hi all, I'm posting what is sure to be a common question, but I can't seem to find much information by searching Q&A over the last month so thought I'd throw this out there. There's a lot of 'what do I do??' questions about 'unnatural link notification', but most of them are from first timers. We're pretty far along in the process and it feels like we're going nowhere, so I was hoping to pick the brains of anyone else who's 'been there'. We have a client that we inherited with an unnatural link profile; they were warned shortly after we took them on (around March was the first warning). We compiled an apologetic letter, specifically identified a previous agency who >was< doing bad things, mentioned things would be different from now on, and provided a list of links we were working on to remove based on WMT and OSE and some other sources. This was submitted in early June. Traffic on the main keyword plummeted; ranking went from top 5 to about mid-page 4. We got hit with that same rash of Unnatural Link warnings on July 23 that everyone else did and after looking around I decided not to respond to those. We got a response to the reinclusion request submitted in June above, saying the site was still violating guidelines. This time I went all out, and provided a Google docs spreadsheet of the over 1,500 links we had removed, listed the other links that had no contact info (not even in WHOIS), listed the links we had emailed/contact formed but got no response, everything. So they responded to that recently, simply saying 'site still violates guidelines' with no other details, and I'm not sure what else I can do. The campaign above was quite an investment of resources and time, but I'm not sure how to most efficiently continue. I promised specific questions, so here they are: Are the link removal services (rmoov, removeem, linkdelete, et al) worth investigating? To remove the 1,500 links I mentioned above I had a full time (low paid) person working for a week. Does Google even reconsider after long engagements like this? Most of what I've read has said that inclusion gets cleared up on the first/second request, and we're at bat for the third now. Due to the lack of feedback I don't know if their opinion is "nope, you just missed some" or "you are so blackhat you shouldn't even bother asking anymore". One of the main link holders is this shady guy who runs literally thousands of directories the client appears in thanks to previous SEO agency, and wants $5 per link he removes. Should I mention this to Google, do they even care? Or is it solely our responsibility? Thanks in advance for any advice;
White Hat / Black Hat SEO | | icecarats0 -
Is this proposal white hat or likely to harm me in the long run?
Hi, I'm considering outsourcing some SEO to a company I got a first month trial sweetener deal with. I've not done this before and am a little unsure about what they propose doing, not sure if I'm being a bit paranoid or too controlly. Details of what they propose: Send them 10 keywords we're interested in ranking for. Work they will perform:
White Hat / Black Hat SEO | | shabbychicoriginals
-Submit site to all major search engines
-Submit 20 social book marks for site
We'll produce 1 article + 19 spun variations of the article submitted to:
-30 directory sites
-10 press release sites and distribution networks Business Submitted to 5 business directories
5 social networks created Work and ranking report highlighting what has been done at the end of the month. Most of the stuff I've done already or can do myself. The elements that make me a bit suspicious are the: - 1 article plus 19 spun variations? 5 social networks created? What does that even mean? I did get this for about £20 for the 1st month with no commitment afterwards so I am tempted to let them try. But should I be a bit wary it might do more harm than good in the long run? Any advice\opinions would be much appreciated.0