Hi Jon,
Situations like this can have very complex causes, so without actually being able to investigate your unique business, my suggestions will need to be broad.
First of all, it's great that you've worked with Darren Shaw's Whitespark tool to begin identifying citations. Please check out Mike Blumenthal's article regarding the time it takes for citations to go into effect:
http://blumenthals.com/blog/2012/09/26/infographic-citations-time-to-live/
And, it's good that you have read through David Mihm's Local Search Ranking Factors, which is the key annual industry report (I'm proud to have been a participant since its inception). The information in the report is exceptional, but it's also a lot to take in. For the sake of brevity, I'm going to excerpt part of a post I published (see: The Rudiments of Local SEO ) recently, summarizing the typical elements of local search rankings:
All Top Rankings Are Not Created Equal
One of the most common questions I receive as a Local SEO is, “How do I get high rankings?” If ‘it depends’ were ever an appropriate answer, this is when. Read this carefully:
The effort and investment you will need to make to achieve high rankings for your local business is completely dependent on the competitiveness of your industry and local geography.
In other words, a personal injury attorney in Dallas, Texas is going to have to make way more effort than a baker in Boonville, California (population: 1,035).
It’s all about the competition. How many competitors you have locally and how much effort they are each making is what dictates the lengths to which you must go to surpass them, if possible.
There is a pretty long list of things your local business can do to establish and promote your visibility on the Internet. What you should do all comes down to how competitive your niche is. David Mihm’s annual Local Search Ranking Factors report is the Local SEO industry’s key survey highlighting the components of high local rankings. Some Local SEOs postulate that there are 200+ factors that go into determining the rankings you receive. Here is a short list of the most basic ones:
- Age and authority of your domain
- Local optimization of your website
- Lack of violations on your local business profiles
- Consistency of data about your business across the web
- Proximity to city centroid
- Number and authority of your citations
- Number and velocity of your reviews
- Social factors
- Number and authority of links
You cannot control factors like the age of your domain or the reviews you receive, but you do have control over other factors like the optimization of your website and the correctness of your local business profiles. Whether you need to make major investments in areas like Social Media and linkbuilding all boils down to how competitive your niche is. Every business is different. Educating yourself or hiring a pro is what will help you discern the amount of effort you need to make to achieve the highest possible visibility.
I'm excerpting this part of my post both in hopes that perhaps you can see something on the list where you realize you can improve what you're doing, and, also, because it speaks to your question as to whether X amount of something is enough.
In other words, 20 citations may be all some businesses need, based on their industry and locale, but they won't be nearly enough for other businesses in different places. What you need to do is based on the competitiveness of your specific situation.
Finally, if rankings for an established business (not a new one) begin to fall off, you must always investigate the possibility of violations, penalties or bugs. Such work typically requires a full audit of the website and all its profiles. There will never be a 'simple' answer when this arises, because there can be several things going wrong and these have to be discovered and, where possible, corrected.
I hope my response is helpful to you and gives you some idea of where to turn your attention. Good luck!