Joomla SEO is indeed a tough game mostly because most of changes that you need to make on the website require development skill to an extent.
If you are in to Joomal SEO here are the following things that you should consider as important.
- Choosing the right Server
This is not really important for Joomla but any kind of website if you are not on the right kind of server you might see some big fluctuations in rankings as soon as your server goes down and all. The best idea in this regard is to choosing the right kind of SEO and go from there accordingly.
- WWW to non-WWW redirect
Especially with Joomla but true for other CMS as well you need to choose one preferred version and redirect all the URLs so that there will be no duplicate content issue on the website. Make sure your website is displaying one kind of URL that is either with WWW or without WWW version.
- URL Rewriting
When using Joomla set YES to “USE URL rewriting” in order to create Search Engine friendly URLs as by default the URLs are messy and might not understand by Google which have a direct impact on rankings.
- Include /images/ in Google index
When using Joomla as your CMS, usually the robots.txt excludes images from the search engine index. Make sure you are removing /images/ from the robots/txt file so that your images can include in the Google image index and rank accordingly.
There are many other factors. Mark writes a detailed post about that on Moz.com that you can find here. http://moz.com/blog/seo-your-joomla-website-in-11-super-easy-steps
I would recommend you to use Marks post as a checklist and make sure that you are on the right track when it comes to search engine optimization.
Hope this helps!