Hello,
You asked a very good question. I have run up across this too, many times before. I know that I am going to get blasted by some but I tend to take the opposite approach as Matt and Miriam these days (no offense to those two at all). Let me explain....
In the past, I would do exactly what you are thinking about. I would optimize one domain for the largest city the company was doing business in and create sub pages for the other smaller cities. With some good, non duplicated content, it would always work out well.
I did this so many times I cannot remember and it always produced great results. A website got traffic from all cities within one giant metro area.
That technique does not work out well for me any longer and I rarely use it. There are several reasons. Google has gotten better at recognizing which suburbs tend to go along with specific large cities. For instance, if you live in Portland, then Google will recognize that Beaverton, Hillsboro, Gresham and Tigard are all part of the same metropolitan area. I have noticed that they let the domain names with the highest trust and authority automatically rank for the smaller cities in the area.
At the very least, I have found that if the domain is strong enough, then all I have to do is mention those cities on the home page or some other strong page within the website and they will rank well for everything.
Currently, the results have been dismal for me when I have attempted to make a whole bunch of subpages,optimized for several cities.
In fact, I am going through this exact problem with two local clients. I think Google is trying to get rid of clutter. They are referred to by Google as doorway pages and Google is trying to get rid of them.
Now, according to them a page is only a doorway page IF it has "poor-quality content," but that is subjective. We all view our content as "high quality" but to Google those subpages does not give their users any value.
I am convinced that they frown on websites with several pages solely optimized to rank for all the smaller cities within a big metropolitan area. My suspicions were verified by Will Renolds in a Summit East 2013 video I just watched three days ago.
Some websites, who have a high enough pagerank and are "grandfathered in," can still get away with it. If you have been involved with SEO long enough you learn that just because another website is successful at doing something does not mean you will be as well. I would view myself as a gun slinger but lately I grown weary of playing the game of Russian Roulette with Google.
I have several websites that I firmly believe have lost rankings (they don't rank anymore for all the smaller cities and lost ranking for the bigger city) because of that. That might not have been the only reason but it appears that way to me. I am convinced that your overall relevance gets diluted when you do this.
Now, having said that, my official response depends on one thing: what is the main city you rank for now and what other city are you trying to rank for?
If the city is not that big, and your domain is strong, then I would just add the keyword of that city to the website you already have.
However, ff you are in a city like Dallas and you want to rank for Fort Worth, then I would start a whole new website for that city. Its a pain in the butt but I think that is the safe way to go.
These days I like optimizing a local client for just one major city. I do not like the idea of spreading a website too thin and end up, thereby making it seem less not entirely relevant for anything. The result is me accomplishing nothing, I just have a site that ranks on page 3 for everything.
If you start another website, you can give it a link from your established website to give it a boost. Just make sure it is not a sitewide link (ie. sidebar, footer or header).
Don't break what is working. You dodged a bullet with that duplicate content issue so be thankful you didn't drop like like an anvil being thrown from a 20+ story building. I would not try to test my luck by watering down your current website in an effort to rank for another city. The chances of it being a success are not as strong as they once were.