Howdy,

We have heard about building microsites on sub-domains example city.domainname.com/keyword-or-service

I think this was more relevant when you were wanting to get an EMD, but I think that this is now less important.

We have heard about building sections off the root domain such as domainname.com/city/keyword-or-service

We have used this and had success with it.  If you look at Yelp, this is what they do as well.  I think you can also consolidate your SEO reputation more effectively. I think the advantage to the subdomain or separate domain approach is that if you get penalized on one, it should not impact the other.  If you keep to white hat approaches this should not be an issue.

We have heard about just pushing more content to the blog which can target a wider range of keywords and cities

If you setup the two options above, you can have a wide diversity of keywords and cities.  Don't see how a blog makes it any different in being able to target more keywords. You can hang a blog off of a folder ie.  domainname.com/city/blog and you can get the same effect.

We have heard about building a new site for each city location replicating the main website and swapping out content for 100% originality

The issue with this becomes scale what if you need a large number of cities covered, can you get all the domain names.  What if you have a user that wants to look in more than one city?  Just does not scale, especially if you want to build your main brand.   Also, when you consider all the different GA and GWT etc etc accounts you would need to setup to manage, it adds a lot of overhead.

We have heard about just building more and more backlinks with different anchor text using the cities in the anchor text

This is something you would have to do regardless of all of the above techniques.  You want backlinks to any page/site/etc with the city in the anchor text but with also a diversity of anchor text.

##########

Bottom line, you need to consider your brand on all of this.  People go to yelp as they are a trusted source and then they rank well with local cities around the US.  Same thing with places like Angies list.  Citysearch uses the subdomain, but that is more of a legacy thing, seems like most newer sites use the a single TLD and then build out locations in different folders.   Brand is what will make you different than all the other companies that sell the same products so IMHO the use of the single domain lets you build on that.

Thank you - please mark my response as Good Answer if it helps.

Cheers!

If you read the article I mentioned above, it really does not hide anything from Google, but you are clearly saying that these links are of no value.

I would use the 301 redirect.

In your case, yes that would save a step to just put /ads/ into robots.

I have not used this tool in this way, but have used it for other crawler projects related to content clean up and it is rock solid. They have been very responsive to me on questions related to use of the software.  http://urlprofiler.com/

Duplicate content search is the project next on my list, here is how they do it.

http://urlprofiler.com/blog/duplicate-content-checker/

You let URL profiler crawl the section of your site that is most likely to be copied (say your blog) and you tell URL profiler what section of your HTML to compare against (i.e. the content section vs the header or footer).    URL profiler then uses proxies (you have to buy the proxies) to perform Google searches on sentences from your content.  It crawls those results to see if there is a site in the Google SERPs that has sentences from your content word for word (or pretty close).

I have played with Copyscape, but my markets are too niche for it to work for me.  The logic here from URL profilers is that you are searching the database that most matters, Google.

Good luck!

I think I understand what you are trying to do, but I may be off with the semantics. The xml sitemap (at it's most basic level) is just a centralized list of links for pages that are on your site to help the search engines find pages without having to crawl all of the pages on your site. They do not contain any anchor text in them.

If you are wanting to crawl your site and find out what all your anchor text is internally, I would use a tool like Screaming Frog.

If you are trying to extract information from your sitemap.xml then you would need to use a combination of a text editor and excel.  Say your URL structure includes product and brand information and you have all the URLs in a sitemap.  You can pull that document into a text editor and then use the "Find and replace" function to get rid of all the extraneous code and then you can be left with just the URLs on one line.  You can then use the excel function "Text to columns" to then split out the URL into sections based on slashes or dashes.  If you have Screaming Frog, it will extract all the URLs for you and then you can split out the info in excel as well.  Use Google to find examples of how to run the tools mentioned above in detail.

If I am totally off, please give a more detailed explanation of what you are trying to do and that will help as well.

Invisible to who?  If they are invisible to the search engines (e.g. blocked via robots.txt) they could not be harmful, but if you gain any link or other quality equity through those pages, your will not get any additional domain authority or organic traffic to them.

That said, if the hidden categories were resorts of the main categories on your site then you would want to hide them from the search engine as they would be seen as low quality duplicate content.  Otherwise if they were visible they could negatively impact your site.

We have actually been doing this on one of our sites where we have several thousand articles going all the way back to the late 90s.  Here is what we do / our process (I am not including how to select articles here, just what to do once they are selected).

1. Really take the time to update the article. Ask the questions, "How can we improve it? Can we give better information? Better graphics?  Better references? Can we improve conversion?"  2) Republish with a new date on the page.  Sometimes add an editor's note on how this is an updated version of the older article.  3) Keep the same URL to preserve link equity etc or 301 to new url if needed 4) mix these in with new articles as a part of our publication schedule.

We have done this for years and have not run into issues. I do not think Google sees this as spammy as long as you are really taking the time to improve your articles. John M. and Gary I. have stated unequivocally that Google likes it when you improve your content. We have done the above, it has not been dangerous at all. Our content is better overall. In some cases where we really focused on conversion, we not only got more traffic, but converted better. Doing this will only benefit your visitors, which usually translates into Google liking the result.

I would ask, why take a few months where you only recycle content, to just mixing it up all year long?  If you were going to designate 3 months of the year to just update content, then why not take the 3rd week of the month each month or every Wednesday and do the same thing instead.  You accomplish the same thing, but spread it out. Make it a feature! Flashback Friday etc.

Bonus idea - make sure you get the schema right

We have something new with our process. Previously, we only marked up the publication date in schema. So when we republished, we would change the publication date in the schema as well to the new pub date.  Now that Google requires a pub date and last modified date in schema we have changed our process. When we republish content, we will leave the original publication date as the publication date marked up in schema and then put the new date that the article is being published marked up as last modified in schema. This is a much more clearer and accurate representation to Google as what you are doing with the article.

We are also displaying the last modified date to the user as the primary date, with the publication date made secondary. The intent here is that we want to show that this is an article that has been recently updated to the user so they know the information is current.

To get this to work properly, we had to rework how our CMS interacts with content on both published date and last modified date, but in the end, I think we are giving better signals to Google and users on the statuses of our articles.

Check with the IT folks or hosting service for your client.  I think this is an outside chance, but if you have been running spiders from your home computer to check the site, you may have been hitting it too hard and slowed the site down and the server may be blocking your IP as you are seen as a spammer.   That is why you change ISPs you are golden as you are seen as a different "user".

I took down one of our sites once with a spidering tool.  They were pushing new code right when I hit the site.  Also, the number of requests a second I thought were ok, well, it was during peak traffic time. (DOH!)

I adjusted my crawl rate down and everything was ok.   Again, this is just a guess, but worth checking considering your symptoms.

Good luck!