It would not be a good idea to interlink them for SEO reasons as you mention "creating a link wheel/pyramid". I would ask these questions to myself: 1. Do all the sites have unique content ? 2. Do they have any duplicate/repurposed content at all between theme ? 3. Do uses on one website need to navigate to the other website, since it provides value to the user ? 4. Have you considered nofollow links ? This is a bigger strategy discussion that you have to do to figure out your next steps. If they don't provide too much of a value, how about you aggrgate the content on 1, maybe 2 or 3 combined sites instead of 50 and 301 the rest ? It's really difficult to answer further without knowing more in detail. I hope this atleast invokes some questions and a long term SEO strategy.

I understand your concern. I am basing my answer based on the fact that if you don't have a robots.txt at all, Google will still crawl you, which means its an allow by default. So all that matters in my opinion is the disallow, but because you need an allow from the wildcard disallow, you could allow that and disallow next.

Honestly, I don't think it matters. If you think the way a bot would work, it's not like robots.txt 1 line is read, then the bot goes crawling and then comes back reads the next line and so on. Does that make sense ? It reads all the lines in the robots.txt and then follows the directives. But to be sure, you can do either of the scenarios and see for yourself. I am sure the results would be same either way.

I implemented the canonical tag last year in a similar situation and yes, it's on the original page as well. No issues. Look at eBay.com, they also have canonical tags on all pages including the original page.

I have not seen/encountered any issues. Now when I go check the cache for http://www.example.com/duplicate-url-3.html which has a canonical tag referring to the original-url, Google shows me the canonical URL. The duplicate pages literally disappear from the SERPS, because essentially its like a 301 redirect.

Sure and FYI here's my question about the same topic I asked day before.

http://www.seomoz.org/q/seo-conferences-which-one-s-do-you-attend

I would ask you the following questions:

1. Are all the client sites on the same server ?
2. Do you have a clients page where you link back to each of your clients ?
3. How does your link profile look ? Do you have lots of other links ?
4. How many client sites are we talking ? 10/20/50/500/1000 ? And then how would you compare them to the total links you have ?

If your link profile is mostly natural, I wouldn't worry about it that much. But going forward you could nofollow them. Also, what's the anchor text in your links ? Your Company Name or Website Designed ? Do more of Brand Name, combined with Anchor Text. So all sorts of variations. Do not make them consistent.

I hope that helps.

If it's a sitemap for the users and the page appears under 2 different categories, it's okay. Think usability here.

However, if/when it's the XML sitemap, it should only be there once.

As for the intermediary pages, If I were you, if they offer a good purpose (the reason you have those pages in the 1st place), they should then be listed in both the XML and HTML sitemaps if they help the users.

So to answer your question more directly, if your question is:

Is it bad to have same page listed twice in XML sitemap? Yes, it will generate an error/warning in webmaster console.

Is it bad to have same page listed twice in HTML sitemap? No, if that's what is needed for users, it's okay.

I hope that helps.

Is it possible for the CMS to include the canonical tags ? I would suggest you look into it and have www.mydomain.com/my-custom-url.html contain the canonical tag to www.mydomain.com which should also take care of this issue for the homepage. That way Google will deindex www.mydomain.com/my-custom-url.html and only index www.mydomain.com