You are correct, adding code to a page to 'adjust' the bounce rate can effect your 'average time per visitor' statistic.
This is because of how google measures the time spent on a page...
Normally, if a user opens one page, then does not visit any more pages on your site, it will count as a bounce (even if the user had remained on that page browsing for 10 minutes). This is because there is only one call made to google analytics when the page is opened. There is no call made to google analytics when the page is closed.
So normally, the 'time on page' is calculated by taking the time stamp of when the current page is opened, and comparing it to when the next page on your site is opened. The difference between the two is your 'time on (previous) page'.
So what happens when a user only opens one page on your site and leaves (bounces)? This will be counted as a 0 second visit (even if the user was on the site for 10 minutes). Thus bringing down the average visit time for all visits.
What happens when you add the 'adjusted bounce rate' code to your page, is that a 2nd call is made after x seconds to the google server.... Allowing google to know that the user has in fact remained on the page for an extended period of time. So now a whole bunch of these '0' second (bounced) sessions will be converted to longer sessions based on the time between the 2 time stamps.
The more 'one page only' visits you have to your site, the more this has the potential to skew your average session time.
On a side note, this will also effect the last page visited of multi-page sessions, as normally google would not know how much time was spent on the last page of the site as well.