There are possible a few reasons for this:
1. The user is filling in the contact form, but then decides not to send it
2. The user is filling in the contact form, but then can't send it because of an error with validation or something server side
3. The user is filling in the contact form, and is sending it (you don't specifically say that they don't take any action on that page)
4. a percentage of the users will almost certainly be looking at the contact page for social proof - is there an email address I can copy, is there a physical address, do they have a phone number, is there an easy complaints procedure etc. - these people will then often navigate on (back to the homepage or another site page)
5. People might want to see where the company is based and then decide that the company is not relevenat to them because they are from a different, country/state/country/town/city/etc.
Plus, I'm sure there are a million other reasons.
I recommend installing ClickTale on the site. The free version of the product should be sufficient to gather some video recordings of what the userts are actually doing on the page. This is the best way to see what the user is doing (but not a great way to know what they;re thinking! - although you can make some educated guesses). Check it out at http://www.clicktale.com