Come to think of it, commercial GPS recievers (personnel ones) don't tend to have that problem, and the ones i've seen don't have external, onboard compasses.
Personal gps receivers based on sirf, ublox, nemerix, etc. chipsets fudge a heading using time difference of position, or velocity from doppler measurements, but all of them tend to 'fudge' your heading using some sort of navigation state filter. You can see this by plotting the output of a GPS receiver, as it will wander around in a 2-5m circle over time, you're not actually moving but your receiver will have velocity noise around 10cm/second, and therefore it will think you are drunkenly stumbling around the maypole
Some of them that cost a little more include both barometers and magnetometers (
https://buy.garmin.com/shop/shop.do?cID=145&pID=310) so you can actually get an accurate altitude over time and heading. (In satnav your vertical component is wost because you never have satellites below you to help resolve the vertical position error - while you do have them on either side - and as discussed above heading is a state that is unresolved when stationary)
Professional products that need a heading that is accurate while they are stationary often integrate inertial sensors, or multiple antennae (as jersandall mentioned)
http://www.hemispheregps.com/Products/P ... fault.aspx
The farther apart you can get them, the better* (in most situations).
If you have any questions about the theory of GPS let me know, I've been studying it in a navigation lab for the last few years, and am fairly comfortable with it
Bottom line is you shouldn't rely on GPS to give you a heading at very low speeds, or to hold onto an accurate one when you're stationary.
