Consider a knee chair (without back support). You can only use it with an upright posture. It feels very natural, but is a little more tricky to get in and out of.
I still need to keep my right knee straight in front of the computer, or it will start hurting after a while, so I can't use one.