If you're feeling brave, you could give the nouveau drivers a shot. What's the worst that could happen....? :-)
http://nouveau.freedesktop.org/wiki/I'm going to get around to installing them one of these days.
(The problem you're having is a driver problem. Not sure why the graphics drivers lock up on mouse events, but that's the beauty of closed source software. You just get to live with it.)
