You could even just take pre-payment for the keys and then charge shipping at the time you ship.
From my **very** limited experience this seems to be a fairly common way of going about things and seems to keep everyone moderately happy.
If someone pays for the keys and then doesn't come through with the shipping costs then, in my opinion, after a reasonable period (3 months? 6 months?) you get to consider the keys as defaulting back to you and can either keep them or auction them off on GH or whatever seems right.
I don't think you should be shy of rounding things up a bit and getting a little bit of rounding error as your reward for your time being the organiser of the GB. I very much doubt you could ever make what your hours are worth at work and only the most childish of people really think people should work for free (or lose money as you seemed to have before).
If you are really squeamish about keeping the rounding-up errors then donate it to iMav for site upkeep or a charity of your choice or buy your partner or kid or whoever a little present to compensate for the time you spend on this you could have spent with them.
Extra key caps are total luxury items for those with spare cash and I am confident that most of us are very happy to pay a bit over the minimum to those who enable our habits in this regard.
Don't burn yourself out being so fair to others that you are unfair on yourself.
Anyway, peace and love and heaps of kudos for be a Group Buy organiser
lysol