I have a USB micro DS card reader that's like that, it's so tight that it's almost impossible to push into my USB ports, and getting it out is really difficult, especially as it's tiny so it's hard to get a grip on the thing.
A suggestion: some USB devices like mp3 players and wifi dongles come with a very short USB extension lead because they are so wide that they block access to adjacent ports if plugged in directly. I believe they come with these extensions so that they can claim to be USB compliant.
If you find one of these extensions (they are very short, like 4-5") you could plug your keyboard into it and leave it attached. It would only make the lead a few inches longer, and you wouldn't have to worry about damaging your keyboard/USB ports when plugging and unplugging.