Some of them have the hole centered, some have the hole offset, so rotating it will move where the hole is in the barrel.
The insert for the shift key should be a centered one. I believe the vertical ones on a numpad are the ones that are offset. I could be wrong.
Looking at your pic, does the hole seem offset a little to the left or is that a trick of the angle? You might want to pull it, rotate it, try it again.