A switch is a switch - when you press it the pins connect, current flows through them and the controller notices
As the diode for e is the right way round it could be defective - you should only see continuity through it one way, if it goes both ways it's broken.
edit: Realise I should explain myself here.
Either you messed up 13 out of 14 columns and 4 out of 5 rows, looking at your wiring it looks good so I don't believe for one moment you messed up that badly!
The other option is you have one dodgy diode and the firmware doesn't match the diode orientation, so it doesn't work (except the broken bit)
The first thing I would do is change it to this in the config - this reverses the expected orientation of the diodes and may well make it work, but I suspect the e key will cause ghosting issues...
strobe_cols = False
It's a much quicker fix than checking the entire matrix, you have nothing to lose