For anyone that is having issues of whole columns firing, I believe that I just figured out what is happening. I don't know if y'all are still checking that have problems, but it's an interesting problem and I put the project away for a while until I finished another one.
So, if you look on the back of the PCB where the through hole pads for the switches are, there is a tiny, almost imperceptible piece of non solder mask there. If you look close, it almost looks like it's part of the pad. To diagram it in text looks like this: (0 and much closer and more difficult to see. If that is bridged on any of the switches, it appears to cause the whole column to trigger. (I believe that it shorts the entire column to ground which is what triggers the whole row.)
I had asked about it and messaged yuktsi about it where it was mentioned that it was solder overflow and to remove the switches and use desoldering braid to take up the solder and to clean with alcohol. I unfortunately got hung up on the removing switches part as I had never soldered switches to this board. (In addition to the mcu not being completely soldered, there were a few bad RGB LEDs that needed replacement and this tiny gap was bridged by solder at some point during the LED replacement process.)
So if you have issues like 4rfv all triggering at once, This is very likely your answer. Desolder, clean with braid (it took a long time for me to soak up the right solder even with flux, but I'm also bad with/hate solder wick) clean with alcohol and keep testing.