try to use another stem, maybe it is the problem of the stem.
I tried to no avail, thanks though

I would replace the entire keyswitch. In a few of my keyswitches the metal parts got bent once and didn't actuate properly, I could bend them back so it might be that.
Yeah that's one of the options I was thinking of, I can't really tell if it was the metal part or not from looking at it, I seemed to get it to work better and actuate on any press but I am afraid it might come back