I would add that it makes no difference as long as it's 180°, if it's 90° or 270° rotation, then the stem will not fit as perfectly on the keycap since the cross on the stem is not symmetric on all directions.
Wow, I never realised. I always put the narrower width of the N and S cross arms down to illumination, but you are correct: there's a difference of around 0.2 mm. It's hard to get a precise measurement with a standard digital calliper, but having just checked a handful of switches (one each of red, green, brown and lock) the N and S arms are around 1.05–1.1 mm thick, and the E and W arms are around 1.25–1.3 mm thick. I cannot get a Filco keycap to go on sideways at all.
When I enquired about the stem spec, I was only able to obtain keycap specs, and they're quite complex — I've just looked at them again and I can't figure out where it says that they're only 180° symmetrical.
Holding a brown switch next to a green one, it's quite evident with the brown, but not the green, due to colour and angle of lighting. Both measure the same though. Ironically, Cherry's own diagrams are not only incorrect, but not even remotely self-consistent within the same document.
Interestingly, there are keyboards with keys where switches are mounted sideways, mostly stabilised keys. I guess they had special keycaps with sideways mounts.
[
Edit: Now I have to redo all my diagrams!]
