I have a Filco TKL with WASD blue o-rings and it's amazing how big a difference in sound the o-rings make. The clack of an undampened key bottoming out nearly completely drowns out the click. An MX Brown has no click, so those are quite quiet when dampened.
And don't forget the impact of the desk you use. The cheap, hollow IKEA desk I have at home acts as a resonator, making any sounds from the keyboard much louder. (Even the Apple aluminum wireless sounds loud on it.) At work, where the Filco is, the solid desk doesn't resonate, so it's quiet enough that none of my office mates complained, even though it's clicky.
Try this: put a towel or blanket on your desk with the keyboard on top and see if that makes a difference in noise. (It can also be quite pleasant on the hands.)