The keycaps. I got the version with the black nearly unbranded case (there's a small logo on the back which doesn't bother me). The set from EKB is $30 shipped for me so if I can find a cheap solution to deface the existing ones that would be great.
If the case is already (mostly) unbranded, then all you need to do is replace keycaps. If you want blank keycaps, EK has sets for $20 (as well as the printed ones).
Apart from that, see if you can find a cheaper keyboard on eBay and salvage the keycaps from that.
or, as you have now > 60 posts, you can create a WTB thread in Classifieds to see if anyone has a cheap set they want to get rid of.