Even with debounce, there can be an improvement in overclocking. You see, debounce impacts how long after the key is pressed that the keyboard can register when the keyboard can consider that key unpressed. Debounce does not impact how long after the key is pressed that the keyboard can report to the USB controller that it has been pressed. To summarize, a well programmed debounce filter should never add latency to the initial key press, but will add latency to a quick key release after a keypress.
That being said, you have to be really really good and have a great network connection for the 3.5 ms on average that you gain going from 125 Hz to 1000 Hz to matter.