Alas, I have almost no knowledge in µcontrollers or electronics, and can't really follow what you are talking about.
Could you link me to some more information on this project (maybe the existing code), and/or some documentation?
Hmm, there isn't exactly much existing code yet!
By buffer I just mean a 2D array for storing the brightness for each LED, and double-buffered so that you can draw into one while the interrupt is reading from the other.
What I was thinking though, is that the drawing buffer could have more pixels in it than we have LEDs - perhaps a full 7x21 grid. That would make certain effects easier to implement, and more, well, effective :-)
Also, how about having multiple LEDs under the spacebar, to make it more of a grid there? No reason we can't have some LEDs that don't have switches.
These high-level things need thinking of as well as the low-level stuff!
So it will either process at the end of scanning and slightly delay/shorten the PWM cycle, or process during the PWM cycle if that's possible. It would be ideal to process it during the PWM cycle.
Can it process the PWM cycle and rotary encoder code at the same time?
Couldn't you process the encoders' signals on interrupts? You could have one for each signal wire. Your interrupt might be delayed a little if the scanning interrupt fires just before you get an input, but it shouldn't be by too much, and wouldn't happen often. The reverse, where your interrupt fires just before the scanning interrupt, could disrupt the LED brightness, but if you code your interrupt to be quick that should be ok too - I reckon you could code it very cleanly that way, since in each interrupt handler you already know which signal has changed.