So my workaround is to just avoid using modifiers on any layer other than the default, but I would like it if I could actually fix this problem. Is this possible with the QMK firmware?
It's definitely fixable, but doing so is tricky, and needs a change to how layers work. A slight change, but still one. The way around this problem is to remember which layer a key was pressed on, and when releasing, release as THAT key. This solves all kinds of problems, like layer keys having to be transparent on the target layer. In Kaleidoscope, we implemented this by having an in-memory cache of the currently active keymap. Whenever a key is pressed, we look it up from the layers, considering the currently active ones, and push it onto this in-memory keymap. Then, whenever we need the key, we look it up from there. So a release will use the same keycode as the press did, no matter how many layers you changed in between.
This is pretty simple, and only uses a single layer worth of RAM, but makes a lot of things much more predictable and intuitive.
As far as I remember, you can't *easily* do it with QMK, but you can always open a feature request there, and perhaps they'll like the idea enough to implement it.
(Or you can port Kaleidoscope to the IRIS, and not have this problem. But porting is not for the faint of heart, though should be reasonably simple as Iris uses an Atmega32U4...)