A bit different from what you've experienced, I also started with QWERTY and went about 80-90, but it was inconsistent, jumping from seventies to around a hundred. Instead of Colemak, however, I learned Dvorak, which apparently is supposed to take longer to transition from QWERTY than Colemak given that there was no effort to correlate them at all, and nearly all of the symbols are in different places.
It took me about a month, practicing from 45-75 minutes pretty much daily, to become equally proficient. By five months, I was pushing upper-110 and after a broken-laptop break of about two months, at eight months (six really) I'm at mid-120's top and mid-110 normally. I mostly went cold turkey straight conversion, made easier by the fact that I'm not of age to hold a job, and I would go back to QWERTY for intensive tasks but after a three weeks or so I couldn't really use either, it took me until I was proficient at Dvorak to have a usable format.
If you learn these sorts of things quickly and you practice regularly (even more importantly than for a long time) you should be able to get back to your old speed in two months or less.
Best of luck, and don't give up!