Luckily I haven't encountered a game that totally ignores Colemak (for typing a message), but then again I don't play much in Windows. In Mac OS X it took a while
to get StarCraft 2 to work properly, it only semi-worked before that: the keyboard was Colemak for typing, but hotkeys were QWERTY. Changing to grid layout in SC2 worked to make them Colemak, even though the buttons had the wrong (QWERTY) label.
I'm currently playing Skyrim in Windows, and as with many other Windows games, its hotkeys are QWERTY. I don't care though because I can still for example chat with friends (steam chat) using Colemak.
Also, my iPhone still uses QWERTY, which is OK, because I can't touch type on a touch screen anyway. Hunting and pecking QWERTY is the thing I do naturally when I look at the keyboard (which usually has QWERTY labels). Same thing of other people's keyboards.
As for speed, you will get back to your pre-switch speed, but it may take a while.
It took me almost a year (OTOH after 3 weeks I was at half speed already, which was more than good enough for everyday use).