Yeah I think that's the route I'll go. Low wpm while I adjust to it isn't a big deal since my job has nothing to do with computers anyway. So it's not like it'll hold me back. And even if I go back to qwerty it would be nice to just be able to know another layout.
If it's frustrating, you might want to try switching slowly instead of doing a cold turkey switch (something like tarmak for example). It's what I did, and it takes longer, but you still maintain a decent speed. It took me about two weeks to make all the letter changes to colemak and get pretty comfortable with it, but it took me a couple of months to get fully back up to my previous speed. It really varies depending on the person though.
On the other hand, I really wouldn't recommend switching just for speed. Maybe you'll pick up some speed from having to relearn to type (possibly with better habits), but you're not going to see some massive increase in speed just from switching to an alternate layout. I think practice is going to make the difference regardless of whether or not you switch.
Depending on what layout you choose, it will effect your qwerty speed if you stop using qwerty. I followed the suggestion that you type qwerty while looking down and the other layout while looking at the screen normally. I hardly ever use qwerty, but if I have to I still can type at a decent enough rate by looking downwards. I'ts really strange though because I can't type colemak on my phone. Qwerty works better with swiping anyway though. Distinguishing them based on matrix vs staggered might be more effective like suggested.
Ideally, you'll never have to use qwerty. If you have to use a computer that isn't yours that you can't use something like PKL with, you shouldn't have a problem if you get an ergodox.
If you want to try a new layout, I suggest T9-QWERTY. If you just want to improve your speed on your current layout, I suggest shorthand.
Shaaniqbal's idea to use text expansion for shorthand for common words will probably increase your speed more quickly and easily than switching to another layout would.
Does anybody know much about switching to Workman? I like the ideas behind the layout, but it seems like it doesn't have as much of a following as Colemak.
I think Linkbane used to use Workman. I looked into it, but I didn't find it particularly appealing. The numbers for how much effort specific keys take are pretty subjective, but if you agree with them, you might like it.