Remembering back to when I was learning qwerty I would constantly glance down at my keys on my keyboard to give my brain a visual cue where the keys were before I had them totally ingrained in my mind.
Glancing down at the keyboard does not help with Dvorak, since the letter your are looking for is very likely right under a finger of yours. Moreover, you will keep glancing, and that will slow down your memorization process.
As it has been suggested already, do not swap keycaps on your keyboard, but print the layout and hang the print somewhere. I know it's difficult, but it can be done, and it will be rewarding in the long term.
One of the things I'm having difficulty remembering is that it's a o e u i layout for the vowels, this slows me down a bunch having to remember which key is under which finger constantly.
Yes, vowels are an issue while you are learning. I remember having issues with V and W too.
By all means, practice with a tutor like ABCD.
Keep up the good work!