Somewhat OT but I have a love/hate thing with my Oly E-PM1 (similar to your E-PL3) with several lenses, especially the pany 20mm f1.7. The image quality is great, with that lens especially, but the autofocus is just too slow. It's faster with newer m43 lenses but still pretty annoying when shooting anything other than landscapes.
With a cheap adaptor I'm guessing you'll be limited to manual focus. As long as you keep that in mind it could be a lot of fun. Could also be really frustrating.