Yes, there is a regular AC adapter available for the gateways. Customers who do not have the Voice service do not receive a battery back up, they just get a normal plug-in brick.
If you call support and ask very nicely, it used to be possible for them to put in an order for you to receive a power cord free of charge...I don't know if that is still the case.
But, as has been said already, the battery backup is completely harmless under normal circumstances.
Now, much like anything, they can and do fail...and like anything that plugs into the wall, it can fail pretty catastrophically. But if you're afraid of the battery backup then you probably shouldn't use any electronics at all, a toaster is far more dangerous to leave plugged in.