It really comes down to the amount of data available to the sensors and the data already applied to their algorithm. As long as people are driving there will be room for error, but I think safety nets can be put into place. For example, most people have cell phones so there could be a something that pings your cars location to nearby automated cars if they are out of sight of the sensors.
But the idea of mobilization for the elderly, disabled, and blind. or automated delivery for food and local stores. Like so many small shops are out of business by huge online retailers, but what if a small mom and pop shop had a self driving vehicle that could just bring you their product on demand in minutes? That would be amazing! Amazon will likely use that idea first though.