What's the alternative? Brushing with just water? Brushing with baking soda like the olden days? Using fennel or a hazel twig plucked from a field like medieval times? I think to answer the question, you'd have to establish a control to compare it to.
Whatever potential downsides there are, I think the upsides outweigh them by a lot. An acquaintance never brushes and now has abscesses and will likely have to get dentures, and he's only in his mid-30s. My mom has had poor dental hygiene for a long time, and her teeth are destroyed. Unless there's very convincing evidence that there's a better alternative, I am not going to chance it.
Personally, I have fantasized about oral nano-bots that live in the mouth and automatically swirl around and clean all the nooks and crannies at certain intervals throughout the day. Why hasn't someone invented that yet? Better yet, why not total body cleaning nano-bots so that I never have to take a shower again? I guess they'd have to have some place to put the dirt that comes off so that they don't just smear poop and sweat everywhere.