the issue of the pinky isn't training muscles and reflexes. even if you train them equally as the other fingers, the pinkies will still be the slowest and most cumbersome.
the real vulnerability with the pinkies are the weak joints that are easily bent, causing pain. notice the pinkies are curved inward relative to the other fingers. this means while the big fingers hit the keys straight on, the pinkies hit the keys at an angle that can easily disjoin the pinky joints, causing great pain. this is exacerbated by the common keys, including Enter and Backpsace, on the standard keyboard, when hit with force and repetition by the right pinky.
i know because it personally happened to me. my pinky has mostly recovered after i quit my IT job and use only Kinesis keyboard at home. Kinesis naturally, smartly puts the Enter and Backspace on the thumb cluster. I also moved the Shift keys to the thumb cluster. Holding down a key with pinky is just really poor ergonomics.
You're right that Colemak is almost as bad as Dvorak when it comes to pinky usage, and thus impacts their comfort. Qwerty actually has light load for the right pinky, which may be slight boon for typing speed.
see clear AdnW-streaks in yellowfour's Beakl layout. Now wonder, given the use of the AdnW optimizer. Interestingly, the AdnW-optimizer ranks AdnW 'proper' better than Dvorak but slightly behind Colemak, for English text. And this is exactly what stevep finds for Beakl, using his own optimizer
Things like distance and effort scores are mostly arbitrary per the designer/coder of the optimizer and analyzer, so I don't really put much stock in those scores. There are other factors that are less arbitrary (and perhaps even more directly affects speed and comfort) which AdnW measures. In particular how AdnW gives insight to bigrams and trigrams. Such as ratio of inward/outward rolls, seesaws, hand alternations.
underestimating how comfortable the Qwerty C and M keys are, they are of almost home row quality if you use your index fingers for them.
I've always thought these are good ergonomic keys. The problem is they aren't great for optimizing rolls when many of the good keys are located at the home and top rows (and rightly so.)