I have the feeling - but its just a feeling, you are the one with the board and the Neo experience.
I feel that if you got rid of the numbers on top row and only kept the ones in a layer, you could remove a layer.
Then you have less Mod buttons to worry about... and maybe remove things you will never use, it makes the board harder to read.
In my design example I have most the punctuation and the arrows in the main layer.
The rest are in the shifted or alternate graphics layer (Ctr+Alt). Only the shift button is present on both sides as the alternate layer is less used.
As the caps lock (Fn+Left Space) one key combination (Fn+Right Space) should lock the alternate graphics layer as it can be used as a Num pad.
Again I am basing this on feeling not experience.
You are not wrong, I really like what I have here. And there is nothing hard to read. And I don't understand why you want to have punctuation on the main layer. I don't use ? or ! very much so it is a waste of space putting them on the main layer.
I will try to explain you.
This is the minimum keyboard I need to do anything, except some limitations using desktop shortcuts. I can type text, I can develope software etc. I like to use enter, backspace or the cursor keys on layer 4 because I don't need to move my hands out of the home position. On normal keyboards I have to "open" my hands to reach enter or backspace. To use the cursor keys on a normal keyboard you have to move your hand completly of the keyboard. I just have to move my right thumb less than 2cm to the right and my left index finger 2cm down for enter. Easyer than typing uppercase letters on a normal keyboard where you have to "bend" your pinky to reach shift.
Everything around this minimum keyboard is just convenience, if you want to or if you have to use just one hand.
If you are writing soure code you will love layer 3. Typing this line of code with your design would freak me out
int i,j=0; for(i=1;i<=5;i++) { j+=i; j--; }
You have to switch between 3 layers to type characters like = () {} [] +-;: and many of them are in the top row, far away from the modifier keys. I have all these characters on one layer and I can reach them without breaking my fingers and without leaving the home position. Many german developers use the QWERTY layout because it is hard to type {}[]\ because they are in the top row and AltGr is on the bottom row
on the same hand. Thats awful
Is that two layouts ideas or a double numpad, to be used with either hands ?
It is just an example if you assemble the PCB from the back side. You can use it right or left handed.