Hi,
nice design, but let me share a couple quick concerns:
1. I am not entirely sure about atmega32u4, but most atmegas do not have internal pull-up resistors (or they have some very weak ones) on the RESET line, so I always put 10k there; unless you're sure it's okay like this, I'd add an extra resistor,
2. your traces look very thin (4mil?), not many places will produce such a board cheaply, if that really is 4mil, even oshpark won't do it, and they do have very good specs; I think this board should be easily routable with 8/8,
3. U$3 - why are both switch pins connected to uC lines? You could connect the other one to ground,
4. USB is based on differential D+ and D- lines, so it's recommended to keep their lengths equal and the 2 lines as close to each other as possible; I'm almost sure your design will work in most conditions, but for the highest frequency lines on board, I would try to keep them on the same layer and close to each other,
5. it is very highly recommended to put one 100nF cap for each VCC/GND pin pair of the micro, as close to the chip as possible; this will supply the higher current during peak consumption and eliminate brownout,
6. watch out for your clearances, especially around the qfn package; the pad in the middle has to be soldered (to ground, I think), so you have to make sure the soldermask completely covers those traces around it,
7. if you're going to have the board soldered, remember about proper preparation of the solder paste layer (which isn't shown here), in particular, you cannot apply a square of solder paste as big as the thermal pad of the QFN, or the chip will be lifted too high by the surface tension and the little pads will end up floating above the board; consult google for "qfn thermal pad paste pattern",
8. use thicker power lines, I suppose even 4 mil should be enough for a chip that won't take more than 30mA, but it's good practice to use significantly thicker VCC and GND lines if possible,
9. and last but not least - consider adding a ground pour (on both sides of the board), this will provide additional shielding (especially of the USB differential pair) and shorter ground loops in some cases.