loading times are fine on Switch. and that's coming from someone used to M.2 nvme speeds.Being able to use m.2 nvme storage drives instead of SD cards for much faster loading times.
1. m.2 drives use way more power than a cartridge/sd card. that'll affect battery life. they can use up to ~7W. might not sound a lot but in a portable device it's a big difference.
2. m.2 drives are quite big for a handheld device. most PCs use the 2280 size. you can get 2242 size but they aren't as common and will probably cost more.
3. m.2 drives run hotter and therefore need better cooling. again not ideal for a portable device. a basic PCIE3 drive can hit 40C without doing too much and with a heat sink on it. a PCIE4 drive will get hotter than that. PCIE 5 drives are expected to require fans.
of course if they release a traditional console that hooks up to the TV that won't be an issue but i think they'll do another handheld.