What are the two aims of doing it? Presumably to return as much revenue as possible without breaking the game experience, and create a product that players would feel enticed to buy.
So, cosmetics have been a tried and tested system that has seen massive profits in a number of titles. MO already features a cosmetics system whereby the items don't affect the gameplay, so this seems an ideal fit. Fan reception to said cosmetics has been overwhelmingly positive, especially with regard to designs featured in past Mario titles.
With a loot system, to create maximum drive, there must be tiers of loot. Simply getting the base reward on an even playing field each time doesn't provoke the same response as if you always felt like you might get something more valuable from the next one. As a result I'd say we needed some form of exotic item, a rare item, and the common item. You could go further but I feel three tiers is more suited to this game.
Common: Standard skins
Rare: More unique designs, with minor model differences
Exotic: A great skin with some effect
What effect would work well and he attractive enough that players would want to chase it? Well, the main new 'thing' with MO is throwing Cappy and becoming enemies. So I would suggest that when you throw Cappy with an exotic skin it yields some effects (sparkles, flames, smoke), with an additional effect on impact.
You could go one step further and have the above with the addition of different results on the enemies, but I'm honestly not sure it's needed relative to the additional work.
So now we have the product - a box that could contain one of the three things. How is it packaged? Good thing for us - coins and blocks are iconic to Mario.
So, you have purple coins (mixing red and blue - but perhaps this has been done in a Mario?) that can be bought and used to unlock special [?] boxes (loot blocks!). When you choose to buy one, one appears on a screen, Mario comes along and the block starts spinning. You hit jump to hit the block and Mario Kart style (complete with the music) the items rotate until you're left with the one you get. Mario does his "Aw yeh!" and you can choose to preview the item or equip it.
Skins already shown would be free, that would be too far a step across the line at this stage. Coins in game can be exchanged for purple coins, or used to save up and buy skins for a certain rate. Look to Overwatch for rates on the time needed to save for tiers.
That's about it I think. Oh and Mario Amiibo can be tapped once a day for purple coins equalling 50% of that required to buy a loot block. As skins are bought with gold currency this stops the bypassing of the system by just cashing in Amiibo and picking what you want.