That's true, one-handed skill won't improve unarmed damage.
Really, I made a build with the enchantment, and with the way enchanting works, it's either very underwhelming damage, or completely overpowered, depending on how much you're willing to exploit the alchemy/enchanting loop bug. With maxed out natural enchanting, you won't be able to put any amount of damage on daedric gauntlets that will be meaningful at levels where obtaining daedric gauntlets is feasible. On the other hand, if you abuse the enchanting/alchemy loop bug to put a powerful fortify unarmed on a pair of gauntlets, you may as well put them on anything, it won't matter what you put the enchantment on because depending on how far you take your enchantment, you'll probably be killing most things in a few hits regardless of what gauntlets you're using.
All in all, if you're going for 100% fairness and no exploits, unarmed just isn't a viable combat method on any difficulty higher than apprentice (while also keeping in mind that you won't actually be able to max out your damage for a very long time, due to the difficulty of obtaining daedric armor and raising enchanting to 100 without exploits), and cheating will make the game boring. I'd say the only real, fair use of fortify unarmed is strictly limited to quickly winning brawls.