Small aluminum fishing boats and jon boats aren't designed for skiing or racing, so it's not necessary to outfit them with a high horsepower motor. However, they weigh more than a dinghy and require a little more thrust to move them through wind-driven waves.
Equipping your low-key incognito boat with a 75 horsepower motor is overkill, and may even be dangerous. You want something simple and quiet that can produce just enough power to move you to where you need to be efficiently and comfortably.
Small outboards in the 8-20 horsepower range are perfect options for small boats. They'll give you enough power to move across a large lake, while also offering the capability to purr peacefully into a weed-choked cove without disturbing the serenity.