I sometimes use tow haul even with just 5-600 lb in the bed or a load extended our the back. One thing the mode does is linearizes the throttle and brake sensitivity, making the driving experience smoother, it also gives the transmission different shift point instructions, gives you an extra few flashes of signal lights when you tap etc.
The result of this is a smoother driving experience, likely at the cost of a bit of fuel and brake wear. But if I’m trying to drive carefully in town to take care of my load that is really a good thing IMO.
The settings are there for you to use, but you aren’t going to do any harm by ignoring them completely either (trucks worked just fine for decades without the modes). Just play around and see what you like, and read the manual on what switching the mode actually does and use that knowledge to your advantage. There are even some cases where tow haul or off road might be useful tweaks when not hauling or off road for example, the settings are tools at your disposal like anything else rather than defined boxes we need to live in.