In fact, if you do it in a hard fork, you can redesign the whole transaction format at will, no need to do so many different hacks everywhere to make old nodes unaware of the change (these nodes can work against upgraded nodes in certain cases, especially when some of the upgraded hashing power do a roll back)
No, you can't-- not if you live in a world with other people in it. The spherical cow "hardforks can change anything" ignores that a hardfork that requires all users shutting down the Bitcoin network, destroying all in flight transactions, and invalidating presigned transactions (thus confiscating some amount of coins) will just not be deployed.
This is a networked society, I don't think a hard fork is that difficult as you said. Ethereum just had one and no one complains
Just like a soft fork, you have a long period to inform all the users to upgrade, those who don't care, their software will just not be able to talk to the network and the transactions will be dropped. Anyone can make a hard fork right away, but if major exchanges, major service providers/merchants are not accepting his coins, there is no point of that minority coin
When a large bank upgrading their system, all the users of that bank can not access the banking service for at least hours or whole night/weekend, no one complains. And sometimes when they have an incident, that could happen during middle of the day and suddenly all the payment can not be done in the whole country, still no one cares, only a piece of news appear on the newspaper
Of course banks can always reverse transaction so it's a bit different than bitcoin. However, bitcoin is use at your own risk, no one will compensate anyone's bitcoin loss due to incompetent devs or forks, so it is the user's responsibility to keep himself updated with the latest change in bitcoin