I don't trust hardware wallets, I use a 2 laptop setup with 2 Bitcoin Core instances, one used as cold wallet and another as hot wallet with watch only addresses.
If you can trust Bitcoin Core, i don't understand why you don't trust hardware wallet. There are some hardware wallet which is fully open source, where you can even build one by yourself.
Let's just hope no fatal bug is added along the way as such experimental features are added on the most used node software.
If most of the network uses code that eventually adds some sort of exploit or bug with all that experimental creativity being added you will care as you see the price crash irrespective of how you are holding the coins.
Good thing Bitcoin Core isn't the only Bitcoin full node software out there. Anyway, if you're looking for something that less "bloated" than Bitcoin Core, you should check gocoin[1]. It's solo developed by one person for more than a decade and have less code lines.
[1]
https://github.com/piotrnar/gocoin