...If someone were to own large amounts of Bitcoin though, it is impossible for him to just die without leaving a clue for his family in order for them to access his wallet.
Agree, but recoverability of such coins would vastly depend on how knowledgeable the family is. If they aren't aware of what Bitcoin is and of the holdings the deceased person - then the risk of them throwing away any pieces of paper with passwords on it (or erasing usb devices etc) is much higher.
There's also popular opinion that it's best not to share information of you holding significant amount of cryptos even with close family - for the safety reasons or in case of divorce.