true, you would need to choose a bigger search radius or recover from the old map data.
It sounds like your solution to this is to make people brute force their own private keys, which is far from optimal.
However if you are in risk of losing your paper wallet for example in the refugee case, carrying it on you physically might be a bigger risk than forgetting it.
Sure, there are times when committing a wallet to memory would be advantageous. But why would you use a human selected location and password, which as I mentioned above, are of low entropy and much easier for an attacker to guess/hack than simply memorizing a 12 word seed phrase which, if generated securely, is completely unhackable.