I guess more than 1 million, it doesn't matter, it can't affect the future of bitcoin.
Of course it can affect the future of Bitcoin, imagine that there's 1 million coins lost and another 10k coins lost every day. This is likely possible to happen and it will affect Bitcoin a lot. Price will go up and there will be smaller and smaller rate BTC / user.