It is relative and depends on the amount, although bitcoins from that time in wallets are usually large amounts now, because back then they were worth much less and could be mined from your computer. The ones that could cause panic if they move are the ones attributed to Satoshi, although I think it is likely that he destroyed the keys to them as a donation to the community.