You just need to wait until the Bitcoins are fully confirmed. Then they're yours to spend or keep as long as you want. And yes, somebody could send you stolen Bitcoins, but there's probably some fiat coins or notes in your physical wallet right now that have been stolen or got 'dirty' somewhere down the line.