So, I send you 25 BTC, wait for you to accept it and send me my physical item, then I spend the same UTXO to myself and both are destroyed by your new rule, giving me my money back. This is exactly the same as a regular double spend.
You would lose your money, not get it back.