I'm guessing the deposit is for the currency exchange part(to speed up the process) since OP mentioned the possible delays could go up to a day and wanted to reduce the fluctuations as much as possible.
Ah, I see.. because he uses escrow, it takes days to get the Bitcoin.
It is not fair though if he isn't allowed to add this deposit to the total payment.
Anyway, the seller should be able to do a trick, like increase the estimation shipping cost or the actual price to replace the deposit.