Why wouldn't I default every time it's in my favor?
You would. But, 1) you'd only do this in the movement in exchange rates was larger than the interest rates, which it won't be a lot of the time, and 2) by construction, the loss to the other party from doing this is on the order of (R-1)(P**-P*) i.e. the product of two small things, so the risk premium this adds to interest rates should be small.