I can vouch for that. My tracking of all my swaps includes monitoring the fluctuating GBP/USD exchange rate, and that easily makes bigger swings in my overall gain (as measured in GBP) than the daily payments do... although with a tendency to swing up and down to little overall effect in the longterm, whereas the swap payments maintain a slow ratchet upwards. But if sterling suddenly found some of its old strength I'd be looking at negative returns really quickly.

It also adds a barrier to moving money in/out - the currency exchange rates offered by the retail banks I have an account with to withdraw into are nigh-universally terrible, quite possibly costing me several percent of the total transferred. I'm getting close to the "fuck it, I'm going elsewhere" point, but I really don't want to do that international transfer more often than necessary. Probably means I'll stay longer than I should, then continue to stay out even as rates pick back up again.