How exactly are we losing profitability against LTC if it's profit-switching? Surely it would switch to LTC if it's more profitable instead of mining lower profitable coins

I just don't see the logic there.
Even if you mined only LTC, you would dip below and above expected profitability constantly. It's called variance, or luck.