Only the displayed hash rate drops because it is calculated from the last 5 or so minutes of that coin, not of any coin. Actual hash rate stays the same.