The biggest worry has to be that there's a steep enough fall in the hash rate that it starts taking, say, an hour to solve a block (and potentially 4-6 weeks to adjust the difficulty). There's not much incentive for previously sidelined hash capacity to come online because the difficulty hasn't adjusted to increase the probability of you finding a block.
If the hashing power drops significantly enough to drag out the average block time above an hour, than not only does the odds of a particular miner catching the next block increase (even if it takes longer) but the odds of catching transactions due to backups of processing increases. So there is certainly an economic incentive for a miner on the sidelines to jump in if some major miners were to suddenly drop out, even before the difficulty is adjusted.
Hence, "not much".
Electricity cost would be the most logical reason for a miner being on the sidelines (depreciation affects the decision of whether to buy/expand a rig and start mining, but not the decision of whether to keep mining because the cost of the rig is a sunk cost: once you've spent the money on a mining rig, maximal return is achieved by running that rig into the ground). If 5/6 of the hashing power dropped out, it would then take an hour to solve a block but the remaining miners would have six times the chance of solving a block. The EV for the remaining miners of coin generation over a given period of time doesn't change. Adding marginal hash capacity from the sidelines would only very slightly increase the EV over what it was pre-fall-in-hash-rate. As for transaction fees, we could start with assuming those would be increased by a factor of six per solved block (though the factor is probably lower: some number of the transactions that would have occurred later in the hour that got backed up are spending coins that were spent earlier that hour). If transaction fees are typically 1% of the generation reward (the vast majority of the most recent blocks have transaction fees that won't get to that level for the better part of a decade), then we've got an increase in EV for a miner of about 6%, so only those miners for whom electricity cost was 106% of their return would come back in: any miner whose electricity cost was more than that is still losing money if they come in and can't be reasonably expected to come off the sidelines.