Also note, there is no universal exact time that all miners and pools are using. Because of this, there may be some miners and/or pools that use the timestamp as an additional nonce. The timestamp in the block header can vary quite a bit. So, that 5 seconds might not actually be 5 seconds.
I was not looking at the timestamps for my estimate. If average block time is 10min (600s) , then a quota of roughly 1% of empty blocks found points to an average "empty block mining time" of 6s. Rough guesstimate, of course, but still way beyond the actual "assembly time" for a full block header.
From the previous answers (thanks!) I get that many widely varying economic factors come into play for a pool's decision on how long to keep mining the empty block.