Let's spell it out.  I know it takes a little time to understand but:

V. A. R. I. A. N. C. E.

For example, you're driving to work.  You don't drive 50 MPH the whole way there.  You have to speed up and slow down based on where you travel.  Maybe get stuck behind a slow driver.  You have to start and stop and you don't go from zero to 60 in nanoseconds.  This is the same thing with bitcoin mining.  You don't mine at a constant rate. Sometimes you mine at a lower rate based on hardware errors, luck of shares, bandwidth, and latency.  That's the slow driver you're following.  You might get above your estimated hash rate, that's passing a slow ass driver.   All this averages out to your "hash rate" as estimated via the pool.  When your daily return lessens, it's because work got farther away and it takes longer to drive there.  That's the difficulty going up.  So, to sum it up...

Sometimes you drive fast, sometimes you drive slow or get caught in a school zone, but no matter what, it takes longer every day to get to work.