The difficulty will NOT go down as there is a "ice-age difficulty bomb" in place - "it's programmed to raise difficulty exponentially".
Difficulty does not only depend on the hashing power but also on the number of blocks mined and times passed since the last update.
Difficulty bomb graph: https://etherchain.org/charts/difficultyBomb
Difficulty calculation: https://gist.github.com/CJentzsch/c78768f9837afb8eef74

   August 25th - 25 seconds
    September 24th - 32 seconds
    October 31st - 41 seconds

As you can see, the block times are increasing exponentially and by October 31st, it would take 41 secs to mine 1 block of 5 ETH, meaning we would earn approx half of what we do today. (Not taking into consideration any new miners coming on board which also drives difficulty up).