Thank you all, I'm not sure but I think it's overheating and it probably started slowing down when I swapped the fans.
I still can't figure out the difficulty thing: isn't it slower to process more difficult hashes than easier ones? If it is, what does the 1TH/s speed refer to?
If it doesn't, why bother about difficulty at all?
Simply explained, it's how much work needed to gain shares in the eyes of the pool. If you have a fast miner and your difficulty is too low, there's a risk that your shares will be rejected because you could be spamming the pool.
It's completely unrelated to your miner's hashrate.