I think an increased difficulty just means more hashes are (typically) required to create a block successfully. I don't think it should slow the actual hashing rate though... my guess would be that it's something on the client's host that is consuming extra cycles and depriving bitcoin of CPU time.