There are a couple of things that seem to slow my speed down from +-300 Mhash/s to only 300 Khash/s.
The decrease is usually when i get the messages:

 "long poll: new block...."
and

ADL_Adapter_ID_Get failed, cutoff temperature disabled for 0:0:Pitcairn