There are a couple of things that seem to slow my speed down from +-300 Mhash/s to only 300 Khash/s.
The decrease is usually when i get the messages:

 "long poll: new block...."
and

ADL_Adapter_ID_Get failed, cutoff temperature disabled for 0:0:Pitcairn

I don't know why my speed drops after a new block has been found but it does, and when I stop/start for a sec it will be back to normal.
I don't even begin to grasp what the whole ADL Adapter thingy might mean. Can it be my PSU isn't sufficient anough?