Sounds like driver issues.  Had the same problem when I first started with my 5850. 

Had to do a fresh install of Windows and used Catalyst 13.1 and everything worked fine.