I'm running Ubuntu 11.04 (32 bit) Desktop with ADK 2.4 and drivers 11.6 and cgminer. No problems at all. I'd suggest starting from fresh and following closely the Ubuntu headless tutorial here,
https://bitcointalk.org/index.php?topic=9239.0(in my case I used 32 bit versions where suitable, and I ran Phoenix for a while before trying and prefering cgminer) This combo works great for me and I wouldn't suggest bothering with newer driver versions as there are generally cpu problems and no advantages afaik.