of course. not every driver is good for every cards.
if you compile kernel on wrong drivers you will got bad results.
you can see my results in some older posts.