If everyone went solomining I think it could effect the diff. The hashrate is still the same, yes, but diff is not calculated from hashrate, but from the time it takes to find 2016 blocks.
it won't happen, but we can talk about that here, it's speculation thread

I hope i wasn't suggesting we shouldn't discuss it just that it is a bit out there. I would put pool mining stopping on the same leval of speculating what would happen if every bitcoin hardware manufacture went out of business. If either were to happen it would be because of another huge influence that would be way more devastating or innovative.