From what I understand they simply don't believe in a deity and that's the only thing they have in common. There are some that still have some inkling of past religiosity like those that believe there is a "force" or something out there. I've watched a few vids, they're a rather mixed bunch.
Since atheists have a common belief system and they are pretty organized why they do not consider themselves as a part of a another religion which does not have a god?
Because not playing sport is not a sport.
Nice and simple.