What matters are the consensus rules. To be Bitcoin, you must use the exact same consensus rules. XT, Classic, and BU do not use the exact some consensus rules. They have different consensus rules for a hard fork when certain bits in the version are set and a certain threshold is reached. Those are not the exact same consensus rules as defined by the reference implementation, thus they are an altcoin. It does not matter that they are compatible with Bitcoin, they are, by definition, not Bitcoin.
Don't forget, every soft fork changes the consensus rules. For instance we could soft fork to make the max block size 500kb. We can soft fork, to make it impossible for un upgraded nodes to validate transactions but have them still think they are valid.