Besides real problems regarding binary formats, its psychologically most destructive property is that everybody has the self-proclaimed best encoding, the fastest encoding, the most space-efficient encoding, the most flexible encoding and so and so which only differ in one of two bits somewhere and therefore makes them 100% incompatible. Thank you.

Also from experience, I would recommend using a Type-Length-Value (TLV) messaging structure to facilitate expandability and maintenance.  Endianness is easily dealt with.  Additionally, the binary bytestream may benefit further with addition of compression algorithms, assuming the data is non-random.