Nothing about RBF makes any of the transactions being replaced invalid.

When someone makes a replacement transaction, think of it as a brand new transaction which just happens to spend (some of) the same coins. Because, well, it is. It is not the old transaction with a new signature - it is a completely different transaction. The old transaction is valid. The new transaction is also valid. Both remain completely valid and either can be mined, right up to the point where one of them is included in a block. Only at that point will the other transaction become invalid since (some of) the coins it spends will now be spent elsewhere.