I mean, isn't the answer just that some plurality of the signers have the ability to kick out and override some minority portion of the signers? I guess it would be good for the system to publish rules though about what exactly should trigger this sort of vote, like if a signer is inactive for X time period, the vote is automatically triggered. Or, it always could just be something more whimsical like whenever a supermajority feels like it, they can do it.