But when it comes to non-legacy addresses there is no standard signature format so electrum generated signatures can't be verified by other wallet software.
You are forgetting about BIP-137 which explicitly standardized the format for message signing for various address types (no Taproot though).
https://github.com/bitcoin/bips/blob/master/bip-0137.mediawikiSo if other software cannot verify these signatures, it is their flaw because the standard exists.
I have heard this many times, but I don't agree with it.
In my opinion, it doesn't really matter whether your address is legacy or segwit when signing a message. You sign the message with your private key. Your private key can generate legacy and segwit addresses and your message can be verified using each of them.
Exactly. We should technically be able to verify any signature from any address even from multi-sig addresses (although more complex, since we need the redeem script to be revealed). But lets not forget that the process won't be as clean as it is currently defined in BIP-137, it also would take more steps and computation time. The problem is in step one below where we have
more than one possible public keys.
Under the hood when you get the trio of [signature + address + message] the verification process is like this:
1. You take the ECDSA signature + message (hash of it) to recover the public key
s (there can be multiple pubkeys recovered. See sec1-v2, page 47 the 4.1.6 Public Key Recovery Operation)
2. You use the first byte of the base64 signature called recovery code to decide which public key was the correct one.
3. You also use that recovery code to derive the correct address to compare with the given address in the trio and decide whether the signature was correct or not.
27-30: P2PKH uncompressed
31-34: P2PKH compressed
35-38: Segwit P2SH
39-42: Segwit Bech32
If we ignore the recId then we will end up with possibly more than one recovered public key and we will have to derive multiple addresses to check against the given address. Recovery ID simplifies that. Although it will be a very trivial bunch of steps but it won't be clean as I said above and it is slower.