But unlike other altcoins, the private keys would work
Yeah you'd be able to sign tx but the tx will never be mined or broadcast as they use inputs from some other blockchain. So "work" only in the narrowest sense. I.e. the private tx will only be valid on the altcoin.