Two transactions with different hashes and with overlapping inputs.
The definition is not entirely correct as a malled tx is not a double spend.
it will result in one though if the user has also tried to spend the change output of the original transaction that was voided by the malleable one. this seems to be happening quite a lot based on network chatter. probably because the client allows spending the change while it's unconfirmed.
I'm no expert on this stuff, but I'm wondering, could this behavior in the reference client be amplifying the effects of the malled TX attack as measured in the OP? i.e. When a client attempts to spend unconfirmed change that never gets confirmed because its tx got malled, is that also counted as a double-spend attempt?
Not it would simply be an unconfirmed tx and after the parent is dropped it becomes an invalid tx.
However it is a genuine case for concern. There is no profit in it, but the combination of:
a) malicious user malling all tx
and
b) the QT client using unconfirmed change outputs in new tx
can lead to a lot of confusion and broken transactions. I am not sure what the immediate fix it.
One option would be to either by default or by a user enabled option don't use change outputs until they have at least 1 confirmation. That combined with simply "hiding" duplicates would make the mutated spam pretty much invisible to "normal users". Merchants and service providers not using a payment processor need to understand the mutability of txids but honestly they already should.