The same argument can be made against a contract. One who breaches a contract is not initiating violence or making threats on your property.
A person binds themselves to the terms of a contract voluntarily. That isn't the case with intellectual property.
This exact argument can be made against laws banning murder. The murderer can argue that he never signed a contract not to kill anyone. Thus, this argument ALONE is not sufficient.
Even if you make everyone you sell a CD to sign a contract that says "I will not redistribute the contents of this CD", you still haven't recreated intellectual property because there is nothing stopping a third party that didn't sign that contract from doing so.
This exact argument can be made against third party resellers of items acquired by theft. He never signed a contract with the original party and should therefore be allowed to keep the stolen goods. Thus, this argument ALONE is not sufficient. Illegal organ sales for instance, should by this logic be completely legal.
A contract is binding because it's voluntarily agreed to. You can't automatically bind third parties which is why intellectual property isn't legitimate.
The same can be said about a slave trader. The third party who buys a slave didn't originally enslave the person. Now that slave is property and so third party slavery is ok. He never agreed not to enslave that party. Thus, this argument ALONE is not sufficient.
I completely agree that it doesn't make any sense, just like piracy doesn't require that you are causing me physical harm, but rather take away FUTURE revenue from my mental work.
That's because you've been deluded into thinking that you own the products of your work. That's false though. If you steal from me a slab of marble and then chisel a statue out of it, do you own the statue now? No, the statue is mine and I also claim that you've destroyed my slab of marble and owe me for damages. Once you empty your head of that delusion, you will stop feeling entitled to own an idea just because you thought it up.
The only thing you prove here is that the labor argument ALONE is not sufficient. I need to specify that it only applies to
peaceful work, i.e. work that does not violate the property rights of others. But whose property do I violate when I write a book? In what way is sitting in my own home writing a book on my own computer which I bought with my own money that I earned fair and square not peaceful?