I agree with all of that, but what makes the product of my mind (which also takes time and labour) any different from physical property?
It did not start off as an unowned resource, and it is not scarce. Scarcity is what creates the need for property, as it is the source of dispute. If I steal your physical book, you can no longer make use of that book. But if I infringe on your copyright, you still have the original book, and the book-pattern, the symbols in a particular order that form a story. Stephen Kinsella calls these "ideal objects" in his short paper
Against Intellectual Property. As they do not fit into the category of property, my questions stand.