Thus, the incentive of using slave labor is killed; the last resort is to ask the state to protect your slave-owning habits, but since that can't happen in a society of rationals, for what rational seeks a guiding force but his own, it's effectively squashed and business continues as usual. Behold, ostracism
I think you would agree there are many forms of slave labor starting from underpayment, so in a sense any businessman hiring people could be treated as a slave holder of a kind... If what you alluded to but didn't say directly were true, we would not have such activities as illegal drugs trade (remember Silk Road?), murder-for-hire, child pornography and the stuff like that (and yes, slavery too)
