There's a big difference between being a slave and a plantation worker. There's a difference between being a slave to your job and being property.
Around the Haw Ivan the emperor was treated as a slave but also in the traditional society 99% of the traditions were that slaves were among the lower castes along with the upper classes of the society. They captured the property of the poor and enslaved them.