I read somewhere that cats rarely feel attached to the owner, but rather to the house. Cat goes where the food is and doesn't care who provides it, that's why people who steal cats and have no problems keeping them in their new home.