I do not think you need to worry about this cat. Because he was doing fine before you moved in he will survive on his own. It is sad to leave him but if you do not take him with you you can not feel bad about him.

He will miss the daily offerings yes. If you are lucky then the next person who lives in your place will leave him offerings. If you want you can leave a letter for the next people so if this cat comes looking for his offerings he does not get kicked away.