Yes, the problem was indeed that my coins get moved when the public key changes. Which means I would have to keep register my new public address with the Byteball bot and sign a message, which is a time consuming task. TryNinja's solution was fine.