Yes, the problem was indeed that my coins get moved when the public key changes. Which means I would have to keep register my new public address with the Byteball bot and sign a message, which is a time consuming task. TryNinja's solution was fine.
If you're just wanted to have some balance in an address for the air drop, why don't you create a single address wallet and keep the funds until the air drop?
You can create your single address wallet using this,
https://github.com/pointbiz/bitaddress.org