many seed wallets need to "activate" addresses they use
by this i mean you ask your (cold)wallet to supply you an address of the seed as a receiver address
however due to cold status it has no blockchain to check for a list the amounts/balances of.

if it was hot that made the tx and requested the receiver address from seed, it then triggers the wallet to scan the blockchain for any transactions that use that receiving address.

now here is the thing, because a seed is not a privkey for 1 address but a phrase for many many thousands of addresses ..most wallets dont scan the blockchain for all possible addresses of the seed. each address you intend to use you need to activate in the wallet to tell the wallet to start searching for balance info when you make payments or request addresses to receive payments
this is why you need to trigger the actual hot wallet to actually watch for the receiving address so it can scan for the transactions

basically the hot wallet has been told to watch the sending address via you importing a single pubkey of sending address but has not been triggered to also watch the receiving address because the hot wallet has not been given the receiving address to watch. because you are only handing the hotwallet public keys and as your OP mentions, you only sent it the public key of the sending address and dont want to send it every receiving address each time. thus the hot wallet does not have the public address of receiving coins because you have not told it the receiving address to watch

the easiest option is to re-use addresses where you get payments from yourself and others to a specific same address repeatedly thus its always scanning just one address you imported to watch, thus not needing to import public key per payment