As said, that's what happens.

When someone creates a transaction and funds one of your addresses, they create a UTXO, which stands for Unspent Transaction Output. The more the inputs you have (which are essentially your UTXOs), the more expensive it'll be to spend them all.

Also note that if you use Segwit, it'll come cheaper.