Is this something I should be concerned about?
The concern is privacy-related. No, you don't harm anything other than your privacy when reusing a Bitcoin address.
A similar, common question is the change addresses. If you haven't noticed already, your wallet software is likely to categorize addresses in "receiving" and "change". The former is when you hand over an address to someone, and expect to receive funds. The latter is, as the name suggests, your change; the destination after you've spent a
transaction output, just like in real life when the cashier gives you change.