Yes, of course. It wouldn't be very convenient otherwise. You can even go one step further, print the private key to paper and delete it from your computer, and you would still be able to get to all bitcoins that would eventually appear on the address.

To clarify a bit, there is no such thing as "bitcoins being sent to an address", per se. Bitcoins are there for the taking by everyone who has the appropriate private key.