Yep.  Of course, there are no actual "coins", that is just a term used to make it easier for the average person to talk about.  But what represents the value of transactions in the blockchain ("inputs" and "outputs") can be traced back from address to address all the way back to when it first came into existence with a miner.

Now multiple inputs get joined together into fewer (or more) outputs so you end up with quite a tree of connections after a while, but if you want to follow it from address to address the information is all there.  The blockchain is a record of every valid bitcoin transaction that has ever existed.