I actually stopped thinking of the BitCoin adresses as my Account number, and started considering them a reference number for each transaction

That is what I think and do. I create new address for every transaction so that I know who I receive the money from.