You always aggregate from the genesis block to the required block... The latest block does not specify all unspent outputs (which you can use to calculate the total value funding each of your addresses). Basically, a block contains a bunch of transactions. Each transaction just tells you which unspent output(s) from the UTXO set are used, and which new (unspent) outputs are generated. Offcourse, there is also some other data in a block (like the signature , the block header).

In order to generate the UTXO set (the set of unspent outputs that you can use to calculate your balance), you start from the genesis block and follow each unspent output. As soon as a transaction uses an unspent output from your current utxo set, it must be removed from the utxo set, and the output of the transaction will be added to the utxo set... The coinbase transaction also generates a new unspent output. If you do this upto height x, you automatically know all unspent outputs at height x, so you can use them to calculate the balances.