The bounty is paid by the person finding the block being allowed to simply add a 50BTC transaction paying them to the block.  It's a reward for solving the block, and comes from nowhere except the work required to solve the block.  This is permitted by the network because every client is able to verify that, for the block which contains this transaction, 50BTC is the right amount to add... i.e. by consensus of the fact that everyone is using the same implementation.  When this drops to 25BTC next year, anyone putting a 50BTC free bounty transaction on any blocks they solve will simply get their block rejected.  These special 'generation' transactions, because they contain relatively large amounts of BTC, take 120 blocks to verify.  This is called 'maturing' your generation block.  At 6 blocks an hour, 120 blocks takes 20 hours - and this is why pools often have 'unverified' and 'verified' funds - 'unverified funds' are those funds related to a generation transaction that has not yet matured.

Blocks cannot be equated to gold or BTC.  If you need a real world analogy, blocks are like the ledgers that contain all the transactions of buying and selling gold and the amount that has been mined.  If all the gold that everyone owned was stolen and put in a huge vault, then it would be possible to go through all these ledgers and work out who owns what bit of gold.  This is what your bitcoin client does all the time - it starts with 0 BTC in your wallet, and then goes through every block and every transaction in that block to see if anyone has paid you BTC, and if so, adds it to your wallet (it determines this using public key cryptograpy), it also goes through and sees if you've paid anyone BTC and subtracts that.  Thus, a complete record of the ownership of every BTC in the network can be determined by simply parsing the block chain.

Will