I have some questions that have to do with how new blocks are created and how the total number of bitcoins is ultimately limited to 21 million. I vaguely understand that new blocks are created through "solving a cryptographic problem" and that this takes an immense amount of computing power. But what I don't understand is what the cryptographic problem they solve is or how it was made part of the blockchain.
Wow, you don't ask for much, do you. This is a huge topic. Please start by reading the whitepaper...
http://bitcoin.org/bitcoin.pdfAnd come back with more specific questions....
Could the entire 21 million BTC blockchain technically be solved only from information that was in the very first block or whatever algorithm is used for BTC?
No.
Would it be possible (in theory) for all of the information to be known by the creator/Satoshi by having a private key or something that forms the basis for the blockchain?
No.