So is difficulty the only price driver or one of many drivers?
Most certainly not the only price driver.
Deconstructively, what does difficulty represent?
How much computation is required (on average) to solve one block.
Time to get a block, which is what? Electricity. What other price drivers are there? Demand for bitcoins. Which is based on what? Right now I think the demand is based on investment speculation/expectation. Am I missing anything?
Well you can't eat them so sure it's (almost) all investment. There is some trade going on too, and even a bit that is increasingly closed loop. There are people doing services for BTC and then using those BTC to buy stuff, etc. Of course this is still fairly small in absolute terms, but I would guess the rate of growth is really high. Some people are probably using it for near-costless money transfers instead of checks, bank transfers or cash, especially if the recipient wants BTC anyway.
Over time international transfers should be big. Doing those with banks can be really inefficient.