The primary issue is not with IP addresses, though that's certainly one way for information to leak.

The primary issue is with insufficient obfuscation of the block chain. Consider Joe Sixpack who receives his salary in bitcoins (for now ignore that some countries have already made that illegal). He gets one gigantic output at the start of every month and then spends it. Now anyone who happens to receive money from Joe can just trace back the transactions in a block explorer until they arrive at a salary-sized output and make the very reasonable assumption that this is how much Joe earns. Probably he would feel that this is a major privacy violation.

Bitcoin is full of these sorts of things. Fixing it is going to be a lot of work. We started already - the payment protocol is laying the foundation for resolving it and improving the privacy of the system by allowing single logical payments to be created out of multiple independent Bitcoin transactions.