I also hate the confirms but what I also hate is the wallet numbers, I sometimes make a mistake and due to that I pay the wrong guy, its very annoying.
The wallet numbers work to the advantage of those who want to have as many wallets as they can. They can just create a new one everytime for whatever purpose it may serve. And the many digits in a wallet makes it more secure, but also as you mentioned could easily get mixed up and you could be sending it to somebody else. Although what I don't really like is the transaction speed, I am hoping that in the future this thing can be improved to make transactions almost real time.