I suspect that gmaxwell started this discussion in order to let LTC fail. Changing hash algo is very dangerous and is the easiest way for a hard fork. Enough altcoins have dies because of hard fork.
It indeed can be dangerous if the network topology is truly decentralized, but not so much when there are no more that several dozens mining pools. We already have a precedent, the transition can be done safely in the same way that the BIP16 transition was done, i.e. first a majority of the miners vote that they are willing to accept the change in the future, and only then after there's a significant majority they switch to a client that actually implements this change.