you can't fight speculators and prevent them from coming in, in fact all the things you said (the adoption by merchants) is going to bring even more speculators in. it is because of the price and the way it moves. as it is adopted more then there will be a price rise because of the increasing demand. and as that happens we will see more newbies and profit-seekers come in and price goes even higher.
as far as we don't lose the objective altogether (which is adoption as a way of payment) we are fine, even with lots of speculators and weak hands.