It is because of the uncertainty behind the future of Bitcoin.

If the core devs were not so against any kind of increase in the max block size then the market would react differently. When the first block that "voted" for XT was mined someone sold 20k BTC on bitfinex (likely because of the uncertainty)