Because economically speaking the volumes are tiny. The amounts of bitcoin that gets traded are in the millions, but the amount of fiat that gets traded each second are in the billions or trillions. This means that when things happen in the world economy, you have it slosh into the bitcoin economy where it moves the market wildly.