When price is high compared to difficulty more is spent on mining hardware and less on directly on bitcoins. Therefore difficulty increase does have a small and delayed effect on the price of bitcoin.