Factor in difficulty increasing 50% every 10 to 14 days with no increase in BTC's value. The last two 50% increases the average BTC/USD$ exchange rate has actually gone down, not up.
It would be myopic to lend so much credence to a microbubble that was largely from worldwide media hype and come to the conclusion that the BTC is on its way down. Discounting the artificial spike, if you look at the 2 month curve up to today, the BTC has enjoyed a steady and substantial increase in value. It was around $8 and change at the start of June IIRC. Today it's $15.xxx. That's nearly a 100% increase in a little over 3 weeks.
If you're going to factor in difficulty increases, you HAVE to factor in a logical valuation of BTC progression that follows each increase. But then again the market sometimes has a mind of its own. That's the risk we all take when "investing" in new hardware. But if historical charts mean anything, I'd say it's not at all too late to build a rig and start mining.