Aside from the slow transaction in the blockchain, this is one of the issues BTC is facing for a long time. It's transaction fee's way to high! But sometimes it's not that high, you'll have to make your transactions in a perfect timing. Do it whenever the transaction fee is low.