The issue is, I didn't mess with the transaction fees since I installed Bitcoin Core.
Only to this address the priority is "Medium"
And 6 hours has been passed already, we are entering into 13 hours right now.
There is no amount left to use as transaction fee and hence, the fee is 0.00001004
BTC. Wait, that is what is best to do now. Next time onwards, check transaction fee before you send.