I would like to thank all of you for helping me out.
I've created a bitcoin.conf file with the following input
dbcache=10000
banscore=10
listen=0
and adjusted the vm specs to a be more reasonable.
Everything was fine, but every time I start the Bitcoin Core, it take a hell load of time until it starts downloading the blocks, even after I've finished the initial bootstrap, as if you are forcing a car to move after letting a few guys give it a push.
This is on the VM and the main PC (host).