It should indeed be better to take wallet lik Electrum. I thought Bicoin Core was the safest.
Bitcoin Core is safer than Electrum, but Electrum itself is secure for most people.
bitcoin core's block verification is single-threaded
When syncing, Bitcoin Core uses all my CPU cores.
Only some part of verification uses multiple thread/core.
I have a server running a Xeon E5-2650 which is at 2GHz and that also runs Core kinda slow, not really abysmal as OP's waiting times but not as fast as I'd like, which is speed of the node running on my E3-1240 at 3.3GHz.
It's unfair comparison since your Xeon is launched 3 years after Intel J1900 and Xeon isn't low end processor.
Using older processor such as Core 2 Quad Q9000 which released 4 years ago and also have 4 core at 2 GHz is better comparison. According to
http://cpuboss.com/cpus/Intel-Core2-Quad-Q9000-vs-Intel-Celeron-J1900, Q9000 is faster J1900 on most tasks.