It's never used for mining with E5 v1/v2 - either it's disabled in BIOS, or the miner's threads are bound to the physical cores.
That's just you. Don't assume everyone disables HT, most don't because it helps compute bound algos. The number of physical cores
is irrelevant, it's the number of miner threads, whether HT is enabled or not.
Whether by design or by coincidence most CPUs have around 2MB of cache per physical core and RandomX has been
engineered to that spec.