Edit: Yeap. They've changed things. Downloaded Electrum 4.3.4 on Ubuntu and let it sync, even deleted the blockchain_headers file and let it be downloaded again. In both cases the file contains all zeros up until block height 747936. Which is a waste of space if you ask me!!! This also means what I explained above won't work using newer version of Electrum.
That's what i suspect. I just found out Electrum protocol also have API call which can ask single block header[1] and block headers starting from specific height[2]. But those API call i mentioned could be used as alternative to obtain block headers.