I think the reason why my script was so slow is because of Python, maybe if I make a C++ version of it it'll be faster. I just have to implement bitcoin.prvtopub myself, however I do it.
There might be tools to help with that. I've just googled something called "Cython".
However, in my experience, those sort of translators never fully work and there is some tweaking to do.