Open up your makefile, and add -O3 -march=native -s and recompile it from source.

Also, do that with libgmp for even more performance.