You might want to uninstall the old apt package so you don't accidentially run the wrong one at some point. But the binary is a standalone with no separate includes, libraries, etc needed. Just put the bitcoind wherever you want and run it from there directly.