Will that mess up an apt-get install at all?
You might want to uninstall the old apt package so you don't accidentially run the wrong one at some point. But the binary is a standalone with no separate includes, libraries, etc needed. Just put the bitcoind wherever you want and run it from there directly.
Thanks, I went ahead and did that. For now I'll just stick to manual updates of bitcoind so I don't have to rely on the PPA.