...Bad CPU performance isn't necessarily a show stopper...
upgrade from Pentium D 915 2.8Ghz (dual core) to i7 3.2Ghz reduced GBT latency by 1s on unchanged OS (linux), p2pool, and bitcoind config.
That's higher than I would have expected, are you using bitcoind 0.8.3? If not, you should upgrade.
GBT latency is latency in bitcoind not p2pool and it doesn't matter much as forrestv pointed out months ago and tests confirmed.
That said, on a dual core which is used for some other CPU intensive tasks, you may experience lower efficiency (you need to have free CPU time for p2pool *and* bitcoind). A quad-core will probably handle this without lowering efficiency (count one core for bitcoind, one for p2pool and other ones as needed for the CPU intensive tasks you want to run in parallel).