Tune bitcoind and P2Pool
If you can't use the methods above or they aren't enough you can limit the number of connections used by bitcoind and P2Pool.

For bitcoind, use the parameter maxconnections in bitcoin.conf

There's a compromise here: the more connections, the faster your node will be notified of new blocks and avoid wasting work, the faster it can include transactions with fees in the coinbase and the faster it will propagate a P2Pool block minimizing chances it would become orphan.
But the less connections, the less bandwidth used and the lower the latency.

More than 20 for maxconnections is probably overkill: from my experience (trying various values from 6 to 100) it seems there's not much gain to have past this value (and if you don't have enough WAN bandwidth it can hurt your latencies by queuing transfers between P2Pool nodes during peaks).
Note that this may change in the future if the behavior of bitcoind/P2Pool network changes: when in doubt, monitor your interface(s) bandwidth usage and raise this value when most peaks are below your link capacity.

If your orphan rate is fine, don't tempt the devil and try tuning maxconnections below 20: you may reduce your income more than you increase it...

Warning: P2Pool needs one connection to bitcoind and bitcoind tries to fill all the allowed connections after startup. If you start P2Pool just after bitcoind (or before, it will repeatedly try to connect), it shouldn't have problem connecting because bitcoind will still have connection slots available. If you set maxconnections too low P2Pool might not succeed connecting.

For P2Pool, you can do the same by passing parameters to P2Pool:
Note: orphans will quickly rise if you have very few connections (they are the means to be notified of other shares after all). I would prefer reducing bitcoind connections before P2Pool's.

In my experience you can get as low as 6 total connections (3 in, 3 out) without noticeable efficiency changes. The default values seem overkill (6 outgoing, 40 incoming). The large number of incoming connections (--max-conns) is designed to help the whole network (some nodes are behind firewalls that don't allow incoming connections). You probably should allow more incoming connections (and check that your network setup allows incoming connections) to do your part in helping the network.