Thanks.
Why does it adjust the difficulty and say "LONGPOLL from pool requested work restart" so often? Is that optimizing the hash rate? Should I set the difficulty with my username to reduce this?
This is normal p2pool behaviour. New work is set every time a share is found, which in our case is 30 seconds on average (10 seconds in BTC p2pool).
The short version is that the work to produce the "next" block depends on the previous block. The moment a new block is found, any in-progress searching is aborted because it won't produce anything useful. This is important because it encourages a "found" block to be used and distributed quickly.
It might very briefly decrease your mining hashrate because of the time required to fetch new work from the pool/start working on the new work. It should be noted that p2pool has 10-second blocks so you'll see this message about once every 10 seconds. That's why you can't use p2pool with devices that take a long time to start new work like BFL singles.
Difficulty is adjusted every time a new share is found.
In any case, it is considered a good practice for miners with big hashrates to set their difficulty manually because it reduces p2pool's minimum share difficulty, helping small miners to achieve less variance. I would recommend tuning it in such way that
your estimated time to share is several (5-10) times less than
p2pool's time to block. So your difficulty can be calculated as
p2pool_difficulty * (your_hashrate / p2pool_hashrate) * (time_to_block / 30s) / 10
Details:https://bitcointalk.org/index.php?topic=18313.msg2060914#msg2060914