You mean big enough like you currently are?

Unfortunately, yes.
On that note, I've been meaning to ask other p2poolers what I should do about that, if anything. My nodes currently comprise around 46% of the p2pool hashrate. This means I could perform selfish mining attacks if I wanted to, and if I boosted my hashrate a bit, I could 51% attack the p2pool network and get 100% of the shares. Mostly, this happened because p2pool shrank from 2 PH/s to 0.8 PH/s, not because we grew much. One way of dealing with this issue is that I could take some of my hashrate off of p2pool (and maybe solo mine instead), but that would make p2pool blocks even rarer. (I think my nodes have found the last 4 p2pool blocks.) I could also maybe spread some of my hashrate among other nodes, although this would reduce our revenue. Or we could get more people to mine on p2pool, somehow.
I think, you don't have to do anything. Neither go to other nodes nor remove your hashrates from p2pool. Everyone who wanted to leave have already gone on ckpool and further reduction of hashrate is unlikely. There left those who will never remove their powers from p2pool, and, most likely, will attract other people to join here.