is there built in support for failover to different configurations if (eg) a p2pool node goes down, or would one need to write their own external script for that?
No native support for that at this point, sorry. See
here for some sample failover scripts.