What about pollard-rho versions of this people talk in other threads, but there is no publicly available tool to try it (to my knowledge)?

You'd probably need to perform string mangling on the GPU to keep the cores working as much as possible, because the transfer of each and every candidate phrase from CPU/system memory to the GPU could end up being a severe bottleneck. Perhaps a deliberately slower algorithm like warp may work better, since the GPU will spend most of its time calculating rather than transferring to/from the host... but how common are non SHA256 brainwallets? As you state, any passphrase based search (which is essentially a set of random keys) will be slower than a sequential search.

..For VanitySearch, having a smaller group size is better (This is a reason why I worked a lot on this DRS62 ModInv implementation). I can double the size of the group (I will definitely do it) but not more. The GPU kernel performs one group per thread and send back hash160 to the CPU. If the group size is too large, memory transfer and allocation become a problem. Divide and rule