I personally think that it depends at the end how much that bring to your pocket. If on the big pool you get more, stay there. If on the small pool you get more, you stay on the small pool. Big not always is better.
Stability is also other side of the question. Just keep in mind that at the pool with low hashrate you need to mine at least 60 to 90 blocks to get good statistics.