TL;DR try to get your rejects as low as possible for a given pool. Don't compare reject % from one pool to another pool.
Optimizing reject percentage makes sense in order to get the best results you can with your setup on a given pool. Do whatever you can to bring rejects down as low as you can FOR A GIVEN POOL.
Comparing your reject percentage from one pool to another pool (especially coin-switching pools) makes absolutely no sense whatsoever. As soon as you say something like, "my rejects at CM are much higher than my rejects at _____", then you are focusing on the wrong issue and there is never going to be a satisfactory answer.
The only thing that makes sense to compare from one pool to another pool (aside from support / aesthetic / personal stuff) is what is your profitability at pool X vs. pool Y with your rig optimized for each.
If your rejects are 50% at CM and 3% somewhere else, but you are still more profitable at CM even with the 50% rejects, then it's an error to 1. complain about the high rejects by comparing them to somewhere else, and 2. mine somewhere else merely because you think that it is bad to have high rejects.