mine didn't take effect until i reboot, saved the configuration a few times before doing so

That's exactly what I meant by saying
it's only important to actually activate the settings
Having said that, in most miners I dealt with, saving the configuration settings whether you are adding a new pool, changing the worker or altering the fan speed, the miner will actually reboot itself or at least reboots the miner's software (Cgminer /BFGMiner), and that alone is good enough to activate the setting, but for a few miners you actually need to hit the reboot button, so it became more like a habit for me to reboot the miner after the settings have been saved.