You are quite correct that fees are not always needed (I have rarely paid any) - provided that your UTXOs aren't too small and are old enough then no fee is required (although you may have to wait quite a while for the tx to be included in a block).