I am holding legacy.
Confused, should I go segwit for seemingly lower fees?
First, I tend to agree with all comments above that have more advantages is using the Segwit address which far better when we talk about the fees. But what is the purpose of switching Bitcoin wallets that supports Segwit address if you aren't regularly having transaction, if you're holding it for a long term, that is okay using a legacy address.
Pros and cons of each?
It was said above, Segwit wallet is for the reduction of transaction fees, also the best of signature, and in the network optimizations while the legacy wallet address is the origin of the Bitcoin wallet which is having high fees.
Can you go back and forth?
It is absolutely Yes but it depends on which Bitcoin wallet you've used because there are some Bitcoin wallets that didn't fully support Segwit addresses, nested, or native addresses. Probably it's better to check it first before you decide to have a transaction.