In most places I know of electricity is cheaper than natural gas/heating oil so yeah I guess you would be saving $ on heating costs.
In my part of the US, Heating/Cooking/WaterHeating with Natural Gas is significantly cheaper than doing the equivalent with electricity.