I don't understand why you want to reduce it and you're dividing f*(SA+SB) - SA*fA by the length of the new transaction.
Note that f
B is the fee rate you should use for the new transaction. It's not the absolute fee.
As you truly mentioned, the fee required to be paid for the new transaction is f*(S
A+S
B) - S
A*f
A.
In your new transaction, you should set the the fee rate to (f*(S
A+S
B) - S
A*f
A) / S
B, so the absolute fee paid for it is f*(S
A+S
B) - S
A*f
A.
Let's say the fee required to be paid for the new transaction is 1000 satoshi and its size is 200 vbyte. I should set the fee rate to 1000/200 = 5 sat/vbyte.