Okay, apparently I wasn't clear. I was trying to argue that simply taking something by force does not make it robbery or theft. I gave the examples of the bailiff and the police confiscating stolen goods. What, then is the difference between the police confiscating stolen goods and a bank robber taking cash from a bank since both involve taking goods by force?
My answer was that the thief doesn't have a legal claim on the property that the police confiscates, whereas the bank does have a legal claim on the money stolen from it.
So whether the government is robbing you depends on whether they have a legal claim on the money they are taking. Simply taking it by force is not enough.
I don't expect you to agree with me now, but is it any clearer where I am coming from?
OK. So what you're missing is that it's not about not applying force, it's about the non-initiation of force. In your scenario, the robber initiated the use of force and is therefor in the wrong.