Imagine someone said to you "Go take $100 out of that wallet that is lying there, use it to buy bitcoin, and send the bitcoin to my address. It's totally fine, that's definitely my wallet and my money, I just can't do it myself." Would you do it?
This is essentially what has happened. Someone has given you log in details to a third party's (or parties') account(s). You have logged in to the account of these innocent people, stolen their money, used it to buy bitcoin, and sent it to the scammer. If you are prosecuted for this, then your lawyer is right. The excuse of "Someone else told me to do it" isn't exactly going to stand up in court, even if you have proof of them telling you to do it.
I think what happened here is someone first transferred money to OP's bank account, and then OP used the money he received to buy bitcoins on an exchange. If he were to use a bank account belonging to the phished person, he would have trouble passing KYC.
Not that it's huge difference for him, if his lawyers are saying he'll have to pay it back, that is probably the case.