I'm sorry for your lost and then no hope to get back your amount and it looks like all of your balance has already stored to the scammer's wallet. Never try to use ledger wallet because it's crap wallet and never believe any other person that suggest you to use it. Sell it and buy trezor wallet. it puts a lot of security layer to your amount and that will be more secure. A bunch of people have complained about how shitty ledger wallet is. I use trezor from 2016 and i never get any problem with it.