Which bitcoin wallet you are using? Bitcoin fee depends on size of transaction rather than amount you want to send out.
If you have received small amount of bitcoin multiple times before than that will increase your transaction size.