Bankers can not influence the situation with bitcoin. Bankers can take an unlimited amount of Fiat money, but to have access to bitcoin they can't. Besides who wants to borrow in bitcoins if the price can rise to it. Such a loan can result in bankruptcy.
Currently I doubt if anyone will borrow in bitcoins because if the upward trend but the opposite holds true too, you can also borrow and pay less if Bitcoin prices goes down.
Yeah the risk is simply too high, if bankers borrowed in bitcoin they will be probably forced to charge interest rates which will be very high for the average person in order to try to cover for possible losses due to the changing nature of the price of bitcoin.