I would say that for most, it's the UX friction, especially for those getting used to CEX-based experience, imo.. Other than that, I think it will become better and better as time goes on + the BTC payments made through Lightning is only going to increase..
I would say UX friction comes second after adoption. The lack of adoption is really the biggest reason. Lower fees are all nice and good, even if you have a decent UX design that allows even newbies in crypto to easily navigate their payment experience but if no shops are offering Bitcoin payments then lightning is not going to be any good in the first place.
Lightning is still too early, basically.
But it has the chance to grow, adapt and become more amazing and will be ready for the future adoption.