Is it simply due to lack of interest?
I mean all of that was back in 2018, but maybe now no one really demanding confidential TXs in Bitcoin, or what?
Probably yes. Other privacy option such as Coinjoin gaining popularity and it doesn't require change on Bitcoin protocol.
Can you guess what would be the worst case scenario in case if confidential transactions get introduced after soft-fork in Bitcoin?
I am sure there would be much higher transaction fees, and blockchain size would increase a lot, but I am asking for other things that could happen.
Sidechains and second layer solutions can be good for privacy, but I would prefer if Bitcoin introduced privacy on mainnet years ago.
Rather than bloating mempool/blockchain, it's more likely there would be lack of adaption partially because higher fee (due to bigger TX size) and centralized service block all coin from confidential transaction (CT). And with lack of adaption, it'll weaken privacy/anonymity offered by of CT.