You didn't logic it correctlly. the block size limit was used to protect against "poison" blocks. The block size limit is in the way of natural growth.
There's nothing natural about it.
A different solution to "poison" blocks exists now so block size limit is redundant. Segwit has nothing to do with any of this
It isn't a solution. It is a workaround that prevents certain types of transactions.
If Bitcoin adoption continues then natural growth occurs!ergo he block size limit is in the way. This is a reasonable assertion so what followed still stands. If you refute that Bitcoin adoption is going to continue than you have a point. Do you refute that. Do you think Bitcoin will not grow?
Given that Bitcoin will continue to grow and that there is a known lead time to safely deploy code that removes the limit (75% activation and 28 days grace) and that it cannot be reliably known that Bitcoin growth will not exceed current limits and that exceeding those limits affects the normal operation of the network then that code must be deployed now to mitigate the risk of the network not being able to operate normally.
Certain types of transactions are the problem. Excluding those types of transactions is a solution.
It's a workaround the same way the 1MB block size limit was.
If it causes problems and it can be solved differently then it can be removed.