Blockstream won't win yet until BU plan to hardfork or SW plan to use UASF, but i agree that non-voting party shouldn't be counted in few condition where update is needed.
But, i'm sure more bitcoiner will support SegWit even if hard fork happen in future.
Also, i wonder if we can accept SW but refuse to support LN

I don't think BU will plan to hardfork if they don't get enough support even they have majority hash rate. The HF should be carefully prepared.