The advantage of segwit is that it elegantly fixes a couple of other hard problems (malleability, O(n^2) sigops issue)
What about fixing those "other problems" (I don't want to say "hard", because IMO they aren't "hard" by themselves) without the segregation? Impossible or just not worth it?