Just raising the block size was a simple and elegant solution that could have avoided a contentious hard fork
Evidently not. The option to hardfork to a larger blocksize was freely available, but not enough people chose to go with it. The majority decided that a softfork to an opt-in 4MB blockweight with SegWit and Lightning was a more sensible option. Plus, it's arguable that one hardfork to increase the size wouldn't be enough and future hardforks would likely be required.
The Lightning Network is a "second tier" payment protocol that operates on blockchain (most often Bitcoin). It allows instant transactions between participating nodes and has been touted as a solution to bitcoin scalability issues.
People often describe it as "second tier", but in the big picture, I tend to think of it more as "inter-tier". It will effectively weave any future tiers together and allow you to move your funds between them.
Did people really choose not to go with a hardfork? I don't think it was ever up for debate. The core team refused to go along with a 2x increase, so instead it was picked up by a different team which basically made it a hostile takeover of Bitcoin. There was never a search for a consensus around the blocksize increase, because core team refused it in the first place. There is nothing dangerous about a hard fork if you get a consensus, but since the devs said no that was the end of it. The majority of people in the spring opted for segwit plus 2x, but then core said no to 2x so the 2x idea became basically a competing chain and that risked the network so it lost support.
To other people's answers, there is no reason a hard fork should take years to do. The coding could probably be done and tested in weeks. You just have to build a consensus around the fork, and for most of last year their was most definitely a consensus that bitcoin needed bigger blocks but core stopped the consensus (which is a terrible thing considering Bitcoin's development is supposed to be based on consensus). A block size increase would have alleviated the insane fees at the end of last year a lot, and it would have left the Bcash shills with much less propaganda. Now of course block size is only one part of a solution for scaling. But it is needed, absolutely. The question is how much do you do it so as to weight security/decentralization vs capacity. Probably a 2x would have been appropriate at that time considering segwit already increased the amount of data being sent around.
As it stands LN will likely only be able to handle at best a few million users, at worst maybe a few hundred thousand users, which even now is only a tiny amount of all bitcoin users. One would think the simplest scaling solution would be done first which would have an immediate impact unlike segwit/LN which will take years for its network effect to really become revolutionary.
Anyone who says onchain scaling is a terrible solution is as bad as anyone who says 2nd layer is a terrible solution. Both are needed if bitcoin is ever actually going to be used by the world.