I finally got it running! It took some fiddling with Mullvad VPN (for some reason it keeps blocking internet every time I create a new installation) and installing curl, autoconf, pkg-config, libtool, libc6-dev, build-essential and libboost-all-dev (thanks to other people with the same errors for recommending all those packages).
The result is a running bitcoind on testnet4. But there's no bitcoin-qt, which I kinda expected to be part of the default Bitcoin Core source compilation.
The total blocks directory is only 297 MB. This must be how Bitcoin felt in the early days.