Yeah, this is what I was looking for. It was confusing since I didn't see any changes, so I assume this was hidden in some further steps.
So basically, you choose the addresses you want to use normally with Coin Control, but simply click on "Create Unsigned" instead of "Send" so you get this .psbt file, then you load this file in the offline computer which hosts the private keys so you can fully sign it by loading it with the GUI (File - > Load PSBT) so this is now fully signed, and then again you back to the online computer to broadcast it. Here is where im not sure how do you do it at least in the GUI. Looks like there's some command called finalizepsbt but im not sure about that. I mean I guess there is some way to finish it in the GUI.
A little suggestion: if you are looking for an easier one and don't need to download or sync blocks, you can try Electrum as your watch-only wallet. The PSBT should be cross-compatible with Bitcoin Core, and it should be straightforward to broadcast your transaction.
But if you want to keep using Bitcoin Core, you would need "finalizepsbt" to make it raw hex and broadcast it to any of this
list or load it as text on Electrum and then broadcast.