Private keys are used to sign Bitcoin txs to "prove" the tx was created by someone with access to the private key.
Public keys are used to verify Bitcoin txs are properly signed by the correct private key.
When you say Bitcoin txs are signed, what exactly is signed? I mean, is everything signed? the outputs, the inputs, and any other parts making up the transaction that I might have missed
In general yes.
The technicalities are a little complex.
Bitcoin tx can have more than one input. So for EACH input a simplified version of the tx consisting of that input and all outputs is constructed, that is then hashed and signed using the private key for that input. The signature and public key for the input is recorded in the final tx. So if a tx has 4 inputs there were will be 4 signatures and 4 public keys.