Private keys are used to sign Bitcoin txs to "prove" the tx was created by someone with access to the private key.
Public keys are used to verify Bitcoin txs are properly signed by the correct private key.
When you say Bitcoin txs are signed, what exactly is signed? I mean, is everything signed? the outputs, the inputs, and any other parts making up the transaction that I might have missed
A hash of a simplified version of the transaction.
If I remember correctly it is a signature of a hash of everything except the scriptSig fields of the inputs.