1) Bitcoin base layer - Who determines the bitcoin transaction fees? Is it the sender? For example, if the sender wants to increase the likelihood of miners including the senders transaction into the candidate block, then the sender can set a higher fee to compensate the miner?
Yes, it's the sender. But take Bitcoin node usually won't broadcast transaction with fee lower than 1 sat/vbyte, so in practice sender can't set fee lower than 1 sat/vbyte.
If this is the case, then how come most of the common exchanges have a fixed bitcoin withdrawal fee (usually around 0.0004 bitcoin or around $20), which I am unable to change?
Usually it's because exchange take profit from withdraw exchange. Only small portion of the exchange withdraw fee goes to bitcoin transaction fee, but usually high enough to ensure fast transaction confirmation.
2) Lightning network - If A wants to send to D and it appears that the only way to construct a route from A to D is if A goes through the payment channels A -> B -> C -> D, then does that mean in order for B and C to act as "intermediaries", they must be compensated by transaction fees?
No, B and C can choose to route the transaction without any fee.
And if so, does that mean if we have A -> B -> C -> ... -> Z, then transaction fees would be really higher, potentially higher than if A sent directly to Z via Bitcoin base layer?
Theoretically it's possible, but most LN wallet will choose route with lowest you and will warn you if the routing fee is higher than specific threshold.
And how are these transaction fees determined within the lightning network?
It depends on each node.