I just noticed that every address starts with a capital letter or with a number (after its prefix) and I want to understand why. I'll take a p2pkh address as an example.
Its specific to v1 addresses as they're case sensitive. Just to correct you, not every address starts with a capital letter or a number. It is simply just that the distribution of it is very skewed; addresses with lower case can only be 33 characters long due to the nature of base58 encoding. 33 characters v1 addresses are actually much rarer than those with 34 characters.
Address starts with a lower case and also not a number: 1sakm3t8NqYyeuGtdP265z6GGU8LctNoE