Smoke is not the only causative of cancer and other diseases which we don't need to assume that smokers could be the only ones that could die young.
The word only here sound somehow funny. This shows people knows it could have significant effect on their body. Like you said other things can cause people death as well but people always see smoker as people buying and doing what can easily cause their death. Although I don't against smoking but I hate being addicted.
There are other many causal of cancerous disease which may or may not be deliberately engaged in that exposes us to cancer or causes us to suffer it. But in the case of smoking it's obvious that no smoker is unaware that smoking causes cancer to the lungs. It's vividly written on the pack as a warning. So for non-smokers it's like those that smokes are giving themselves a death sentence and for the love they have towards them as brothers or sisters they can't hold it back from expressing their displeasure about how they smoke. Whether or not we say smoking is bad we can't deny it that it's harmful to our health in the long term.