There is no setting to hide the number, the number will already be hidden if the other person is not having your number saved in his phone on first place.
You had a point only owner of mobile number can know it. And if I'm not mistaken if the number stored in our mobile have also an telegram account he/he automatically can we seen their names at our telgram account.