If I answer you no then I'll be lying. Even though my hope is for us to see 1 btc equivalent to 1 btc some day with the price of fiat mean nothing to us, the current movement of the price still gives us the indication in how well accepted btc is and its demand. I do agree with your reasoning but until the I will still closely follow the btc price movement.
Frankly yes, It does matter 
Specially, for me as a student, bitcoin is just like a helping hand, it's a source of income and when the price gets high it can help me a lot specially right now. When the price gets high lots of people gets curious about it and as people knew about it the demand also gets high making bitcoin to have a value in our everyday life.