How about losing money? For sure it would really add up some stress because even if you do say that it doesnt really matter but as a human where emotion is a bit fragile ---cut.
Which really is the entire point of when people repeatedly advice,,, never put in more than what you can afford to lose. What people do not realize this really means is, put in money you are prepared to lose. In fact, expect that you will lose it.
Then the stress is not part of the equation:)