Yeah, that's correct, it's on us, ourselves. Even if you go to a doctor who specializes in this kind of problem, it will still be on you if you will change or not.
Being agitated when betting is normal because that's your own money that is on the line, if you cannot handle those kinds of things or emotions then it's better to just quit gambling so that you won't experience that feeling anymore. If it is stressing you out then maybe it's not for you and there are always other ways to be entertained that don't need the same amount of money that you are spending on gambling. Hobbies or sports will be a good choice.
When anxiety becomes something that causes us to be unable to accept defeat, then it can be said that we are not fully ready to gamble because we are still afraid of losing money and that is very dangerous because it allows us to do something we should not do. In gambling, we must be fully prepared to lose money, because if not, we will have problems and eventually we will chase victory or want to return the defeat which is actually not allowed in gambling.