If you are in a league with friends, then I don't consider it gambling. Other fantasy leagues that you can play against the public are the same as gambling and can lead to other forms of gambling.
Playing with friends or in a closed organized group can be pretty nice, and I wouldn't consider it gambling. It is rather organized entertainment.
Some may argue since some fantasy sports events have prize pools depending on who organized it so can be considered type of gambling. Do you have to watch a match to know who the best player is when you can just look at his stats in fantasy athlete.
Everything that has a prize pool to enter is gambling? No that is not it. It is more of a contest with a fee with which they are limit fake or uncommitted entrants. You have to pay to sign up for it, and since it is not refundable you are not likely to fail to show up.