So basically, those big bag holders will sell at a certain point too. And guess what's going to happen to the price if that happens?

Basically yes. They are not used to hold for long. If they want to make such action, they can start it and because most in the community can easily bait even with less price margin, they will ride the trend.
That's why we must never be late in case they started a succesful trend.