I've also had the same problem of planets with 10000 population that suddenly only have 300.  It's especially disappointing when it's a planet that only has +11 population growth.