They seem biased to me. A friend of mine who's doing an ICO told me that they paid 10ETH for 5 quality ratings and that's that! I noticed that some ICOs with good project, team, and good social media presence are struggling with getting good reviews.
With all that said, should it still be a factor to look at when investing in ICO?
Short Answer: Take the rating with a pinch of salt. Spend more time reading the Whitepaper and evaluating the team.
Long Answer:
1) You can pay ICO rating sites to be listed as premium/hot/special/featured/editor's pick listings. These are the ICOs that appear right in your face on the home page so they can more visibility.
2) Ratings received by ICOs can vary greatly from one to another. And ICOs will only feature their best ratings on their website and omit the bad ones.
3) Do some homework and see who rated the ICO. ICOBench, for example, has Benchy (their bot) to do the first round of rating and typically gives quite a high score. Experts rating are supplemented later and usually pulls the overall score down.
4) Scores are not stagnant, it gets updated over time. It is often affected by things like social media activities, how active is their Medium blog etc.
5) On that note of point 4, many of these can be faked. It is common for ICOs to have bounty programs to increase their social media activities, add fake followers in their Telegram group etc.
[Note: I've been part of an ICO team before so I've seen all this happen before my eyes.]