All of those had weaknesses
1. The Borg were defeated,
2. The Replicators where held indefinitely in a statis field.
Now the Q - beings that were only (as I recall) weak against their own kinds - now THEY could be definite contenders.
The Borg were never defeated. Captain Janeway just returned to earth.
The replicators all returned to one planet. Just don't do that.

I'm pretty sure that the replicators would be the biggest problem for everyone. You could theoretically defeat any race if you held every single individual in a statis field, so that point is invalid.