Maybe VAR refs should just tell the ref what to do and skip the bit where he runs off for 10 minutes to watch a tiny screen.
Even the VAR decisions turned out to be the wrong ones. The best example was of the penalty kick for Serbia, which was turned down by the VAR ref (Mitrovic kicked down inside the penalty box).
You are referring to that notorious Switzerland - Serbia match, right? The German referee Felix Brych was sanctioned and excluded from refereeing any more World Cup games by the way. The Serbian football federation made a complaint to FIFA after the match about the controversial penalty.
I don't think the VAR referees can tell the main referee what to do. They can only advise him on the situations, but they can't influence his decision. The main referee could refuse to watch the replays altogether as it happened in a couple of matches in Russia.