Sure it might impact miners who are too small to have direct dedicated high bandwidth pipes to the top 51% miners, but the top 51% miners will not even notice if every one of you irrelevant non-mining nodes all stick your heads in the sand at every block on the chain. You just don't matter to the big boys, the most you could do would be help the big boys to shake the small players out of the game faster.