Yes, because then only miners will be able to verify data. If you ban all nodes that are not mining, then you will have only miners in the network, and then the whole P2P concept is gone, because all other users will then be entirely banned, and would need to connect to the network by using some miner as a proxy.