That is one way to look at it and you could call that a downside, another one is you'd have less physical activity, which isn't so healthy, except of course if you prioritize taking a walk and going to the gym. That said, nowadays people who work remotely are also able to network online, with the people who are in the same field as them, they might not meet physically, but they are able to get better opportunities through networking online and through recommendations.