Maybe Chelsea is the club that makes him difficult. That's because he's never managed a big club of outstanding quality. He was also at a loss to choose players to support his target. It was proven at Chelsea, so maybe what you said is true. Potter is smarter to manage ordinary players in ordinary clubs. There is no heavy pressure from supporters and management, it seems that he will be more comfortable bringing out his abilities. I think right now he just needs to get a job back soon and find a lot of experience to make him even better.
Forget Graham Potter's fiasco with Chelsea and let's talk about how Mauricio Pochettino is as their new coach. I know Pochettino as a coach who has a lot of experience in the Premier League, he also won the Ligue 1 title with PSG but now Pochettino is trying his luck with Chelsea. Chelsea fans believe Pochettino can get Chelsea back in contention at the top flight, but some doubts exist for him as well. Internally Chelsea doesn't seem safe yet, but they hope to get up and fight to find their best form again.
If someone has doubts about the ability of Pochettino and Chelsea, then I think it's still normal. Pochettino's task is tough, that is because Chelsea is inhabited by many new players who have not adapted well to their teammates.