So , are people saying that i should UNCONNECT from connected sites to m.mask , then re connect when i need to use them ?
It sounds you didn't need to do that. They will need to connect with your metamask once you wanna try to connect with them again. The site will ask you again to connect with your metamask. Somesites may still able to connect with your metamask such as dex or something like this.
Disconnect your metamask was also a good decision to prevent malicious activities with your metamask.