Do I understand this correctly that the blockchain would only be stored on the "server version" of the client and not by every client ? I'm not sure that's a good idea. I don't know how people would feel
Nope, you don't need whole chain for security purposes.