(As in, I think that that would fix the specific problem you're talking about, but, not without producing some other consequences, too.)
I may not be understanding what other consequences there are, but would "object-fit: contain" help the other consequences?