Yes, but the clear message here is that it is the firmware that is open source, not the SE itself. I think that talk of "open source SEs" and the like is very misleading.
So in case open source secure element was used in closed source hardware wallet, I would wrote NO in my Open Source column.
Again, there is no such thing as an "open source secure element". Only the firmware can be open source. You call it nitpicking but this is a crucial difference.