I wouldn't say there is anything wrong with it at all. Everyone needs time to themselves, just because you have a family doesn't change that. I finish work around midnight and I enjoy being able to come home and do my own thing for a couple of hours before going to sleep.
It's essential to have 'me time' in a relationship, not having so risks co-dependency (or symptoms thereof). What OP is describing seems very healthy to me, as does your comment (:
Unfortunately, even if both in the relationship have “me” time, codependency can still be a likely possibility.
I think to avoid codependency one must truly be able to enjoy one’s own company no matter the circumstances, or have a very active and fulfilling social life outside of the relationship.