The whole “sex object” complaint never made sense to me tbh.

The key word here is object. When people complain about sexual objectification, most often they are complaining about people being treated as if they were sexual objects devoid of agency. There is nothing wrong with sex.