Despite increasing awareness, sexual harassment remains a significant concern in the workplace. The enduring problem of sexual harassment seems to persist due to a lack of clarity regarding what behaviors qualify as harassment. Furthermore, the interpretation of these behaviors is influenced by contextual and relational factors, contributing to the complexity of addressing and preventing such incidents. This study builds on existing research by investigating how the severity of sexually harassing behavior, the response from the target, and the gender of the participant contribute to labeling behavior as sexual harassment. Using an online experimental scenario-based survey, 1,700 (850 female, 850 male) currently employed participants were presented with a single workplace scenario that manipulated the severity of the sexual harassment behavior and the target’s response. Participants were then asked to assess the appropriateness of the behavior, label it as sexual harassment or not, and rate their confidence in their labeling decision. The results revealed that less severe sexual harassment behaviors, targets who displayed interest, and male participants were more likely to perceive the behavior as less inappropriate and were less inclined to label it as sexual harassment. These findings have implications for shaping the definition of sexual harassment and designing training programs for heightened awareness.