Indoor thermal comfort/perception (ITC) would be affected by contextual factors. The present article reviewed findings (thermal responses represented by neutral temperature, NT) of ITC studies published in recent decades. There were two types of contextual factors: climate (latitude, altitude, and distance from the sea) and building characteristics (building type, and ventilation mode). Through associating NTs with their contextual factors, it was found that people's thermal responses were significantly affected by climatic factors, especially latitude in summers. Increasing latitude by 10° resulted in NT decrease of around 1 °C. Others— e.g., building characteristics— were influential to some degrees. Effects of ventilation mode (natural ventilated, NV; air-conditioned, AC) varied for the seasons. Generally, people in NV buildings had higher summer NTs, such as 26.1 °C in NV and 25.3 °C in AC in Changsha. The results demonstrated significant human adaptations to climatic and microenvironmental influences. The design and construction of future residences could be more fine-tuned with the building insolation and heating/cooling technology to fit the thermal preferences of local residents for the best internal temperature settings. This study's findings may potentially serve as a foundation for future ITC research.