This study presents a Quality of Life (QoL) survey to understand the influence of the housing environment and needs of residents from different housing sectors. The research focuses on Hong Kong where living conditions have become the main affect for people’s QoL. Through a household survey using a standard instrument “Word Health Organisation (WHO) Quality of Life-BREF”, the article found that among the four WHO QoL domains (Physical Health, Psychological Health, Social Relations And Environment), Environment, particularly its constitute aspect housing environment was the most influential factor for overall quality of life for the public rental housing sector where low-income people live. This research also found that different groups of people have differing needs of their housing environments: the low-income group needs better location and privacy while the medium and high-income groups need better architectural quality. Based on differentiating their needs and wants, this research argues for prioritizing the low-income group’s needs for effectively improving their QoL.