(1) Background: Environment is an independent factor that affects one’s quality of life (QoL), where studies suggest that health behaviours also affect one’s quality of life. The purpose of the present study was to examine the association between environmental conditions and QoL and how individual health behaviours affect this association. (2) Methods: Participants aged 20 or above were recruited from 11 tertiary planning units in the central part of Kowloon. These tertiary planning units were selected as they represented the overall living environment in Hong Kong, with a mix of the poorer urban areas alongside relatively affluent districts. A mediation analysis was implemented using multiple linear regressions to examine the effects of environmental conditions on QoL. (3) Results: Of the 607 eligible participants included for analysis, 390 were female and 217 were male, with a mean age of 47.4 years. Living within 500 m of a green space area had benefits on the physical aspect of QoL and physical activity but no effect on the psychological aspect of QoL. Moderate satisfaction with public spaces affected QoL positively. In contrast, less satisfaction with public spaces affected QoL negatively in both physical and psychological aspects through the mediating effect of stress. Poor environmental quality affected all domains of QoL negatively through the mediating effects of increased stress and poor sleep. (4) Conclusions: Environment is an important factor that affects individuals’ overall well-being. The interaction between environmental conditions and individual variables, especially perceived stress and sleep, is extremely important when assessing its impact on QoL. The findings of this study support the importance of individual stress and sleep in mediating the relationship between the environment and QoL for health. Further studies should be conducted to include objective measurements, such as those of cortisol levels for stress and physical fitness tests.