Although increasing body weight has been regarded as protective against osteoporosis and fractures, there is accumulating evidence that fat mass adversely affects skeletal health compared with lean mass. We examined skeletal health as a function of estimated total body lean and fat mass in 40,050 women and 3600 men age ≥50 years at the time of baseline dual-energy X-ray absorptiometry (DXA) testing from a clinical registry from Manitoba, Canada. Femoral neck bone mineral density (BMD), strength index (SI), cross-sectional area (CSA), and cross-sectional moment of inertia (CSMI) were derived from DXA. Multivariable models showed that increasing lean mass was associated with near-linear increases in femoral BMD, CSA, and CSMI in both women and men, whereas increasing fat mass showed a small initial increase in these measurements followed by a plateau. In contrast, femoral SI was relatively unaffected by increasing lean mass but was associated with a continuous linear decline with increasing fat mass, which should predict higher fracture risk. During mean 5-year follow-up, incident major osteoporosis fractures and hip fractures were observed in 2505 women and 180 men (626 and 45 hip fractures, respectively). After adjustment for fracture risk assessment tool (FRAX) scores (with or without BMD), we found no evidence that lean mass, fat mass, or femoral SI affected prediction of major osteoporosis fractures or hip fractures. Findings were similar in men and women, without significant interactions with sex or obesity. In conclusion, skeletal adaptation to increasing lean mass was positively associated with BMD but had no effect on femoral SI, whereas increasing fat mass had no effect on BMD but adversely affected femoral SI. Greater fat mass was not independently associated with a greater risk of fractures over 5-year follow-up. FRAX robustly predicts fractures and was not affected by variations in body composition.