Currently, there is a lack of evidence to show that exercise therapy improves sarcopenia in older patients in clinical practice. We therefore conducted a retrospective cohort study to clarify the effects of chair-stand exercise on improving sarcopenia among patients diagnosed with sarcopenia undergoing convalescent rehabilitation after stroke. According to the latest Asian criteria, sarcopenia was diagnosed when both skeletal muscle mass index (SMI) and handgrip strength (HGS) were low. Patients were asked to perform a repeated chair-stand exercise as whole-body resistance training, in addition to the rehabilitation program. Outcomes included sarcopenia rates, SMI, HGS, and physical function at hospital discharge. Multivariate analyses were used to examine whether the frequency of daily chair-stand exercise was independently associated with the outcomes after adjustment for potential confounders. After enrollment, 302 patients with sarcopenia (mean age: 78.6 years; 46.4% male) were analyzed. Overall, sarcopenia prevalence decreased by 21.9%, from 100% at admission to 78.1% at discharge. Multivariate analyses showed that the frequency of the exercise was significantly associated with the presence of sarcopenia (odds ratio: 0.986, p = 0.010), SMI (β = 0.181, p < 0.001), and HGS (β = 0.101, p = 0.032) at discharge, respectively. The chair-standing exercise was effective in improving sarcopenia in these patients.