Bone marrow hemophagocytosis is a frequently observed but not mandatory finding for the diagnosis of hemophagocytic lymphohistiocytosis (HLH). However, the impact of bone marrow hemophagocytosis on the diagnosis of HLH is still not clear in adult patients. Thus, we retrospectively analyzed adult patients with bone marrow hemophagocytosis between 2000 and 2014 to determine its clinical significance. Among 264 patients with bone marrow hemophagocytosis, malignant disorders were the predominant underlying cause (n = 170, 64 %), especially T/NK-cell (n = 88) and B-cell (n = 45) lymphomas compared to infectious disease (48/264, 18 %). The data for HLH-2004 diagnostic criteria was available in 182 patients, and only 29 % (77/264) of patients with ≥ five positive criteria could be diagnosed with HLH. Among the criteria for the diagnosis of HLH, increased serum ferritin (89 %) was more common than hypofibrinogenemia, hypertriglyceridemia, and bicytopenia (<40 %). The median overall survival was worse in patients with malignancy (9.0 months, 95 % confidence interval [CI] 5.6-12.5) than in those with non-malignant disorders (71.8 months, 95 % CI 56.5-87.1, P < 0.001). In patients with malignancy, the overall survival of patients fulfilling the HLH-2004 criteria was significantly worse than patients who did not (P < 0.001). In conclusion, our results suggest that bone marrow hemophagocytosis might be an important finding in the diagnosis of HLH in adult patients. Considering the high incidence of malignancy as a predisposing disorder for HLH, immediate evaluation should be performed in adult patients with bone marrow hemophagocytosis.