We investigate how the ghost condensate reacts to black holes immersed in it. A ghost condensate defines a hypersurface-orthogonal congruence of timelike curves, each of which has the tangent vector u^\mu=-g^{\mu\nu}\partial_\nu\phi. It is argued that the ghost condensate in this picture approximately corresponds to a congruence of geodesics. In other words, the ghost condensate accretes into a black hole just like a pressure-less dust. Correspondingly, if the energy density of the ghost condensate at large distance is set to an extremely small value by cosmic expansion then the late-time accretion rate of the ghost condensate should be negligible. The accretion rate remains very small even if effects of higher derivative terms are taken into account, provided that the black hole is sufficiently large. It is also discussed how to reconcile the black hole accretion with the possibility that the ghost condensate might behave like dark matter.