To achieve its precise neural connectivity, the developing mammalian nervous system undergoes extensive activity-dependent synapse remodeling. Recently microglial cells have been shown to be responsible for a portion of synaptic remodeling, but the remaining mechanisms remain mysterious. Here we report a new role for astrocytes in actively engulfing CNS synapses. This process helps to mediate synapse elimination, requires the Megf10 and Mertk phagocytic pathways, and is strongly dependent on neuronal activity. Developing mice deficient in both astrocyte pathways fail to normally refine their retinogeniculate connections and retain excess functional synapses. Lastly, we show that in the adult mouse brain, astrocytes continuously engulf both excitatory and inhibitory synapses. These studies reveal a novel role for astrocytes in mediating synapse elimination in the developing and adult brain, identify Megf10 and Mertk as critical players in the synapse remodeling underlying neural circuit refinement, and have important implications for understanding learning and memory as well as neurological disease processes.