There have been efforts to use building demolition waste as an alternative aggregate in concrete to decrease the use of natural resources for construction. The World Green Building Council estimates that the construction industry is responsible for more than 50% of all material extracted globally and that construction and demolition waste makes up 35% of global landfills. As a result, incorporating recycled aggregate (RA) in concrete production is a prudent course of action to reduce the environmental impact. This study reviews prior research on using recycled aggregate instead of conventional ingredients in concrete. The composition and morphology of different types of RA, the behavior of RA in fresh and hardened states, keyword co-occurrence and evolution analysis, and the various additives used to enhance the inferior properties of RA are discussed. The RA showed different physical properties when compared with natural aggregate. However, the addition of pozzolanic materials and various pretreatment techniques is desirable for improving the inferior properties of RA. While building waste has been utilized as a substitute for fine and coarse aggregate, prior research has demonstrated that a modified mixing approach, an adequate mixing proportion, and the optimum replacement of cementitious materials are necessary. Based on the review, the recommendation is to use RA at a replacement level of up to 30% and the addition of precoated and pozzolanic materials as a treatment to provide concrete with adequate workability, strength, and durability for structural applications.