Mulberry ( Morus spp .) is an economically important plant as the main food plant used for rearing domesticated silkworm and it has multiple uses in traditional Chinese medicine. Two basic chromosome numbers ( Morus notabilis, n = 7, and Morus alba, n = 14) have been reported in the genus Morus, but the evolutionary history and relationship between them remain unclear. In the present study, a 335-Mb high-quality chromosome-scale genome was assembled for the wild mulberry species M. notabilis. Comparative genomic analyses indicated high chromosomal synteny between the 14 chromosomes of cultivated M. alba and the six chromosomes of wild M. notabilis. These results were successfully verified by fluorescence in situ hybridization. Chromosomal fission/fusion events played crucial roles in the chromosome restructuring process between M. notabilis and M. alba. The activity of the centromere was another key factor that ensured the stable inheritance of chromosomes. Our results also revealed that long terminal repeat retrotransposons were a major driver of the genome divergence and evolution of the mulberry genomes after they diverged from each other. This study provides important insights and a solid foundation for studying the evolution of mulberry, allowing the accelerated genetic improvement of cultivated mulberry species.