Idea engines: A unified theory of innovation and obsolescence from markets and genetic evolution to science

Abstract

Innovation and obsolescence describes dynamics of ever-churning and adapting social and biological systems from the development of economic markets and scientific progress to biological evolution. The shared aspect of this picture is that agents destroy and extend the "idea lattice" in which they live, finding new possibilities and rendering old solutions irrelevant. We focus on this aspect with a simple model to study the central relationship between the rates at which replicating agents discover new ideas and at which old ideas are rendered obsolete. When these rates are equal, the space of the possible (e.g. ideas, markets, technologies, mutations) remains finite. A positive or negative difference distinguishes flourishing, ever-expanding idea lattices from Schumpeterian dystopias in which obsolescence causes the system to collapse. We map the phase space in terms of the rates at which new agents enter, replicate, and die. When we extend our model to higher dimensional graphs, cooperative agents, or inverted, obsolescence-driven innovation, we find that the essential features of the model are preserved. In all cases, we predict variation in the density profile of agents along the spectrum of new to old such as a drop in density close to both frontiers. When comparing our model to data, we discover that the density reveals a follow-the-leader dynamic in firm cost efficiency and biological evolution, whereas scientific progress reflects consensus that waits on old ideas to go obsolete. We show how the fundamental forces of innovation and obsolescence provide a unifying perspective on complex systems that may help us understand, harness, and shape their collective outcomes.