A dedicated germline has been interpreted as necessary for somatic specialization or a device to suppress selfish conflict in multicellular organisms. These views are difficult to reconcile with the lack of a germline in plants and basal metazoans. Here we show that the need to control deleterious mutations amongst mitochondria could have driven the evolution of the germline. Unlike nuclear genes, which are clonally transmitted through mitosis, the mitochondrial population doubles and segregates at each cell division. In organisms with low mitochondrial mutation rates, random segregation over multiple cell divisions generates sufficient variation for selection to control mutation load. This explains the retention of somatic gametogenesis in many simpler organisms. But with higher mitochondrial mutation rates, selection favours the transition to early sequestration of non-dividing germ cells. We predict this occurred in active bilaterians and promoted the emergence of oogamy, atresia as well as complex developmental processes.