Epigenetics provides a mechanism for cells to ‘remember’ early developmental decisions in the absence of the signals which first initiated them. Methylation of CG dinucleotides is amongst the most iconic of all epigenetic systems because there is a well defined mechanism by which it transmits molecular memory following cell division. Accumulation of methylation during development is essential for mammalian development, and its removal can help cells to attain the naive pluripotent state that defines the most developmentally potent cells of the body. Currently, the extent to which CG methylation exists in divergent vertebrate species is largely unknown, and the extent to which it can be erased and manipulated is poorly characterised. We report our recent results in this area and discuss its implications for the understanding of epigenetic memory systems.