The first pigments used in cave paintings were a combination of finely ground earth (usually in the form of clay) mixed with animal fat.

The only cave paintings I have managed to take photos of are these recreations of the white scratchy animals in Cuevas Tito Bustillo in Spain. After a visit to the Altamira cave in Spain, Pablo Picasso exclaimed “after Altamira, all is decadence”. This too is a modern reproduction; although the original earth pigments are ‘permanent’ and do not fade upon exposure to light, they can be damaged by the CO₂ and moisture from the breath of too many visitors.

Source: Wikipedia PD. A modern interpretation of a bison from the Altamira cave.

Clay was the preferred way of grinding the earth into coloured powder by allowing nature to do most of the heavy work. (Buckskin Gulch, Utah, a few days after a flash flood)Some modern cave paintings done in a reddish mud on the walls of a cave in Vang Vieng, Laos. Red Ochre or Red Earth, also called Red Oxide, is found in iron-rich soil and one of the oldest pigments still in use. The colour range of red earths go from orange all the way to a dark red that is almost purple. Red oxide pictographs on a pink stone in the arid country of BC’s Okanogan, Canada. Burnt Umber prepared the traditional way for pots in the pottery village on Inle Lake, Myanmar. 
Earth comes in a variety of colours. Driving through Death Valley we stopped at a rock formation called Artist’s Palette, which has copper green amidst the usual brown earths. 
Our guide to the multi-coloured sandstone formations near Cafayate in Argentina spoke only Spanish, but from what I could understand the yellows come from sulphur, the blue-greens from copper, the greens from a form of iron and the variety of reds also from iron (sketch and notes in the journal that I was carrying around at the time). The Hill of Seven Colours (El Cerro de Siete Colores) lies just behind the town of Purmamarca in Northwest Argentina. An adobe building in Purmamarca in Yellow Ochre with Red Ochre trim.Another adobe building that blends into the coloured sandstone surrounding it. Once when I was in Santa Fe, New Mexico, a friend took me to a home that had been completely free-formed out of adobe including the interior walls and the fireplaces. The base was mud bricks like those below, which was then coated with a mud plaster, and on one of the exterior walls a mural had been painted using the various coloured muds of the region. I don’t have any photos but am dying to get back to Santa Fe and see if I can find the place.A line of terracotta pots drying in Tzintzuntzan, a ‘Pueblo Magico’ near Patzcuaro in Michoacan, Mexico (Terracotta = Baked Earth).
Terre Verte (Earth Green) is an iron silicate mixed with clay. The post office in the recreated village of Den Gamle By in Aarhus, Denmark was painted in the historical colour of earth green.
White was sourced through ground calcite aka chalk. These chalk cliffs are at Stevns Klint in Denmark.Home-made chalk drying in Laos.

• Historical Colours at a Reconstructed Village in Aarhus, Denmark.
• A fun site about colour pigments throughout the ages: http://www.webexhibits.org/pigments/.
• More of Cee’s Fun Foto Challenge: Earth.
• More of the Lens-Artists Photo Challenge: Earth.