Caves form in response to the structural control of local geology on cliff form. The exploitation of joints, faults, cracks and other irregularities all lead to the opening of caves. The caves can be either tunnel or dome shaped, reflecting the type and inclination of the geological structures. Caves are most common where massive layers overlie weaker beds with resistant ribs.

On the Devonian sandstones of Orkney, cave development can be observed in perhaps its simplest form. The rocks often have low angles of dip and orthogonal fracture systems which allow waves to attack cliffs formed of cuboidal blocks.

Box caves mark the first stage of development, where fracture-bounded blocks are removed. Block removal can occur not only at sea level but also at various heights on the cliff face, a reflection of the way in which the level of wave attack on the face changes with wave height, itself a function of coastal configuration, exposure and storm intensity. The box caves are most common at sea level and may join together in time to form a marked notch or slot at the cliff base. Other box caves and slots occur up to 10 m above sea level where hydraulic action has pulled blocks from the cliff.

Small openings may be enlarged by abrasion and by hydraulic action. On Orkney the considerable height range over which wave action can attack the base of the cliff, due to swell and the frequency of high seas, together with the way that many cliffs plunge into deep water means that abrasion is often insignificant. The compression of air in rock crevices is probably the most important process in enlarging caves. High shock pressures are generated against a cave roof when a lens of the air becomes trapped between the incoming waves and the roofs. Similarly, very low air pressures may be generated for an instant as the water leaves the cave. These extreme pressure variations promote the opening and extension of joints. The hydraulic forces responsible for liberating blocks are unclear but presumably require wave water to enter joints at high pressure. As a cavity opens it can enlarged by block collapse from the roof. Caves may eventually connect to the surface through a joint or fault-controlled shaft, known as a blowhole or gloup.