We present basic observational data on the 6C** sample. This is a new sample of radio sources drawn from the 151-MHz 6C survey, which was filtered with radio criteria chosen to optimize the chances of finding radio galaxies at z > 4. The filtering criteria are a steep-spectral index and a small angular size. The final sample consists of 68 sources from a region of sky covering 0.421 sr. We present Very Large Array radio maps, and the results of K-band imaging and optical spectroscopy. Near-infrared counterparts are identified for 66 of the 68 sources, down to a 3σ limiting magnitude of K ~ 22 mag in a 3-arcsec aperture. Eight of these identifications are spatially compact, implying an unresolved nuclear source. The K-magnitude distribution peaks at a median K ~ 18.7 mag, and is found to be statistically indistinguishable from that of the similarly selected 6C* sample, implying that the redshift distribution could extend to z >~ 4. Redshifts determined from spectroscopy are available for 22 (32 per cent) of the sources, over the range of 0.2 2.5. Four sources show broad emission lines in their spectra and are classified as quasars. Three of these show also an unresolved K-band identification. 11 sources fail to show any distinctive emission and/or absorption features in their spectra. We suggest that these could be (i) in the so-called `redshift desert' region of 1.2 <z <1.8 or (ii) at a greater redshift, but feature weak emission-line spectra.