Grid cells in the medial entorhinal cortex (MEC) encode location through firing fields that form grid-like maps of the environment. At the same time network activity in the MEC is dominated by oscillations in the theta (4-12 Hz) and gamma (30-100 Hz) bands. The relationship between oscillatory activity and grid firing is not known. Our recent experimental data establishes that feedback inhibition between excitatory stellate cells and inhibitory fast spiking interneurons is the dominant form of synaptic connectivity within layer II of the MEC. To determine if this synaptic architecture is sufficient to explain network oscillations or grid firing fields, we constructed a network of model stellate cells and interneurons. The model contains 4096 excitatory stellate cells and 1024 inhibitory interneurons. In this model, stellate cells connect exclusively to interneurons, while interneurons contact only stellate cells. We show that external excitatory conductances can drive the network into an attractor state. Feedback inhibition onto the model stellate cells has a synchronising effect. The activity of both populations discharged in the gamma frequency range (30 - 100 Hz), and when coupled with a theta modulated external drive (8 Hz), we observed the synchronisation during the trough of the theta signal only. When the borders of the network are connected with a twisted torus topology and velocity modulated inputs are applied to the circuit, excitatory neurons in the circuit generate grid-like firing fields. Due to only a partial stability of the attractor state the grid fields appeared noisy. These results have implications for mechanisms of both formation of grid-like receptive fields and also population coding and information transmission between brain areas.