Reasoning about neural network perception in the situation calculus

The Situation Calculus is perhaps one of the most widely studied formalisms in knowledge representation communities where we consider actions and the changing state of the world resulting from these actions. The formalism is defined in first-order logic with limited features from second-order logic, and standard Tarski semantics suffice to interpret the properties of a domain. The situation calculus has been extended with many powerful features ranging from narrative actions to agent programs, multi-agent programs, epistemic, and hybrid aspects. Despite many attempts have been made to connect it to robotic applications. It is still somewhat of an art, rather than science, as to how exactly one can design high-level agent specifications that uniquely connect with perceptual models. In this paper, we are mainly concerned with the following question. Assuming that we have a perception unit whose functionality is provided by a neural network, how exactly are we to interpret the results of the output of the neural network as objects in the situation calculus language? We propose a simple and elegant solution to this problem by using the likelihood of actions in the probabilistic situation calculus. We develop a number of examples to show the ideas.