Comparing Mean Field and Exact EM in Tree Structured Belief Networks

We make a thorough comparison between a variationally-based learning approach and exact EM using tractable xed architecture tree-structured belief networks, and so gain valuable insights into learning with mean eld methods. We then introduce disconnections into the model showing how they can be folded into a single structure by viewing them as degeneracies in the conditional probability tables, and investigate learning with them. The results suggest that mean eld performs suciently well to be useful in learning in more complex models where standard approaches are intractable.