This paper presents a hierarchical learning framework that can learn a wide range of push recovery and balancing behaviors, i.e., ankle, hip, foot tilting, and stepping strategies. The policy is trained in a realistic physics simulator using the robot model in a setup designed to be able to easily transfer and deploy synthesized control policies to real world platforms. The advantage over traditional methods that integrate high-level planner and feedback control is that one single coherent policy network is generic for generating versatile, unprogrammed balancing and recovery motions against unknown perturbations at arbitrary locations (e.g., legs, torso). Furthermore, the proposed framework allows the policy to be learned with any state-of-the-art learning algorithm. By comparing the proposed approach with other methods in literature, we found the performance of learning is similar in terms of disturbance rejection with additional benefits of generating generic and versatile behaviors.