This paper proposes a multi-agent control strategy to coordinate power sharing between heterogeneous energy storage devices distributed throughout a DC microgrid. Without requiring a central controller, the proposed control strategy extends the benefits offered by hybrid energy storage systems to DC microgrids with batteries and ultracapacitors spatially distributed at different levels of the power distribution hierarchy. The proposed control strategy has the following advantages. 1) The high frequency microgrid load is provided by the ultracapacitors. 2) The low frequency load is provided by batteries used for bulk energy storage during islanded mode, and the main grid during grid connected operation. 3) The ultracapacitor voltages are regulated at a desired reference. 4) State of charge balancing is provided between the batteries. 5) The energy storage systems cooperate based on neighbor-to-neighbor output feedback over a sparse communication network. The only communication requirement is a spanning tree from the ultracapacitor leaders and battery leaders to their respective followers. Simulations are presented demonstrating the performance of the proposed control strategy for a 380 VDC datacenter during grid connected and islanded operation.