This paper proposes multi-agent energy storage system aggregation as a means of scaling energy management to low voltage microgrids with distributed energy storage systems. Based on this concept, a hierarchical control strategy is developed for an AC microgrid with distributed battery and ultracapacitor energy storage systems. On the tertiary control level, the energy management problem is made scalable by considering each type of energy storage system in aggregate. This addresses the 'curse of dimensionality,' since additional energy storage systems do not increase the optimization problem dimension, and allows nonlinear energy storage models to be used for optimization, accounting for variable efficiency, self-discharge, and lifetime degradation. On the secondary control level, multi-agent state of charge balancing, reactive power sharing, frequency restoration, and voltage restoration are combined, to aggregate energy storage systems for the tertiary control. This includes the novel use of multi-agent sliding mode control for state of charge balancing between AC microgrid energy storage systems. Unlike a linear state of charge balancing strategy, circulating currents are prevented, increasing efficiency and reducing lifetime degradation. An RTDS Technologies real-time digital simulator was used to verify the performance of the proposed control strategy.