Monitoring the Dynamics of Network Traffic by Recursive Multi-Dimensional Aggregation

A promising way to capture the characteristics of changing traffic is to extract significant flow clusters in traffic. However, clustering flows by 5-tuple requires flow matching in huge flow attribute spaces, and thus, is difficult to perform on the fly. We propose an efficient yet flexible flow aggregation technique for monitoring the dynamics of network traffic. Our scheme employs two-stage flow-aggregation. The primary aggregation stage is for efficiently processing a huge volume of raw traffic records. It first aggregates each attribute of 5-tuple separately, and then, produces multi-dimensional flows by matching each attribute of a flow to the resulted aggregated attributes. The secondary aggregation stage is for providing flexible views to operators. It performs multi-dimensional aggregation with the R-tree algorithm to produce concise summaries for operators. We report our prototype implementation and preliminary results using traffic traces from backbone networks.