Big Things from Small Data

Scott Barvick
By | May 02, 2014

Posted in: Hosting Provider DDoS Protection

Big data is big, and security vendors across the globe participate in the phenomenon by collecting, slicing, and dicing representative traffic (good and otherwise) on customer networks to better identify, predict, and mitigate attacks on the front lines. Still, for all the data used daily to baseline, extrapolate, and diagnose, it is sometimes surprising what the right piece of ‘small data’ on the screen can tell us.

An example of this, which we see regularly, while continuously collecting data from our solution deployments - often before the end customer is aware that they are under attack. The Corero SecureWatch™ dashboards and apps are fed by sampling of traffic and specific security events. This data typically amounts to several gigabytes of data per day, per site, quickly qualifying as big data.

In the collection of this data, Corero employs proprietary technics as well as the standard Sflow traffic sampling to a provide statistically high confidence level for the representative traffic, both good and bad in a customer network. For example, a single 10G link running at an average of 640 bytes in each direction would see about 3 million packets per second. Sampling every 3000th packet would result in 1000 sampled packets per second per 10G link. By standard statistics theory, this is enough to provide a high confidence view of traffic distribution each second.

This is pretty big data, and we make great use of it. However, in addition to the volumetric attacks that easily show up in a statistical view of the data, attackers continue to develop attacks that try to hide within larger attacks or are just not based on the generation of large numbers of packets. Sometimes even seeing, let alone diagnosing, these attacks is difficult and measures other than the crunching of big data is necessary.

In these cases, it sometimes comes down a single sighting, the exact opposite of big data, that helps us identify the attack vector trying to get into – or out of – a customer network. The trick is to have a data collection mechanism that works for small data as well as big data. Corero DDoS attack and cyber threat defense products do this by scanning every packet for indications of being part of a larger attack, even if the larger attack is more sophisticated than a brute force duplication or amplification attack. If a packet looks suspicious, it is registered as a security event and is sent into our data collector for mining along with the statistical sampled data.

Imagine seeing a security event indicating a source port of 0 in a stream of data destined for (SIP) port 5060. If the attack is cycling over all source ports, there may only be a 1 in 65000 occurrence of this packet, but it would be enough to alert the support team to the potential attack. At that point, analysis of the big data could be used to pinpoint the actual attack profile and begin mitigation processes.

That is just one example of the many that we have seen in our customers’ networks. The exciting thing about it is that we can leverage big data and not-so-big-data to pretty quickly get to the root of the attacks that continually evolve in their attempts to get past a network’s First Line of Defense®.

You May Also Be Interested In: