The Evolution of Analytics in Cybersecurity

Although information systems logs have been around since the early mainframe days, the concept of collecting and analyzing logs for security purposes is still a relatively new concept.
From my limited research, the term SEM (Security Event Management) was pioneered by a small company called E-Security in 1999.  SIM (Security Information Management) or SIEM (Security Information and Event Management) came along about 2005 and appears to have been coined by some folks at Gartner.
The reality is, as security practitioners, we have come to rely more and more on logs from a variety of systems including applications, firewalls, servers, operating systems, identity and access management systems, and data leakage systems, among others, to help understand where systems are in our environment, and who is using them and when.  We also use them to help trouble-shoot problems, conduct root-cause analyses, and do forensics investigations, as well as other matters.
One of the obvious challenges is that most of our environments have become more and more complex.   More applications, more virtual machine and cloud usage, more mobile devices, more “unmanaged” devices, more complicated global networks and more potential security gaps.  We have also tweaked many of the logs from these systems to give us more security related data.
The result: giant, never-ending piles of data that need to be stored, correlated, and analyzed, with the results serving a purpose such as alerting, blocking, and measuring. But what if, like they do in other sectors we can leverage that data identify potential problems and predict future ones?
Mosaic Security Research estimated that there were about 73 log management products as of November of 2014, and I suspect that number is significantly greater now, especially as the latest buzzword UBA has come on to the scene over the past two-plus years.  I am also hearing the newly apparently Gartner coined term UEBA (User and Entity Behavior Analytics) used more and more frequently.
So, why do I care and why am I bringing this up?  Because, I think we are missing the boat by not better leveraging existing technologies that have the same basic concept in mind, and are substantially more mature. As well, we are not collaborating as tightly as we could with other functions in the business with significantly more expertise in this arena.  By the way, product and service vendors need to do the same thing.
Within the enterprise the marketing group, machine analytics group, and science and research, are prime examples of teams that are experts in crunching and analyzing behavior and additional information to baseline “normal” to make decisions based on normal activity and anomalies.
Why aren’t we better leveraging their expertise and the multitude of tools they offer?  What about healthcare, aviation, and other industries that are heavy users of data analytics and anomaly detection.  Our CISO colleagues and our security vendors should be able to take advantage of existing process, approach, maybe even tools and algorithms already in use.
As more and more devices get connected to the Industrial Internet or Internet of Things (IoT), the amount of available log data is going to increase by orders of magnitude.
So Partner Up Security Practitioners, Vendors, Suppliers, and Consultants!  Let’s go take advantage and better leverage the expertise and tools already at our disposal.
I’m good with UBA but what we need is not just user related.  We need Behavior Analytics period.  We need fast, automated systems that can quickly baseline normal for everything – applications, databases, users, networks, and locations – then correlate the information to make decisions and act to block or allow.  In the worst case, the decision can’t be made in an automated fashion and must go to the slowest part of our ecosystem, the human.
If we all work together to leverage existing expertise and similar ecosystems across other industries, we can accelerate our ability to more quickly detect, respond, manage, contain, and learn.  Then if we get really smart and truly begin to share data across disciplines, we can start to make a dent in the bad guys profit and reduce the current avalanche of theft and mayhem in cyberspace.
Happy hunting!