15 Jun 2012

Big Data: Can it solve the unfulfilled promise of network security?

0 comments Permalink Friday, June 15, 2012
We have now had nearly 20 years of vendor promises that if you buy and deploy this software or that appliance, you will be protected. And yet breaches are still occurring at an increasing rate, and organised crime relating to online systems is at an all time high. What went wrong?

The vendor’s assertions that their systems will detect, block and report all manner of threats are only partially true. No one system can do all of this with 100% accuracy. So we have continued to purchase and install a range of complementary systems under the guise of “defense in depth”, but in reality to cover the weaknesses or gaps found in our other systems. And still properly motivated individuals will probe your defences looking for, and quite often finding, a way in.

Current thinking dictates that we will block everything that doesn’t match an access policy, and then we will deploy a range of real time threat detection systems to deal with threats we are seeing in that accepted traffic. Finally we will add some sort of correlation system to consolidate and present the findings. We generally deploy this technology at the most obvious locations such as our Internet or partner links, remote access points, or around publicly accessible systems. We then need to employ a handful of skilled people to administer these systems, interpret the reports and alerts, and react to the findings.

This approach has a number of weaknesses;

  • Detection systems that find “zero day” attacks usually can’t do so until they have an updated signature.
  • The volume of information is large and accelerating
  • Single threat systems have their own error rates, which include false positives and true negatives
  • It only shows you what is happening right now, at this moment in time
  • It only shows you what you have asked it to show you, based on what the vendor can detect, collect and store.
  • It doesn’t allow you to query the data in any other way, to infer any other relationships.
  • These systems are biased to the vendors view only, which is carried through to the logs and therefore the reporting.

Most importantly, it doesn’t tell you anything about how secure you actually are. It doesn’t deliver any sort of baseline or metric that you can use for comparisons. It doesn’t tell you whether you are overly targeted. Assume for the moment the worst happens, and you are compromised. This current approach to security logging does little to assist you in determining when and where an attacker gained access, and it certainly does not help you identify what else this attacker did after gaining access. Compounding all of these problems is the increasing use of encrypted protocols to mask an attackers actions.

The team at Packetloop have worked in the IT Security industry for the past 15 years, and have seen the above problems replayed constantly with our customers. Our consultants have configured countless threat detection and SIEM systems applying everything we have learnt. We have done our fair share of security reviews and incident responses but have always been plagued by the question, “What aren’t we seeing here?”. There is always that nagging doubt that we are not seeing the entire picture, and that logs will only show us what was detected at a particular security check point in the system, and not the full conversations the attacker has with multiple systems once they gain access.

We were being challenged with these same problems by CISOs. In the face of increasing security spend, they were being challenged by their Boards who wanted to know:

  • How secure is my organization?
  • How does my organization compare as a target with similar organizations in our sector?
  • Is my organization really maintaining our compliance and regulatory obligations?
  • How effective is my organization's security spend in relation to our level of security?
  • Does my organization have the means for fully understanding the extent of an attack?

We asked ourselves what was the best form of data or logs to analyse in order to answer these questions confidently. The only answer we kept coming back to was full packet captures, taken from the most trafficked areas of the network. The packet capture gives us the exact context of the attack, and it allows us to investigate in a multitude of directions using the original data rather than a vendors logs from a specific network location. Most importantly, because Packetloop stores an exact replica of the data, we can replay it over and over again through a multitude of different threat systems, and we can test old data for the presence of newly identified threats.

At the same time as deciding that full packet captures would be our data source we were also looking at what tools we could use to find the sort of evidence we were after in such a large amount of data. All of the currently commercial tools that could offer some of what we were after required huge investments in capital for probes, collectors, consolidation servers and dedicated storage arrays for holding the data. What we needed was an engine that could process and store the huge amounts of data, but did so cost effectively, presenting the data in such a way that I didn’t need a doctorate to understand the findings.

This is where Packetloop came from. The simple premise of building the tool that we would want to use in our own security consulting business, and that could answer the questions and problems posed above. Packetloop is focused on Big Data Security Analytics, using the efficiency of the Cloud to store mass data. Packetloop aims to deliver high quality business intelligence, in an easy to understand format. Over the next couple of posts we will share some of the more detailed analysis of these problems and the thinking behind how Packetloop has been created to solve these issues.

No comments:

Post a Comment