Posts Tagged ‘event data’

New data suggests that most of the industry’s data breaches could have been prevented by effective log data management, reporting and response. Consider the following from Verizon Business’ latest Data Breach Investigations Report, published just a few weeks ago:

• 86 percent of the victims had evidence of the breach in their log files, but most of them did not detect the breaches for months.
• 61 percent of the breaches were discovered not by the victim company, but by a third party outside the company.
• 96 percent of the breaches were avoidable through simple or intermediate controls.

This surprising data all points to one fundamental truth: Despite huge investments in security tools and expertise, many major businesses still don’t know when they have been breached. They are left trying to explain themselves when a breach that has been in place for months - or even years - is revealed to their top management by a third party.

Why don’t companies recognize breaches when they happen, especially when those compromises are readily evident in their log files?  Why can’t enterprises find the time to scan those logs for anomalies?  Most importantly, what can companies do to reduce the length of time between the initiation of an attack and its detection and remediation?

I see two fundamental reasons why most companies don’t have good answers to these questions. Both come up time and again from customers who come to SenSage having had problems with their old log management solutions. First, their log management solution can’t scale to meet their continuous event loading, storage and management requirements. There has been massive growth in the volumes of event data being generated in recent years - especially application-related events - so performance and scalability are becoming pressing challenges. And second, their log management solution does not support the sophisticated data analysis necessary to isolate the events that matter from the events that don’t. This is especially the case with stealthy “low and slow” attacks where you may need to analyze patterns from months or years worth of data.  Traditional log management systems with indexed searches of archives were never architected to deal with this.

If this all sounds painfully familiar, you should evaluate our unified SIEM and log management solution, purpose-built atop a clustered, columnar database. It scales from terabytes to petabytes while minimizing storage costs via patented compression algorithms and supports sophisticated query analysis through our console or your choice of business intelligence tools utilizing the industry’s only ODBC/JDBC interface to security event data.

permalink

In past blog posts I have often cited the need for a scalable event data warehousing capability to keep up with data collection and analysis requirements to address compliance and security operations. After hearing from dozens of customers about how they’re using SenSage to address their most critical security and compliance challenges, I’ve decided to focus less on event data warehousing and more on how our customers and partners are using SenSage. Towards the end of 2009, we searched for a way to net it out. In the end, it was pretty easy - Security Intelligence. This term sounds lofty at first, but once you learn how we think about it, I think you will find it very down to earth.

Of course, Security Intelligence is a variation of Business Intelligence or BI. BI solutions leverage the data management capabilities provided by data warehouses to deliver decision support information to business managers. Well, that’s exactly what Security Intelligence provides: essential decision support for security, risk management and compliance operations. Done right, Security Intelligence solutions are open, flexible, and scalable like traditional data warehouses while delivering deep security context.

Improved decision support is exactly what today’s security, risk management and compliance professionals are looking for. Detection and response to cyber-threats, regulatory compliance risks and investigating system failures all require thorough but simplified analysis of massive amounts of event data. Whether responding to an incident in real time or drilling through terabytes of related events to investigate the related context or improving a control, security professionals are asking for better decision support solutions.

As compared to Business Intelligence solutions, this is a bit of a niche play. These solutions are tailored to meet the needs of security, risk management and compliance professionals. But compared to the traditional SIEM and log management point products which are built on flat files, Oracle, or, worse, closed database management systems, Security Intelligence is a more flexible and sustainable approach.

SenSage is at the forefront of this technology, delivering Security Intelligence solutions that unify SIEM, log management and controls monitoring through a single analytics environment and data management architecture. Our customers are capturing the benefits of decision support in the security management context, leading to technology consolidation and process improvements not easily accomplished with the point products noted above.

We’ll be talking about Security Intelligence quite a bit in the coming months. Drop me a line, I’d love to hear your perspective.

permalink

These days it seems everyone wants to talk about life in “real-time”. Last week, the San Jose Mercury News, ran a piece called the “Real-Time Web, the valley’s new obsession” (http://www.mercurynews.com/business/ci_13342816). The main theme of the piece, “What’s fast enough?” is thought provoking. In particular, I loved the trendspotter debate about whether Twitter was “real-time” or “near real-time”.

My reaction to the article?  I high-fived my son at the breakfast table. You see, for those of us in the security and data warehousing world, it’s great to see a pervasive, general business debate about the value of real-time analysis — issues we’ve been dealing with for years. Particularly in security, we’ve been debating (or some may say splitting hairs) about “What’s fast enough?” for years. To us, the conversation isn’t new at all; it’s based on real technology and real needs. It is evolving however, from a focus on “real-time vs. near real-time” to one focused on “real-time and all-the-time”.

Why? Five main issues are driving this evolution:

1.  New battles. Speed and accuracy are essential when it comes to new battles such as cyber threats and monitoring core intellectual property. For instance, in markets like healthcare and finance, the cost of a missed security breach can extend to irreparable financial or reputation loss. In defense, it can mean lives are lost.

2.  New data. The key to responding to these new battles is event data — time-stamped, append-only data — and it’s the fastest growing data on the web.

3.  New requirements. Proper response to these battles starts with detecting threats from terabytes of events as they are occurring – in real-time. Proper response extends to analysis of years of collected event data. In some cases, this means sifting through hundreds of billions of records, to find fraud, criminal activity, or, simply, errors. In each case, the need for wickedly fast and complete response is essential.

4.  New technologies. No surprise, vendors are stepping up with solutions to address this large, new market. SIEM and log management firms have been at this for years. More recently, data warehouse firms are joining the fray: Teradata with the launch of their Extreme Data Appliance 1550 and Netezza with their Mantra Compliance Appliance. Set Google Alerts for event data, log management, or real-time analytics and you’ll see what I mean. The vendors are coming.

5.  New expectations. I think the Merc article nailed this one. Thanks in part to the “instant on availability” of social media, SalesForce.com, Zillow, and dozens of other SaaS apps -we’re quickly getting trained to expect results without time-out for manuals or training sessions. In short order, these same expectations will be “table stakes” for business intelligence and security applications.

Last week, I was at an industry gathering hosted by Sierra Ventures – their annual CIO Forum. A few things stood out. The first was the CIO’s peer discussion about delivering “real-time analytics” and self-service to their business users. The second was Joe Tucci’s keynote session about the next wave of IT. In his talk, he stressed the importance of speed and self-service enabled by cloud computing as being at the heart of the next wave of IT. He said the change will be bigger than anything we’ve seen, will give unprecedented power to users, and will have a huge mortality rate for those who can’t adapt.

This echoes what we’ve heard from our customers.  They’re telling us this new wave can’t hit “fast enough”. They’re in the game and deploying solutions to make this vision of “Real-time, All-the-Time” a reality.

permalink