Existing signature-based intrusion detection systems are based on manually-defined patterns that are known to correspond to particular attacks and are therefore unable to disclose any previously unknown threats, such as zero day exploits. ÆCID (Automatic Event Correlation for Incident Detection) alleviates this problem by employing self-learning anomaly detection. ÆCID is capable of automatically learning the complex syntax of log files, classify events, and extract relevant parameters for advanced analysis. This includes the derivation of rules regarding the correlation of events as well as occurrences of parameter values. In addition, ÆCID carries out statistical analyses on the observed values and reports all significant changes of system behavior to security analysts. ÆCID’s open-source log sensor, the AMiner that enables efficient log parsing, allows to build log analysis pipelines using a number of modules. The AMiner is designed as a light-weight component that fits seamlessly into any system and has minimal requirements regarding processing power and required memory. Finally, the AMiner in combination with ÆCID supports connection to existing security solutions, such as SIEMs, by providing interfaces to standard message queue technologies, such as Kafka.

Our talk will consist of two parts: First, we will discuss some basic considerations when it comes to log data analysis and outline our strategies of tackling the encompassed challenges, including the parsing of logs from heterogeneous sources and design of anomaly detection methods. Then, we will present some selected features of ÆCID in a practical demonstration.