EE Seminar: Similarity Detection via Random Subsets in Big-Data using Hadoop Framework

~~
Dafna Ackerman, 
M.Sc. student under the supervision of Prof. Amir Averbuch and Prof. Shai Avidan

Monday, July 6th, 2015 at 15:30
Room 011, Kitot Bldg., Faculty of Engineering

Similarity Detection via Random Subsets in Big-Data using Hadoop Framework
Abstract
The increasing volume of internet traffic to be analyzed imposes new challenges to anomaly detection systems. These systems should efficiently analyze a huge amount of data to discover anomaly fragments within a reasonable response time.
In this work, we propose a method for anomaly detection, such as intrusion attacks, based on a parallel similarity detection algorithm that uses the MapReduce methodology implemented on Hadoop platform. The proposed system processes large amount of data on a commodity hardware. The experimental results on the 2009 DARPA dataset demonstrate that the proposed system scales very well when data sizes increase.