By Joel R.
The world has rapidly shifted from using physical documents to storing all important organizational information into digital databases. This shift has caused a new crime to develop dubbed, cyber-crime. Being recently developed, cyber-crime is now recognized as the most imminent threat to our national security. The steps implemented in combatting cyber-crime involves the use of establishing databases with known attack signatures such as, IP addresses, behaviors, and malicious code among others. The way these databases are created is through the use of big data analytics.
With big data analytics, large amounts of data are captured and stored in a data lake. A data lake and a data warehouse are both data repositories but the main difference between the two is that a data lake is filled with raw data that could be structured, semi-structured, or unstructured while a data warehouse is filled with processed data that is structured.
Norse is a company dedicated to network security and delivers live, accurate and unique attack intelligence that helps block attacks, uncover breaches, and track threats. The way Norse gathers its intelligence is through the use of honeypots and sensors. These honeypots and sensors are deployed into networks and currently emulate over 8 million machines which mimic several different types of images including Microsoft servers, Linux systems, and ATMs. Altogether, Norse can emulate over 6,000 different types of device and application images. The honeypots then track the malicious user’s IP address and send all of the data to a data lake where big data analytics could make sense out of the data. After the data is processed it is added to the Norse database which spans more than 7 petabytes of detailed attack histories.
Benefits of employing Norse appliances as part of your network security include, blocking attacks before they target your network, having access to the 7 petabytes of attack definitions at all times, thwart cloud-vectored malware, stop data exfiltration via tor and anonymous proxies, and even work on encrypted connections. The way a Norse appliance operates involves the use of hardware and software installed into your network. When the Norse appliance detects an attack, it invites the malicious attacker into a honeypot while keeping them away from your network. The honeypot mimics a network, combats the attacker, and logs all of the data at the same time. The Norse appliance could automatically create ACLs against malicious IP addresses while feeding the malicious attacker fake data they may be looking for.
Tools used by Norse to create their extensive attack definitions database included Hadoop and NoSQL which encompass a wide range of technologies and architectures to solve the big data performance issues that relational databases had in processing large volumes of data that needed to be updated frequently or continually. Essentially, Hadoop and NoSQL are just frameworks that support the processing and storage of extremely large data sets. Other software used to analyze and query the data in these frameworks included YARN, MapReduce, Spark, Hive, and Pig, to name a few.
Norse really only has two applications which include network security and real-time visibility into global cyber-attacks. However, network security encompasses many different aspects and to a business most importantly; preventing data breaches and securing your assets. There are few challenges and limitations as Norse is available in 2 different forms, a physical appliance or a virtual appliance. The virtual appliance is power hungry requiring a 2.5GHz+ six core processor, vESXi 5.0+ VMware and an E-1000 network card. Norse is also dependent on data lakes and big data analytics which data lakes have security that is less mature compared to data warehouses and experienced big data analytics experts are expensive to hire.
The use of big data analytics in database security has caused the creation of an unparalleled attacks definition database to cross-reference and thwart attacks before they infiltrate your network. Furthermore, big data analytics is still being developed for many different purposes, database security is just one flavor.
Dull, T. 2015. “Data Lake vs Data Warehouse: Key Differences.” KDnuggets. Retrieved
February 20, 2017 (http://www.kdnuggets.com/2015/09/data-lake-vs-data-warehouse-key
Martinek, L. and Stedman, C. 2014. “Big Data Analytics.” TechTarget. Retrieved February 20,
Norse Corporation. 2016. “Real-Time Visibility Into Global Cyber Attacks.” Retrieved
February 20, 2017 (http://www.norse-corp.com/)
Stephenson, P. 2015. “Norse DarkWatch.” SCMagazine. Retrieved February 20, 2017
Sukhani, S. 2014. “Network Security: How does the Norse – IPViking Live product work?”
Quora. Retrieved February 20, 2017 (https://www.quora.com/Network-Security-How-does