%0 Journal Article %J Technology Innovation Management Review %D 2017 %T Combining Exploratory Analysis and Automated Analysis for Anomaly Detection in Real-Time Data Streams %A Ahmed Shah %A Ibrahim Abualhaol %A Mahmoud Gad %A Michael Weiss %K anomaly detection %K cybersecurity %K exploratory analysis %K real-time data streams %K visualization %X Security analysts can become overwhelmed with monitoring real-time security information that is important to help them defend their network. They also tend to focus on a limited portion of the alerts, and therefore risk missing important events and links between them. At the heart of the problem is the system that analysts use to detect, explore, and respond to cyber-attacks. Developers of security analysis systems face the challenge of developing a system that can present different sources of information at multiple levels of abstraction, while also creating a system that is intuitive to use. In this article, we examine the complementary nature of exploratory analysis and automated analysis by testing the development of a system that monitors real-time Border Gateway Protocol (BGP) traffic for anomalies that might indicate security threats. BGP is an essential component for supporting the infrastructure of the Internet; however, it is also highly vulnerable and can be hijacked by attackers to propagate spam or launch denial-of-service attacks. Some of the attack scenarios on the BGP infrastructure can be quite elaborate, and it is difficult, if not impossible, to fully automate the detection of such attacks. This article makes two contributions: i) it describes a prototype platform for computing indicators and threat alerts in real time and for visualizing the context of an alert, and ii) it discusses the interaction of exploratory analysis (visualization) and automated analysis. This article is relevant to students, security researchers, and developers who are interested in the development or use of real-time security monitoring systems. They will gain insights into the complementary aspects of automated analysis and exploratory analysis through the development of a real-time streaming system. %B Technology Innovation Management Review %I Talent First Network %C Ottawa %V 7 %P 25-31 %8 04/2017 %G eng %U http://timreview.ca/article/1068 %N 4 %1 VENUS Cybersecurity Corporation Ahmed Shah holds a BEng in Software Engineering from Lakehead University in Thunder Bay, Canada, and a MEng in Technology Innovation Management from Carleton University in Ottawa, Canada. Ahmed has experience working in a wide variety of research roles at the VENUS Cybersecurity Corporation, the Global Cybersecurity Resource, and Carleton University. %2 Carleton University Ibrahim Abualhaol is a Research Scientist at Larus Technologies and an Adjunct Professor at Carleton University in Ottawa, Canada. He holds a BSc, an MSc, and a PhD in Electrical and Computer Engineering. He is a senior member of IEEE and a Professional Engineer (P.Eng) in Ontario, Canada. His research interests include real-time big-data analytics and its application in cybersecurity and wireless communication systems. %3 VENUS Cybersecurity Corporation Mahmoud M. Gad is a Research Scientist at the VENUS Cybersecurity Corporation. He holds a PhD in Electrical and Computer Engineering from the University of Ottawa in Canada. Additionally, he holds an MSc in ECE from the University of Maryland in College Park, United States. His research interests include big-data analytics for cybersecurity, cyber-physical system risk assessment, cybercrime markets, and analysis of large-scale networks. %4 Carleton University Michael Weiss holds a faculty appointment in the Department of Systems and Computer Engineering at Carleton University in Ottawa, Canada, and he is a member of the Technology Innovation Management program. His research interests include open source, ecosystems, mashups, patterns, and social network analysis. Michael has published on the evolution of open source business, mashups, platforms, and technology entrepreneurship. %R http://doi.org/10.22215/timreview/1068