AbstractA body of formally represented knowledge is based on a conceptualization: the objects, concepts, and other entities that are assumed to exist in some area of interest and the relationships that hold among them.
Tom Gruber, Co-creator of Siri
The objective of this paper is to show how topic modeling and concept mapping can be used to conduct a literature review in a new domain. The paper makes two contributions. First, it uses topic modeling to map out the literature in the new domain. Topic modeling provides an alternative to manual clustering of articles and allows the identification of non-obvious connections between ideas expressed in a collection of articles. Second, it identifies the underlying concepts in the new domain and their relationships by creating a concept map from the extracted topics . As a case study, the paper reviews the recent literature in the intersection of managed security services and small businesses. In particular, it identifies elements of the managed security services concept as it applies to small businesses. The audience of the paper includes anyone who is exploring a new domain by reviewing the literature, and in particular, students, researchers, and members of industrial R&D projects.
Introduction
Conducting a literature review in a new domain of research presents unique challenges, as previously noted by Weiss (2016). In existing domains, researchers can use established classifications of knowledge to guide their search for and interpretation of the literature. However, in a new domain, such classifications are not available.
There is a need for tools that can assist researchers in forming a conceptualization of a new domain in a short time frame. Gruber (1995) defines a conceptualization as the “objects, concepts, and other entities that are assumed to exist in some area of interest and the relationships that hold among them”. Topic modeling (Blei et al., 2003) was developed as an alternative to manually clustering articles. It allows researchers to identify non-obvious connections between ideas expressed in a collection of articles (see, for example, the topic model created by McPhee et al. 2017 to examine the themes covered in the first ten years of the TIM Review). To add to the discourse of topic modeling, we look at concept mapping (Novak & Cañas, 2010), a tool for representing knowledge as a hierarchical structure of concepts and relationships between concepts that are created by propositions or statements.
The objective of this paper is to show how topic modeling and concept mapping can be used to conduct a literature review in a new domain. The paper makes two contributions. First, it points to topic modeling as a way to map out the literature in a new domain in terms of the underlying topics or themes. Second, it identifies key underlying concepts in the new domain, as well as charting their relationships by creating a concept map from the topic model.
To demonstrate our approach, we review the recent literature at the intersection of managed security services and small businesses, and uncovers elements of the managed security services concept as they apply to small businesses. We hope that the article will be relevant to readers of the TIM Review both for the method used to conduct the literature review, and for the content of the case study, given that security is a central concern to most small business managers as well as technology entrepreneurs.
The paper is organized as follows. Section 1 identifies the case study. Section 2 describes the method used to create the topic model and concept map using literature review. Section 3 presents the results of the literature review. This is followed by a discussion of the implications of the method and results for managers and researchers in Section 4. Section 5 concludes the paper.
Managed Security Services
Small businesses are as exposed to cyber attacks as their larger counterparts, but they lack the requisite expertise and resources to protect themselves. Within the general trend towards the outsourcing of computing infrastructure, it is an interesting phenomenon that businesses have also started to outsource security to managed security service providers (Gupta & Zhdanov, 2012; Cezar et al., 2017). The application of managed security services to small businesses has not yet received much attention in the literature. This paper aims to fill this gap and to identify the elements of the managed security services concept as it applies to small businesses.
A Managed Security Service (MSS) is a solution that protects the security of a business, which is difficult to implement for the businesses themselves (Zhao et al., 2013). Outsourcing the security of a business to an outside party leads to economies of scale and greater access to information (Jansen & Jeschke, 2018). A Managed Security Services Provider (MSSP) can offer its capabilities to multiple customers at the same time and is, therefore, more cost-effective than customers setting up their own internal security controls. Since an MSSP can aggregate security events from its customers and combine them with external threat intelligence, it also has access to more information that can improve its quality of service.
Method
We first identified candidate papers for the literature review from online databases such as Web of Science and Google Scholar. We then used topic modeling to detect latent themes within these articles. Topic modeling is a probabilistic technique for clustering documents that does not require a human to label the documents. A common topic modeling technique is Latent Dirichlet Allocation (LDA) (Blei et al., 2003). Given a corpus of documents and a number of topics, LDA produces a probability distribution that indicates the topics each document is about, as well as a probability distribution of keywords associated with each topic.
To construct the topic model, we used only paper abstracts as documents, rather than full articles. By focusing on the abstracts, we emphasize highlights of the articles as summarized by its authors. We also recommend using abstracts if the documents in the corpus vary in length. In the example of the literature on MSSs, the corpus includes papers, white papers, and theses of widely different length. . We first created topic models of different sizes, and increased the number of topics until stable topics emerge that are common among the topic models generated [the number cannot become stable as it steadily increases, only the topics, which, beyond some number of topics, tend to recur in the topic models generated]. We then sorted those [no processing on topics] topics by weight (based on the number of documents that have each topic as their first) and only included the top 90%, thus serving to eliminate topics that lack support in the corpus (Mathew et al., 2017).
To interpret the topics, we first used agglomerative clustering to identify the main topics and subtopics (Quinn et al., 2010). We then manually created a concept map from the topic keywords. This concept map presents groups of related keywords and graphically shows salient relationships between keywords. Constructing the map was an iterative process, during which time the key concepts related to the new domain were identified. Finally, alongside of the concept map, we plotted the number of articles published per year and the distribution of topics within each year to understand how the new domain is emerging and taking shape.
Results
Corpus of Articles
We first searched the Web of Science database for articles on “managed security services”. Since this search only returned 22 articles, we complemented this corpus with articles from the “grey” literature. By searching Google Scholar for articles that match the query “managed security services” AND “small companies” OR “sme”, we identified 67 additional articles that included conference papers, theses, and reports, but excluded patents, white papers (with the exception of market surveys), magazine articles, and books. Table 1 shows the composition of the corpus of articles.
Table 1. Description of the corpus.
Topic Modeling
To determine the best right number of topics, we iterated the topic model with different numbers of topics, until a set of mostly independent clusters of documents emerged. The literature also suggests that 10-12 topics are a good heuristic value (Mathew et al., 2017). We found 10 topics to be a good fit for our corpus. At this point, several stable topics had emerged, with little overlap between keywords across topics. After sorting the topics by support in the corpus, three topics were eliminated by applying the 90% cut-off. Table 2 shows the topics and their associated keywords.
Table 2. Topics and their associated keywords, sorted by weight.
Clustering the Topics
The topics were clustered by merging two topics (which then become subtopics of a new topic that is produced as the result of clustering) at a time that are similar and proceeding recursively in this manner, until only a single topic remains. (Topics eliminated during the previous step were included in the clustering process and then merged with the most closely related topic.) The dendrogram produced by clustering is shown in Figure 1. The lower the height at which two topics or groups of topics are connected, the more similar they are (Quinn et al., 2010). Labels were assigned to nodes in the dendrogram by examining keywords in the connected topics.
Figure 1. Clustering the topics into main topics and subtopics.
From this analysis of the MSS for small businesses domain, three top-level topics emerged: the security services that a small business adopts, ensuring the operations of the small business, and managing risk. The second topic can be subdivided into four topics: monitoring business operations, adoption of managed security services, tools, and trust. The third topic (managing risk) comprises two subtopics: risk from interdependency, and a managed system.
Emerging Topics over Time
Plotting the topics over time shows an overall increase in interest in the topic of managed security services, as shown in Figure 2. However, we can identify two waves of publications. The first wave peaks in 2009, the second wave in 2015. The three most popular topics in 2009, according to the articles included in our sample, were security services, monitoring business operations, and a managed system. In 2015, they were security services, adoption of security services, and risk from interdependency. From 2009 to 2015, a noticeable shift in focus occurs from operations and systems to reasons for adoption and risk considerations.
Figure 2. Emergence of the topics over time.
The dataset used to produce the topic model is available at: https://github.com/michaelweiss/managed-security-services-for-small-businesses
Concept Mapping
We created a concept map to identify the elements of the MSS concept as it applies to small businesses. We grouped the topic keywords into concepts to construct the concept map in Figure 3. For example, topic 1 contains the concepts MSS and outsourcing, and topic 4 includes the concepts business, managing operations, monitoring, and intrusion detection. Each concept in the concept map is represented by its own cluster of keywords related to the concept.
We labeled the clusters in the concept map: business, MSS, managed system, intrusion detection, outsourcing, managing operations, and risk from interdependency. We established links between concepts when they were either part of the same topic (e.g. MSS and outsourcing), or semantically related (e.g. “hosts” in the managed system concept and “infected” in the intrusion detection concept are related terms: as in a host that is infected by an intrusion).
Note that, in Figure 3 below, links between concepts are not labelled. Rather, concept maps as described by Novak and Cañas (2010) connect the concepts through propositions. These propositions cannot be obtained from the topic model directly, but it is possible to extract them from a close analysis of specific sentences in documents in the corpus that contain those concepts. It is worth adding that existing work on “ontology learning” (Wohlgenannt et al., 2012) also appears promising as a way to help automatically extract links between concepts.
Figure 3. Concept map for the elements of the managed security services concept.
Discussion
For managers, this paper highlights a crucial shift from operations to risk considerations when small businesses outsource their security . The early literature, focused almost exclusively on the benefits of outsourcing security, such as reduced cost and increased quality, for example, Broom (2009). Whereas recent literature takes a more systemic perspective in which businesses and MSSPs should be considered as agents in a complex system of interdependencies, as noted by Cezar et al. (2017).
The implications for researchers are twofold. The paper shows that topic modeling is a promising approach to review literature in a new research area, and (this introduces a new idea that goes beyond topic modeling) proposes a technique for interpreting topics that involves constructing a concept map from the keywords. Previously, the interpretation of extracted topics has relied mostly on generalizing from the keywords associated with each topic. Combining concept maps with topic models provides greater visibility into the context in which the keywords associated with the topics are embedded.
Conclusion
This paper shows how a new domain like that of MSS for small businesses can be studied to discover new insights by combining topic modeling with concept mapping. Topic modeling enables researchers to identify latent topics from the literature related to a new domain and to study how these topics have emerged over time. By creating a concept map from the topics and their associated keywords, researchers can identify a set of concepts in the new domain along with their relationships.
This paper's novel contribution is to identify and begin to articulate the process of creating a concept map from a topic model. Currently, the process has to be performed manually. However, we believe that creating concept maps from a topic model can be, at least partially, automated, and therefore suggest that future research efforts be directed towards that goal.
References
Blei, D. M., Ng, A. Y., & Jordan, M. I. 2003. Latent Dirichlet Allocation. Journal of Machine Learning Research, 3: 993-1022. https://doi.org/10.1162/jmlr.2003.3.4-5.993
Gruber, T. R. 1995. Toward principles for the design of ontologies used for knowledge sharing. International Journal of Human-Computer Studies, 43(5-6): 907-928. https://doi.org/10.1006/ijhc.1995.1081
Gupta, A., & Zhdanov, D. 2012. Growth and sustainability of managed security services networks: an economic perspective. MIS Quarterly, 36(4): 1109-1130. https://doi.org/10.2307/41703500
Jansen, C., & Jeschke, S. 2018. Mitigating risks of digitalization through managed industrial security services. AI & SOCIETY, 33(2): 163-173. https://doi.org/10.1007/s00146-018-0812-1
Mathew, G., Agrawal, A., & Menzies, T. 2017. Trends in topics at SE conferences (1993-2013). International Conference on Software Engineering Companion: 397-398. IEEE/ACM. https://doi.org/10.1109/ICSE-C.2017.52
McPhee, C., Santonen, T., Shah, A., & Nazari, A. 2017. Reflecting on 10 Years of the TIM Review. Technology Innovation Management Review, 7(7): 5-20. http://doi.org/10.22215/timreview/1087
Novak, J. D., & Cañas, A. J. 2006. The origins of the concept mapping tool and the continuing evolution of the tool. Information Visualization, 5(3): 175-184. https://doi.org/10.1057/palgrave.ivs.9500126
Quinn, K. M., Monroe, B. L., Colaresi, M., Crespin, M. H., & Radev, D. R. 2010. How to analyze political attention with minimal assumptions and costs. American Journal of Political Science, 54(1): 209-228. https://doi.org/10.1111/j.1540-5907.2009.00427.x
Weiss, M. 2016. Crowdsourcing Literature Reviews in New Domains. Technology Innovation Management Review, 6(2): 5-14. http://doi.org/10.22215/timreview/963
Wohlgenannt, G., Weichselbraun, A., Scharl, A., & Sabou, M. (2012). Dynamic integration of multiple evidence sources for ontology learning. Journal of Information and Data Management, 3(3): 243-254. https://periodicos.ufmg.br/index.php/jidm/article/view/166
Zhao, X., Xue, L., & Whinston, A. B. 2013. Managing interdependent information security risks: Cyberinsurance, managed security services, and risk pooling arrangements. Journal of Management Information Systems, 30(1): 123-152. https://doi.org/10.2753/MIS0742-1222300104
Appendix A: Papers from the Web of Science database
Beybutov, E. 2009. Managing of information security with outsource service provider. International Siberian Conference on Control and Communications: 62-66. IEEE.
Cezar, A., Cavusoglu, H., & Raghunathan, S. 2017. Sourcing information security operations: The role of risk interdependency and competitive externality in outsourcing decisions. Production and Operations Management, 26(5): 860-879.
Ducatel, G., Daniel, J., Dimitrakos, T., El-Moussa, F., Rowlingson, R., & Sajjad, A. 2016. Managed security service distribution model. International Conference on Cloud Computing and Intelligence Systems: 404-408. IEEE.
Durachman, Y., Chairunnisa, Y., Soetarno, D., Setiawan, A., & Mintarsih, F. 2017. IT Security governance evaluation with use of COBIT 5 framework: A case study on UIN Syarif Hidayatullah Library Information System. International Conference on Cyber and IT Service Management: 1-5. IEEE.
Ferro, M. 2006. Combine the best security options. Communications News, 43(5): 36-38.
Forte, D. 2009. State of the art security management. Computer Fraud & Security, 2009(10): 17-18.
Gras, D., Martinez, F., Oualid, G., & Tapia, S. 2006. Managed security services: From monitoring to response. Alcatel Telecommunications Review, (1): 53-57.
Gupta, A., & Zhdanov, D. 2012. Growth and sustainability of managed security services networks: an economic perspective. MIS Quarterly, 36(4): 1109-1130.
Hodgkins, P., & Hohmann, L. 2007. Agile program management: Lessons learned from the VeriSign Managed Security Services team. Agile 2007: 194-199. IEEE.
Hui, K.-L., Hui, W., & Yue, W. T. 2012. Information security outsourcing with system interdependency and mandatory security requirement. Journal of Management Information Systems, 29(3): 117-155.
Jajodia, S., Park, N., Serra, E., & Subrahmanian, V. S. 2016. Using temporal probabilistic logic for optimal monitoring of security events with limited resources. Journal of Computer Security, 24(6): 735–791.
Jansen, C. 2017. Stabilizing the industrial system: managed security services’ contribution to cyber-peace. IFAC-PapersOnLine, 50(1): 5155–5160.
Jansen, C., & Jeschke, S. 2018. Mitigating risks of digitalization through managed industrial security services. AI & Society, 33(2): 163–173.
Membrey, P., Chan, K. C. C., Ngo, C., Demchenko, Y., & de Laat, C. 2012. Trusted virtual infrastructure bootstrapping for on demand services. International Conference on Availability, Reliability and Security: 350-357. IEEE.
Moulton, R., & Coles, R. S. 2003. A contest to evaluate IT security services management. Computers & Security, 22(3): 204–206.
Neidenbach, N., Wolf, E., & Savii, G. 2007. Contributions to IT security in outsourcing. WSEAS International Conference on Computers: 548-551.
Ren, J., Liu, L., Zhang, D., Zhou, H., & Zhang, Q. 2016. ESI-Cloud: Extending virtual machine introspection for integrating multiple security services. International Conference on Services Computing: 804-807. IEEE.
Savii, G., Neidenbach, N., & Wolf, E. 2008. Mastering IT security issues in outsourcing. DAAM Annals & Proceedings: 1225-1226.
Ulltveit-Moe, N. 2014. A roadmap towards improving managed security services from a privacy perspective. Ethics and Information Technology, 16(3): 227-240.
Van der Walt, A. 2003. Managed Security Services – who needs it? Computer Fraud & Security: 15–17.
Wagner, A., Dubendorfer, T., Hiestand, R., Goeldi, C., & Plattner, B. 2006. A fast worm scan detection tool for VPN congestion avoidance. Detection of Intrusions and Malware & Vulnerability Assessment, Proceedings, LNCS 4064, Springer: 181-194.
Zhao, X., Xue, L., & Whinston, A. B. 2013. Managing interdependent information security risks: Cyberinsurance, managed security services, and risk pooling arrangements. Journal of Management Information Systems, 30(1): 123-152.
Appendix B: Papers from the “grey” literature
Biel, E., Gonedes, N., Peachee, D., Ramirez, E., Yang, J., & Murphy, B. 2016. The quest for reliable cyber security. Journal of Information Technology Education, Discussion Cases: 5, 12.
Broom, A. 2009. Security consolidation and optimisation: Gaining the most from your IT assets. Computer Fraud & Security, 2009(5): 15-17.
Caldwell, T. 2015. Securing small businesses – the weakest link in a supply chain? Computer Fraud & Security, 2015(9): 5-10.
Campo, M. a, Dransfeld, H., & Heuer, F. 2017. The German security market: searching for the complete peace-of-mind service. Cyber Security. Simply. Make it Happen, Springer: 59-70
Carlton, M., & Levy, Y. 2015. Expert assessment of the top platform independent cybersecurity skills for non-IT professionals. SoutheastCon: 1-6. IEEE.
Dimopoulos, V. A. 2007. Effective information assurance with risk management. PhD thesis, University of Plymouth.
Duncan, B., Whittington, M., Jaatun, M. G., & Zúñiga, A. R. R. 2017. Could the outsourcing of incident response management provide a blueprint for managing other cloud security requirements? International Workshop on Enterprise Security, Springer: 19-39.
Engström, A., & Raissi, R. 2002. Firewall configuration system. Bachelor’s Thesis, Karlstad University.
Ferrara, E. 2014. The Forrester wave: Managed Security Services: North America, Q4 2014. Forrester Research.
Firstbrook, P., Hallawell, A., Girard, J., & MacDonald, N. 2009. Magic quadrant for endpoint protection platforms. Gartner RAS Core Research Note G00208912.
Franqueira, V. N. L., Cleeff, A. van, Eck, P. van, & Wieringa, R. 2012. Securing the extended enterprise: A method for analyzing external insider threat. Strategic and Practical Approaches for Information Security Governance: Technologies and Applied Solutions: 195-222.
Furnell, S., Papadaki, M., & Thomson, K.-L. 2009. Scare tactics – A viable weapon in the security war? Computer Fraud & Security, 2009(12): 6-10.
Galvan, M. 2013. Cloud computing: Incident response and digital forensics. Master’s thesis, Utica College.
Ghose, A., & Rajan, U. 2006. The economic impact of regulatory information disclosure on information security investments, competition, and social welfare. Workshop on Economics of Information Security.
Giubilo, F., Sajjad, A., Shackleton, M., Chadwick, D. W., Fan, W., & Lemos, R. de. 2017. An architecture for privacy-preserving sharing of CTI with 3rd party analysis services. International Conference for Internet Technology and Secured Transactions (ICITST): 293-297.
Goode, S., Lin, C., Tsai, J. C., & Jiang, J. J. 2015. Rethinking the role of security in client satisfaction with Software-as-a-Service (SaaS) providers. Decision Support Systems: 70, 73–85.
Gopalakrishnan, S. 2011. Outsourcing in the service based industries: an analysis using transaction cost economics and knowledge-based view. International Journal Knowledge Management Studies, 4(4): 359-374.
Griffin, T. 2017. Strategies to prevent security breaches caused by mobile devices. PhD thesis, Walden University.
Haney, M. 2015. A method for preserving privacy in network monitoring. PhD thesis, The University of Tulsa.
Hassebroek, P. B. 2007. Institutionalized environments and information security management: Learning from Y2K. PhD thesis, Ann Arbor.
Houghton, R. 2013. The chain-link fence model: a framework for creating security procedures. PhD thesis, Utah State University.
Irshad, M. B. M. 2015. The study on awareness and adoption of cloud computing by academics in Sri Lankan universities. Annual International Research Conference on Innovative Perspective in Business, Finance and Information Management: 300-306.
Isse, J. 2010. Cloud computing and the innovation process of technology consulting. Master’s thesis, Blekinge Institute of Technology.
Johansen, M. 2007. Value of investing in information security: a metastudy initiated by norSIS. Master’s thesis, Norwegian University of Science and Technology.
Johnson, K. 2017. The training deficiency in corporate America: Training security professionals to protect sensitive information. PhD thesis, Walden University.
Kallio, P. 2001. Application Service Provisioning – current state and partnership strategies. ISTA: 155-171.
Kelley, D. 2004. Security management convergence via SIM (security information management) – A requirements perspective. Journal of Network and Systems Management, 12(1): 137-144.
Kettler, A. 2014. Ovum decision matrix: Selecting an endpoint protection solution, 2017–18. Ovum.
Klunder, A. 2011. Cloud computing as a controllable it strategy. Master’s thesis, VU University Amsterdam.
Kongnso, F. 2015. Best practices to minimize data security breaches for increased business performance. PhD thesis, Walden University.
Krikos, A. C. 2010. Disruptive technology business models in cloud computing. Master’s thesis, Massachusetts Institute of Technology.
Krone, J. J. 2018. The critical success factors of outsourcing it security services: A qualitative study. PhD thesis, Colorado Technical University.
Kuper, P. 2006. A warning to industry – fix it or lose it. IEEE Security Privacy, 4(2): 56-60.
Lutiis, P. D. 2010. Managing Home Networks security challenges security issues and countermeasures. International Conference on Intelligence in Next Generation Networks: 1-6.
McKenna, B. 2002. Managed Security Services — new economy relic or wave of the future? Computers & Security, 21(7): 613-616.
Misenheimer, K. J. 2014. Exploring information technology security requirements for academic institutions to reduce information security attacks, breaches, and threats. PhD thesis, Northcentral University.
Moon, J., Lee, C., Park, S., Kim, Y., & Chang, H. 2018. Mathematical model-based security management framework for future ICT outsourcing project. Discrete Applied Mathematics, 241: 67-77.
Nero, R. L. 2018. Risks, benefits, and perceived effectiveness of outsourcing it network security in small businesses: A multiple-case study. PhD thesis, Capella University.
Okoye, S. 2017. Strategies to minimize the effects of information security threats on business performance. PhD thesis, Walden University.
Osembe, L. 2015. A study of cloud computing technology adoption by small and medium enterprises (SMEs) in Gauteng Province. Master’s thesis, University of Kwazulu-Natal.
Pereira, L. F. X. 2011. Information systems security outsourcing key issues: A service providers’ perspective. PhD thesis, ISEG-UTL.
Price, P. D. 2002. Toward an internet service provider (ISP) centric security approach. Master’s thesis, Naval Postgraduate School Monterey.
Pumphrey, S. 2017. The effectiveness of training reinforcement on employee preparedness. PhD thesis, Northcentral University.
Rabbi, M. F. 2009. An it service taxonomy for elaborating it service catalog. Master’s thesis, Blekinge Institute of Technology.
Rahman, N. H. A., & Choo, K.-K. R. 2015. Factors influencing the adoption of cloud incident handling strategy: A preliminary study in Malaysia. Americas Conference on Information Systems.
Ray, E., & Schultz, E. 2009. Virtualization security. Annual Workshop on Cyber Security and Information Intelligence Research: Cyber Security and Information Intelligence Challenges and Strategies, 42:1-42:5. ACM.
Robinson, F. 2001. High-end IT security: no longer a distant pipe dream for smes. Computer Fraud & Security, 2001(9): 12-14.
Rosado, D. 2012. Security analysis in the migration to cloud environments. Future Internet, 4: 469-487.
Rowe, B. R. 2008. Will outsourcing it security lead to a higher social level of security? Master’s thesis, North Carolina State University.
Sabottke, C., Chen, D., Layman, L., & Dumitraş, T. 2019. How to trick the Borg: threat models against manual and automated techniques for detecting network attacks. Computers & Security, 81: 25-40.
Sample, R. 2004. Fundamental practices for security of information assets in the small to medium sized organization. PhD thesis, University of Oregon.
Samtani, S. 2018. Developing proactive cyber threat intelligence from the online hacker community: A computational design science approach. PhD thesis, The University of Arizona.
Sarkar, R., Devarakonda, M., & Tanner, A. 2008. Policy-based automation to improve solution engineering in IT services. International Conference on Services Computing, 2: 52-59. IEEE.
Senk, C. 2013. Adoption of security as a service. Journal of Internet Services and Applications, 4(11), Springer.
Simmons, H. 2015. Dell’s third horizon: the innovation imperative. Master’s thesis, OCAD University.
Slatman, H. 2016. Unboxing security analytics: towards effective data driven security operations. Master’s thesis, University of Twente.
Tankard, C. 2012. The promise of managed security services. Network Security, 2012(9): 10-15.
Tarmidi, M., Rasid, S. Z. A., Alrazi, B., & Roni, R. A. 2014. Cloud computing awareness and adoption among accounting practitioners in Malaysia. Procedia – Social and Behavioral Sciences: 164, 569-574.
Trammell, S. D. 2009. Best practices of small IT services providers in the federal arena. PhD thesis, University of Maryland University College
Tyrer, A. 2015. Can the UK cyber-security industry lead the world? Computer Fraud & Security, 2015(2): 5-7.
Van Der Walt, A. 2003. Managed Security Services — who needs it? Computer Fraud & Security, 2003(8): 15-17.
Velasco, J. M. 2017. The situation and evolution of the managed services of cybersecurity, towards 3.0 and beyond. Cyberspace: Risks and Benefits for Society, Security and Development: 153-164. Springer.
Wu, M.-S., Huang, S.-J., & Chen, L.-W. 2011. The preparedness of critical success factors of IT service management and its effect on performance. The Service Industries Journal, 31(8): 1219-1235.
Yang, Y. 2018. Literature review of information security practice survey reports. Master’s thesis, University of Jyväskylä.
Zhou, W. 2018. Application of cloud computing in telecom operators. Insight – Information, 1(1): 14-23.
Zúñiga, A. 2015. Outsourced incident management services. Master’s thesis, Aalto University.
Zúñiga, A. R. R., & Jaatun, M. G. 2015. Passing the buck: outsourcing incident response management. International Conference on Cloud Computing Technology and Science (CloudCom): 503-508. IEEE.