Date of Award
2023
Document Type
Dissertation
Degree Name
Doctor of Philosophy (PhD)
Department
Computer Science
Committee Chair
Mikel Petty
Committee Member
John Bland
Committee Member
Letha Etzkorn
Committee Member
Vineetha Menon
Committee Member
Tathagata Mukherjee
Subject(s)
Computer networks--Security measures, Cyberterrorism--Models, Petri nets, Reinforcement learning
Abstract
Cyberattacks are a growing threat to organizations of all sizes and industries. By better understanding these attacks, better ways to defend against them can be developed. Petri Nets with Players, Strategies, and Costs (PNPSC) is an extension of Petri nets specifically designed to model cyberattacks. The PNPSC formalism includes a representation of the strategies used by the competing players, i.e., attacker and defender, to achieve their goals. This formalism has been the basis for a long-running research program consisting of several interconnected research projects. Projects within that program include automatically generating PNPSC nets from the MITRE Common Attack Pattern Enumeration and Classification (CAPEC) database of cyberattack patterns, verification and validation of the models using several complementary methods, composing multiple PNPSC nets into models of realistic computer systems, and using machine learning to improve the strategies of players present in the formalism. This work describes a novel method of reinforcement learning tailored to PNPSC nets. A combination of game tree and deep reinforcement learning algorithms is used to significantly boost the learning performance of the agents used to improve player strategies in PNPSC nets. Two different deep reinforcement learning algorithms were used to improve the strategies of players present in the PNPSC formalism. These algorithms make use of function approximation that allows them to work effectively even when continuous transition rates are used. In addition to the existing component PNPSC nets, these algorithms were trained to improve strategies of players for composite models consisting of multiple component models, environments with more than one learner present, and models integrating a representation of the system user. The performance of these algorithms is also compared to existing work using Monte Carlo reinforcement learning methods. This work also includes a survey effort used to collect and validate more accurate transition rates used in the PNPSC nets.
Recommended Citation
Bearss, Edwin Michael, "Extending machine learning of cyberattack strategies with continuous transition rates" (2023). Dissertations. 348.
https://louis.uah.edu/uah-dissertations/348