Date of Award


Document Type


Degree Name

Doctor of Philosophy (PhD)


Computer Science

Committee Chair

Mikel Petty

Committee Member

John Bland

Committee Member

Letha Etzkorn

Committee Member

Vineetha Menon

Committee Member

Tathagata Mukherjee


Computer networks--Security measures, Cyberterrorism--Models, Petri nets, Reinforcement learning


Cyberattacks are a growing threat to organizations of all sizes and industries. By better understanding these attacks, better ways to defend against them can be developed. Petri Nets with Players, Strategies, and Costs (PNPSC) is an extension of Petri nets specifically designed to model cyberattacks. The PNPSC formalism includes a representation of the strategies used by the competing players, i.e., attacker and defender, to achieve their goals. This formalism has been the basis for a long-running research program consisting of several interconnected research projects. Projects within that program include automatically generating PNPSC nets from the MITRE Common Attack Pattern Enumeration and Classification (CAPEC) database of cyberattack patterns, verification and validation of the models using several complementary methods, composing multiple PNPSC nets into models of realistic computer systems, and using machine learning to improve the strategies of players present in the formalism. This work describes a novel method of reinforcement learning tailored to PNPSC nets. A combination of game tree and deep reinforcement learning algorithms is used to significantly boost the learning performance of the agents used to improve player strategies in PNPSC nets. Two different deep reinforcement learning algorithms were used to improve the strategies of players present in the PNPSC formalism. These algorithms make use of function approximation that allows them to work effectively even when continuous transition rates are used. In addition to the existing component PNPSC nets, these algorithms were trained to improve strategies of players for composite models consisting of multiple component models, environments with more than one learner present, and models integrating a representation of the system user. The performance of these algorithms is also compared to existing work using Monte Carlo reinforcement learning methods. This work also includes a survey effort used to collect and validate more accurate transition rates used in the PNPSC nets.



To view the content in your browser, please download Adobe Reader or, alternately,
you may Download the file to your hard drive.

NOTE: The latest versions of Adobe Reader do not support viewing PDF files within Firefox on Mac OS and if you are using a modern (Intel) Mac, there is no official plugin for viewing PDF files within the browser window.