Date of Award


Document Type


Degree Name

Doctor of Philosophy (PhD)


Computer Engineering

Committee Chair

Tommy Morris

Committee Member

Mikel D. Petty

Committee Member

David J. Coe

Committee Member

Karthikeyan Lingasubramanian

Committee Member

Seong-Moo Yoo


Cyberterrorism--Prevention., Computer security., Reinforcement learning.


Cybersecurity is an increasingly important challenge for computer systems. In this work, cyberattacks were modeled using an extension of the well-known Petri net formalism. That formalism, designated Petri nets with players, strategies, and costs, models the states of the cyberattack and events during the attack as markings and transition firings in the net respectively. The formalism models the attacker and defender as competing players who may observe the marking of a subset of the net and based on the observed marking act by changing the stochastic firing rates of a subset of the transitions in order to achieve their competing goals. Rate changes by the players incur a cost. Using the formalism, nets were constructed to model three specific cyberattack patterns (cross-site scripting, spear phishing, and structured query language injection) documented in the Common Attack Pattern Enumeration and Classification database. The models were validated by a panel of cybersecurity experts in a structured face validation process. Given those validated nets, a reinforcement learning algorithm using an eGreedy policy was implemented and set to the task of learning which actions to take, i.e., which transition rates to change for the different observable markings, so as to accomplish the goals of the attacker or defender. Experiments were conducted with a dynamic (learning) attacker against a static (fixed) defender, a static attacker against a dynamic defender, and a dynamic attacker against a dynamic defender. In all cases, the reinforcement learning algorithm was able to improve its performance, in terms of achieving the player’s objective and reducing the cost of doing so, over time. These results demonstrate the potential of formally modeling cyberattacks and of applying reinforcement learning to improving cybersecurity.



To view the content in your browser, please download Adobe Reader or, alternately,
you may Download the file to your hard drive.

NOTE: The latest versions of Adobe Reader do not support viewing PDF files within Firefox on Mac OS and if you are using a modern (Intel) Mac, there is no official plugin for viewing PDF files within the browser window.