Date of Award
2024
Document Type
Dissertation
Degree Name
Doctor of Philosophy (PhD)
Department
Computer Science
Committee Chair
Vineetha Menon
Committee Member
Huaming Zhang
Committee Member
Tathagata Mukherjee
Committee Member
Nathan L. Tenhundfeld
Committee Member
Bryan Mesmer
Research Advisor
Vineetha Menon
Subject(s)
Brain-computer interfaces, Human-computer interaction, Machine learning, Electroencephalography
Abstract
We are interested in the utility that artificially intelligent mobile systems such as drones offer to personnel in fast-paced, high-stakes situations such as disaster relief that demand real-time situational awareness. Ideally, these assistive systems place no additional cognitive or physical burden on their user; rather, they should respond to the user's intent with minimal physical or cognitive impact. Artificial intelligence (AI) and machine learning (ML) are already widely applied in both brain-computer interfaces (BCI) and drone navigation. We propose leveraging the robust computer-vision based AI that exists on modern drones to use objects as waypoints and fly a reconnaissance drone mostly autonomously, with electroencephalography (EEG) in an object recognition paradigm for selecting the drone's waypoint. In this work, our goal is to provide a proof-of-concept for the intent recognition portion of this design through a context fusion approach that allows selecting a waypoint without using existing techniques that require environmental modification or techniques such as Rapid Serial Visual Presentation (RSVP) that do not translate to kinetic situations. We outline a framework we call Human Intent-Guided Autonomous Systems (HI-GAS) as a general paradigm for this type of system-of-systems that facilitate human-AI teaming by using decision fusion between biosignal-based intent recognition and sensor-borne context awareness. We introduce the Context-Signal Decision Fusion (CSDF) model to merge EEG with imagery and conduct a 42-subject experiment to explore its viability, requirements, performance, and dynamics. In the end, we show that CSDF shows potential for implementation in the wider HI-GAS framework even with relatively low-cost, portable hardware. We evaluate the model under a variety of dynamics, identify results regarding subject-independence and architectural variation, and present mechanisms to explore the model from an explainability perspective.
Recommended Citation
de Wit, T. Warren, "Context-aware machine learning for low-burden brain-computer interfaces" (2024). Dissertations. 417.
https://louis.uah.edu/uah-dissertations/417