Date of Award
2024
Document Type
Thesis
Degree Name
Master of Science (MS)
Department
Computer Science
Committee Chair
Vineetha Menon
Committee Member
Jerome Baudry
Committee Member
Jacob Hauenstein
Research Advisor
Vineetha Menon
Subject(s)
American Sign Language--Translating, Context-aware computing, Closed captioning, Artificial intelligence
Abstract
Since the advent of Artificial Intelligence (AI), and the explosion of Genera- tive AI, much work has been done around image captioning, providing wildly accurate captions from still images, and more recently videos [3]. Alt-Text has improved dra- matically with efforts to enhance context and sentiment captured during automated captioning. This thesis attempts to expand on a section on Natural Language Pro- cessing (NLP) and its relevance to promoting accessibility and context-awareness in the premise of the American Sign Language (ASL) application. This thesis explores what it means to give full context about a caption or image to the end user, includ- ing sentiments, and sentient features existing within the string of words. This thesis goes to great lengths to expand on accessibility features currently present in image- captioning. Is a user-reliant (dependent on the users to provide a caption for the image) or a generic machine-generated alt-text enough for full accessibility? Could adding a sign language, for example, American Sign Language (ASL), be crucial to giving users more information than what is conveyed in the collection of words called a caption? Most importantly, how can we make this image-caption translation more context-aware to capture the nuances that might be important for user understand- ing? Ultimately, the goal is to make a world where every voice is heard and every individual feels welcome, and this thesis is a starting step towards an AI revolution that could be instrumental in achieving that dream.
Recommended Citation
Aluko, Oluwafolahanmi Adedamola, "Enhancing accessibility : a pilot study for context-aware image-caption to American Sign Language (ASL) translation" (2024). Theses. 720.
https://louis.uah.edu/uah-theses/720