Date of Award
2023
Document Type
Thesis
Degree Name
Master of Science (MS)
Department
Computer Science
Committee Chair
Tathagata Mukheerjee
Committee Member
Vineetha Menon
Committee Member
Sundar Christopher
Subject(s)
Data transmission systems, Cloud computing
Abstract
My dissertation focuses on developing a High-performance, Scalable, and Efficient Data Transfer Framework with Configurable and Extensible Features. The exponential growth of high-volume data poses collaboration challenges for big data providers, particularly in global-scale scientific research. The Earth’s scientific sector experiences an ongoing trend of increasing temporal and spatial data resolution, resulting in massive data archives at organizations like NASA, ESA, and JAXA. To facilitate efficient scientific study with partner agencies worldwide, data transfer solutions must handle this vast data volume across platforms at reasonable costs. The dissertation proposes a 3-phase approach (Analysis, Evaluation, and Implementation) to address the implementation of the data transfer solution. The evaluation challenge is addressed by building a system that assesses available technologies for high-volume, high-velocity cloud-based data transfer solutions and a suitable solution is finally selected. The final implementation is constructed with an orchestration layer, a catalog server, and a dashboard around this solution. As a use case, this system is tested using the European Space Agency’s Sentinel-2 data archive, which is transferred from cloud servers in mainland Europe to NASA AWS servers in the US-west-2 area.
Recommended Citation
Bommala, Uday Kumar, "Large scale data transfers in cloud-based systems" (2023). Theses. 470.
https://louis.uah.edu/uah-theses/470