Date of Award
2019
Document Type
Dissertation
Degree Name
Doctor of Philosophy (PhD)
Department
Industrial and Systems Engineering and Engineering Management
Committee Chair
Paul D Collopy
Committee Member
Bryan L Mesmer
Committee Member
L. Dale Thomas
Committee Member
James J. Swain
Committee Member
Zoe Szajnfarber
Subject(s)
System analysis--Standards, System design, Large scale systems
Abstract
Technical reviews in major complex defense and space projects are high-stakes events that technically assess programs costing up to billions of dollars, probing the work of hundreds of engineers. These technical reviews evolved since the 1960s into an industry-standard systems engineering process. Prior research argues that technical reviews at the subsystem level are the most technically important---conflicting with other work positing that mission-level technical reviews critically contribute to failures of government programs. Reviews are important, but research on reviews is inconsistent, fragmented, and lacks supporting data. To understand the practice of technical reviews and the alignment between practice and literature, this research was split into three related questions: (1) How do technical reviews impact subsequent project engineering and management activities? (2) How do technical reviews affect participant views and self-reported knowledge of the project? (3) How do the findings from the above questions compare with literature or extant theory? The research found, unexpectedly, that system-level reviews are fundamentally different from subsystem reviews in what they do for stakeholders: subsystem reviews better meet the technological needs of the design team, while system-level reviews better meet the system-level integration and external stakeholder needs. Stakeholders have strong preferences for the reviews that meet their needs; thus, design team members prefer subsystem reviews, and stakeholders external to the project prefer system-level reviews. This preference explains the conflicts in prior research on ``what is important about reviews'': it depends on who you ask. It also points to a problem in current ``best practice'': almost all industry standards treat subsystem reviews as ``system-level reviews but smaller.'' Ignoring the key differences of subsystem reviews is a significant misunderstanding and gap in the literature. The author of this report used a case-based qualitative research approach. Six specific cases of DoD / NASA spacecraft technical reviews were studied, covering two key dimensions of expected variability: phase of design and level of review. Project data and review observations were used to develop questions for 25 interviews of project members about technical reviews. Interview respondents were solicited to span a third study variable of participant role. Interview data were analyzed using standard qualitative analysis methods to develop common findings on how reviews impact project work and participant knowledge. A follow-on systematic literature review and analysis developed a literature database of over 300 sources on reviews. An assessment was conducted of the relevant support in the literature for the research findings.
Recommended Citation
Braunger, Robert M., "Technical reviews in complex development programs" (2019). Dissertations. 187.
https://louis.uah.edu/uah-dissertations/187