Technology Title
PRISMA – Performance, Readiness, and Immersive Speech Maintenance Assistant
Tech Focus Area
Workforce Development/Visualization
Abstract
Problem Statement:
Conventional aircraft inspections rely on manual, pen-and-paper documentation that is subsequently transcribed into the Integrated Maintenance Data System (IMDS). This disconnected process is time-consuming, error-prone, and creates a significant administrative burden for technicians. Furthermore, maintaining high-tempo operations increases cognitive burden, and current systems lack the ability to monitor maintainer stress, fatigue, and frustration. This leads to decreased readiness, inefficient training, and potential safety risks.
Graphic
Description of the Innovation Solution:
We propose PRISMA (Performance, Readiness, and Immersive Speech Maintenance Assistant), an intelligent, voice-forward sustainment Job Performance Aid (JPA). PRISMA unifies Cobalt’s Virtual Maintenance and Training Assistant (VMTA) and Voice-Directed Inspection Reporting System (VDIRS) with Performance and Readiness Insights from Speech and Meaning (PRISM). Utilizing context-aware Generative AI and RAG-enhanced Large Language Models (LLMs), PRISMA provides hands-free, natural-language access to Technical Orders (TOs) and 3D models. It leverages data transformation and adaptive XR scenario authoring to visually guide technicians. Simultaneously, the PRISM subsystem analyzes acoustic and linguistic cues in real time to detect affective states such as cognitive load, adapting instruction, or initiating remote Subject Matter Expert (SME) Reach-Back support as needed.
Benefits to the DOW:
PRISMA bridges the gap between qualified and experienced technicians, directly enhancing mission readiness. By streamlining workflows and enabling hands-free dictation, the system improves efficiency by 20-25% during operations and inspections and eliminates manual data entry. Furthermore, real-time affect monitoring accelerates competency development, reduces retraining costs, and mitigates error-prone actions by ensuring maintainers operate within safe cognitive limits.
Innovation Challenges:
Implementing this solution requires overcoming extreme acoustic challenges, as flight line noise frequently exceeds 85 decibels. The system must accurately process complex military terminology, acronyms, and airframe-specific jargon. Additional challenges include seamless integration with legacy maintenance database systems like IMDS and ensuring continuous, secure operation in austere environments without reliable cloud access.
Technical Maturity/ Demonstration Results:
Cobalt’s underlying speech engines operate at TRL 8-9. The VDIRS and VMTA modules currently operate at TRL 6-7, having successfully demonstrated 84% word accuracy and over 95% task accuracy in noisy hangar environments. The PRISM affect-detection subsystem is at TRL 4-5. The PRISMA system uses a lightweight, edge-computing architecture to support deployments in contested areas. Through operational field testing, the integrated PRISMA solution is positioned for rapid advancement to TRL 7+.
