Agile Development of Innovative, Interactive Hazard Recognition and Mitigation Tools/Learning e-Platforms for Workers Involved in Disaster Rescue and Recovery

Workers involved in post-flood reconstruction face an increased risk of occupational exposure to respiratory and other safety hazards as well as threats to their personal security (i.e., exploitation, wage theft, and wage discrimination). Post-flood reconstruction is often handled by day laborers who are predominantly non-English speaking and who have limited access to safety training and personal protective equipment. As flooding events increase in frequency and intensity, there is a critical need to develop tools to help these workers mitigate threats to their safety and wellbeing. To address this need, the interdisciplinary team of this Small Business Innovation Research (SBIR) program have been developing and refining Pocket Ark (PA), a comprehensive e-learning platform for workers in post-flood reconstruction. Their goal in Phase II is to develop the next generation of PA’s e-learning platform to disseminate critical information about hazards to workers prior to deployment to a post-flood worksite. Project aims include updating the PA platform and conducting a high- fidelity simulated disaster response scenario to train 64 workers and evaluate the program’s efficacy. As an outcome, Phase II is expected to yield a production-ready e-learning platform that: 1) delivers quality, audience-appropriate training to workers; 2) disseminates real-time information about potential on-site hazards; 3) improves logistics between workers and coordinating organizations; and 4) provides tools to reduce wage theft and other security risks. The anticipated outcome of PA is significant in that it addresses multiple dimensions of worker safety unique to this worker population.    

Immersive Modular Preparedness Intelligent Tutoring (IMPRINT)

Instructors must deliver engaging, realistic, and immersive tabletop simulations at the conclusion of Hazardous Waste Operations and Emergency Response (HAZWOPER) recertification to support Hazardous Material (HAZMAT) worker and first responder safety and adherence to protocol in the field. This exercise often burdens instructors to deliver a paper or PowerPoint simulation that, although based on real events, fails to meaningfully engage or immerse trainees. When trainees fail to engage, they are putting themselves and others at risk by decreasing their ability to adhere to protocol when responding to HAZMAT incidents in the field. Therefore, emergency response training organizations require a cost-effective training solution that increases the realism and authenticity of tabletop simulations to better equip trainees to execute HAZWOPER safely and effectively when they are in the field.

Charles River Analytics, in partnership with The New England Consortium and Lt. Michael Kates of the Boston Fire Department, proposes to develop and evaluate an Immersive Modular Preparedness Intelligent Tutor (IMPRINT). IMPRINT aims to provide a robust, commercial, portable adaptive virtual reality (VR) solution that will be complemented by an intelligent virtual training system and development framework that actively improves trainees' ability to perform HAZWOPER procedures within a range of realistic field scenarios. IMPRINT will be an untethered intelligent tutoring system (ITS) using the Oculus Quest VR headset to provide an immersive, virtual training experience. With IMPRINT, trainees can apply complex, dangerous procedures in a safe, controlled environment through guided and immersive procedure rehearsal. In Phase II, we will produce a system that complements existing training with a production ready VR case study development suite and a library of VR case studies that replace standard paper and PowerPoint scenario-based activities used to prepare trainees for hands-on assessments and their final HAZWOPER qualification test. 

Interactive Treatments for HAZMAT Training with the PerSim Mixed Reality Patient Simulator

Although current patient simulators have demonstrated improved learning outcomes in medical training, there is a significant lack of realism. Thus, they do not effectively provoke a realistic emotional response in trainees. This significantly limits their educational value to the emergency medical service (EMS) training agencies, as does – in the case of mannequins – their cost, reliance on electricity, and lack of portability. To address these significant limitations of the current medical patient simulators, the PI developed PerSim, a commercial patient simulator product using mixed reality (MR), which includes HAZMAT signs and symptoms. However, additional research and development is needed to effectively support HAZMAT treatments. 

Commercial Need: Since being released in 4th quarter 2017, there are over 100 institutions that have purchased PerSim for $20-30K each, on average, including a $12K per year subscription fee, and are using the system for emergency medicine and HAZMAT training. Based on interviews with our customers and individuals at emergency medicine training agencies, MR-based simulated HAZMAT treatments will address a critical need in HAZMAT education. 

Preliminary Data: The PI developed PerSim, an MR-based patient simulator that can be used in person or remotely over the internet. Via the Microsoft HoloLens 2 MR display, the system projects high-resolution, realistic animations of a patient onto any surface a trainee chooses such as low-fidelity mannequin as a physical reference for haptic input during procedures. The instructor uses a handheld tablet as both a controller for the simulation and an automated assessment system to track trainee performance. The system utilizes another tablet to act as a defibrillator and a physiologic monitor to provide real-time vital sign and heart rhythm data. The system’s control interfaces and registration algorithms are patented. 

Specific Aims: This project proposes to develop and evaluate interactive, virtual autoinjectors and patches for HAZMAT treatments in the PI's innovative MR-based patient simulator, PerSim. In Specific Aim 1, the PI will work with medical artists and a HAZMAT educator co-I to create realistic treatments and integrate them with the PerSim system. The PI plans to develop autoinjector and patches textures and models based on NFPA 472 Standard for Competence of Responders to Hazardous Materials, specifically to treat poisonous gas inhalation, corrosive materials, and poisonous materials. In Specific Aim 2, the PI will evaluate the usability and realism of the HAZMAT treatments with HAZMAT instructors and students.

JiHi, An Artificial Intelligent HAZWOPER e-Trainer for Tutoring and Evaluating Emergency First Response Clinical Skill Proficiency

HAZWOPER emergency response work represents one of the most dangerous jobs in the United States (U.S.) where, in many cases, emergency medical first responders are expected to deliver immediate care to persons suffering from acute traumatic injuries and exposure to hazardous substances (e.g., chemical spills). Therefore, the HAZWOPER standard devotes very specific and detailed attention to training that represents a major departure from classical emergency medical first responder action.  

Although advanced training technologies (ATT) have emerged over the past decade, ranging from mobile to virtual reality technologies, current HAZWOPER ATT are insufficient at tutoring, debriefing, and quantifiably evaluating hands-on skill proficiency while, simultaneously, enabling both hands free to practice emergency medical skills. NIEHS and OSHA require realistic HAZWOPER training that measurably develops hands-on skill proficiency. Additionally, students who continually practice hands-on clinical skills in simulated environments and with patient simulators significantly improve their hands-on skill proficiency.  

For the NIH SBIR Phase I effort, Juxtopia proposes to build upon preliminary research results to develop an artificial intelligent (AI) Juxtopia® Intelligent HAZWOPER Instructor (JiHi) that e-evaluates Fire-Fighter EMTs and Paramedics’ clinical skill proficiency by using deep learning algorithms to auto-interpret granular data generated from Juxtopia® Imhotep Band (JiBand) armlets and e-instructing first responders by displaying multimodal andragogical data on Juxtopia® Augmented Reality (AR) Goggles.  

Juxtopia hypothesizes that JiHi, that e-trains through AR Goggles and e-evaluates through JiBands, will measurably augment instructor training and improve emergency medical personnel (e.g., Fire-Fighter EMTs’) psychomotor skill proficiency while learners practice emergency medical skills in outdoor simulated HAZMAT environments. In NIH SBIR Phase I effort, Juxtopia and the Maryland Fire Rescue Institute (MFRI) will answer the following questions: How can a JiHi deliver multi-modal tutoring of hands-on clinical skills?; How can a JiHi evaluate hands-on clinical skills?; How can a JiHi continually learn from students?; How can a JiHi e-evaluate correct or incorrect clinical steps from JiBand collected data?; How can the JiHi JiBand product be sold at an affordable price? To accomplish the proposed NIH SBIR Phase I effort and answer the aforementioned question, Juxtopia will test the technical and commercial feasibility of JiHi-JiBand at MFRI facilities.