Jian Tao And Team Create Digital Twin Of Disaster City

Jian Tao, Ph.D., an assistant professor in the Visualization program, continues to advance his research on digital twin technology and how it can be implemented in simulations for disaster training.

A research team led by Tao, assistant director of the Texas A&M Institute of Data Science, created an immersive virtual reality simulation for first responders. The team was awarded $1.2 million from the National Institute of Standards and Technology in 2022 to build a digital twin model through the institute’s Public Safety Innovation Accelerator program. The project was completed in 2024, but Tao and his team are aiming to build on its results in hopes of developing sensor technology for everyday use.

Disaster City, a 52-acre training facility managed by the Texas A&M Engineering Extension Service, was the center of the research where the team created a virtual environment representation. First responders are trained there for response and rescue through disaster simulations. The facility has collapsed buildings, derailed trains, hazardous material sites and cars and buses that have been wrecked.

The project aimed to provide a photorealistic virtual reality environment for first responders — like firefighters — to explore and learn from sensing and communication technologies. First responders can explore the simulations and examine factors that impact a disaster in real time such as weather changes. The digital twin also features a simulated radio tower signal strength system to model communication devices. Using mobile technology, the exact locations of first responders can be pinpointed on a map during disaster scenarios.

“With digital twins, you can create a game-like environment,” Tao said. “For this project, we wanted the user to have an immersive experience in the different training scenarios. We tried to make this platform as generic as possible so that you can switch to different scenarios.”

The team used sensors connected to the digital twin at the Disaster City site. These sensors continuously monitored environmental and weather conditions. Data gathered from the sensors and a coverage map of the area were loaded into Unreal Engine, a game engine owned by Epic Games.

“It was amazing when we realized we could build the digital twin with Unreal Engine,” Tao said. “The other portion of our project came from installing sensors, establishing an Internet of Things (IoT) network, and coordinating all the components to improve the accuracy and performance of our radio signal simulations.”

Ray Ivie, director at TEEX Testing and Innovation Center, worked on the project with Tao and said digital twin technology has enormous potential for improving first responder training and strengthening communication.

“Having precise location is valuable for establishing search-and-rescue, tracking rescue personnel and mapping areas,” Ivie said. “Having comparisons between previously mapped areas and post-disaster environments allows incident commanders to highlight routes, safe areas, utilities and potential rescue sites.”

Nicole Hatch and Alyssa Cassity, who graduated in recent years with Master of Science degrees in Visualization, were also crucial to creating the layout in Unreal Engine, Tao said. Hatch said she appreciated how quickly data was sent from long distances, taking only milliseconds.

“Getting to explore the extent of Unreal Engine’s real-time capabilities was both challenging and very satisfying, especially when it’s being used for such a good cause,” Hatch said. “It’s extremely rewarding to know I helped pioneer a system that will hopefully benefit first responders and those they’re helping in the future.”

Cassity took over for Hatch after she graduated and worked on transferring data from Disaster City. She said the project is promising for training possibilities.

“I was told by someone at TEEX how helpful it was to virtually have the ability to walk up to where a first responder was standing in real life,” Cassity said. “This gives the trainers much better information than just a 2D map.”

Tao said the next step in the project is to see if they can continue making the digital twin go beyond research, and use it in everyday functionality at Disaster City. The goal is to develop this framework for testing other sensor technologies.

“We want to showcase our ability to build digital twins, so it is very important for us to demonstrate our work and our capabilities through this project,” Tao said. “Even though this is an interdisciplinary project, we are training some of the best-suited 3D modelers and technical artists for engineering and science, and that is an exciting thing to be a part of.”

This project initiative brought together experts across several departments and partner organizations, Tao said. Co-principal investigators included Michael Fox, Ph.D., executive director of the Center for Applied Communications and Networks; Walt Magnussen, Ph.D., head of Research Development for the Center for Applied Communications and Networks; Nicholas Duffield, Ph.D., professor in the Department of Electrical and Computer Engineering and director of Texas A&M Institute of Data Science; Andre Thomas, Ph.D., former faculty in the College of Performance, Visualization and Fine Arts; and Narasimha Reddy, Ph.D., professor and head of the Department of Electrical and Computer Engineering. Project management was led by Anjuli Renold of the Center for Applied Communications and Networks, and Joan Quintana, who previously served in that role.

The collaboration extended to TEEX. Among those who played critical roles were Ivie, and Jason Moats, who previously served in that role, and Mike Avolio, technology evaluation manager. Additional support was provided by Client Arnett, training manager for Rescue Programs with the TEEX Emergency Services Training Institute. Tao said Scott Ledgerwood, program director at the National Institute of Standards and Technology, provided valuable leadership and program oversight throughout the project.