Posts

Digital Twin, LLM & IIOT

/in , , /by

IIOT for legacy and intelligent factory machines with XR and LLM feedback with a Digital Twin demonstration of real-time IOT for architecture/building applications using Omniverse. 

  • PIs: Sam Anand, Ming Tang.
  • Students: Anuj Gautama, Mikhail Nikolaenko, Ahmad Alrefai, Aayush Kumar, Manish Raj Aryal,c, Eian Bennett, Sourabh Deshpande 

$40,000. UC Industry 4.0/5.0 Institute Consortium Research Project: 01.2024-01.2025

The project centers on the development of a Digital Twin (DT) and a multi-agent Large Language Model (LLM) framework designed to access and interpret real-time and historical data through an Industrial Internet of Things (IIoT) platform. Real-time data is sourced from legacy machines and smart machines, integrating Building Information Modeling (BIM) with environmental sensors. The multi-agent LLM framework comprises specialized agents and supports diverse user interfaces, including screen-based systems, Virtual Reality (VR) environments, and mobile devices, enabling versatile interaction, data visualization, and analysis.

The research evaluates leading DT platforms—Autodesk Tandem, NVIDIA Omniverse, and Unreal Engine—focusing on their capabilities to integrate IoT and BIM data while supporting legacy machine systems.  Autodesk Tandem excelled in seamlessly combining BIM metadata with real-time IoT streams for building operations and system scalability.  NVIDIA Omniverse demonstrated unmatched rendering fidelity and collaborative features through its Universal Scene Description (USD) framework. Unreal Engine, notable for its immersive visualization, proved superior for LLM integration, leveraging 3D avatars and conversational AI to enhance user interaction.

Read more

CVG HOLO

/in , /by

CVG-HOLO – WAYFINDING HOLOGRAM PROJECT

XR-Lab is working with Cincinnati/Northern Kentucky International Airport (CVG), in collaboration with UC Center for Simulations & Virtual Environments Research, to

  1. Develop and demonstrate a wayfinding hologram.
  2. Evaluate the hologram signage’s performance to augment passengers’ wayfinding experience.
  3. Develop concepts of Concourse-B store renovation, integrating emerging digital technologies related to Extended Reality
  4. Develop a digital twin model of the CVG Concourse-B store area.

The project will apply various methods, including eye-tracking, motion capture, motion tracking, and computer vision.

Hologram. Reference Image from SVG news. 10.2023

Project Client: Josh Edwards, Sr. Manager, Innovation Cincinnati/Northern Kentucky International Airport

UC Team:

  • eXtended Reality Lab: Ming Tang, Director eXtended Reality Lab Digital Futures tangmg@ucmail.uc.edu
  • UCSIM Project Lead: Chris M. Collins.  Director. Center for Simulations & Virtual Environments Research
  • ARCH 7014 students. Fall. 2023

concept of hologram in CVG. by students in ARCH 7014. Fall 2023, UC. 

Thanks for the support from the UHP Discovery Summer program. 

Check out more on the student projects and eye-tracking analysis on CVG renovation.  or way-finding research projects and publications at XR-Lab. 

Kao Metaverse

/in , , /by

The University of Cincinnati, through its Digital Futures complex, will work collaboratively with the UC Center for Simulations & Virtual Environments Research, Lindner College of Business, UC DAAP XR-Lab, and Kao to develop concepts and a minimum viable product for a Jergens virtual tanning experience called the ‘Glowverse.’

KAO-STP-Glowverse VR retail project. GLOWVERSE VIRTUAL SPA (VR) development. UC-SIM + XR-Lab +College of Business. Funded by KAO.  PI: Chris Collins. Co-PI: Ming Tang, Noah Van. $52,122. Period: 2.2023-11.2023.

  • UCSIM Project Lead: Chris M. Collins, Center for Simulations & Virtual Environments
    Research
    UCSIM Technical Lead: Ryan Gorsuch, Center for Simulations & Virtual Environments
    Research
  • UC DAAP Design Lead: Ming Tang, Director of XR-Lab. Registered Architect, RA, NCARB, LEED AP (BD+C),
    and Associate Professor at the School of Architecture and Interior Design, College of Design,
    Architecture, Art, and Planning
  • UC LCOB Marketing Lead: Noah Van Bergen, Asst. Professor, Marketing, Linder Business
    Honors Faculty, Carl H. Lindner College of Business

VR Egress Training

/in , , , /by

 

Immersive vs. Traditional Training​ – a comparison of training modalities​

Industry 4.0/5.0 grant. 2023

PIs: Tamara Lorenz, Ming Tang

  • Dr. Tamara Lorenz. Associate Professor. Embodied Interactive Systems Lab, Industry 4.0 & 5.0 Institute (I45I), Center for Cognition, Action, and Perception (CAP)
  • Ming Tang. Professor. Extended Reality Lab, Industry 4.0 & 5.0 Institute (I45I), Institute for Research in Sensing (IRiS)

Consortium Research Project: evaluate the effectiveness of an immersive training protocol against different traditional training modalities. 

Grant. $40,000. By UC Industry 4.0/5.0 Institute 01.2023-01.2024

Open Questions​

  • Is immersive training equally as effective or better than traditional training? ​
  • Is immersive training beneficial for specific types of training (skill, behavior), while other modalities are better for other types (e.g. knowledge acquisition)?​
  • Does the benefit of immersive VR training warrant the initial investment in equipment and subsequent investment in project building, running, and sustenance?​

Proposal​

  • Evaluation of the effectiveness of an immersive training protocol against different traditional training modalities. ​
  • Evaluation of modality-dependent benefits for different learning goals. ​
  • Derivation of assessment metrics for VR training against other training modalities. 

Training scenario: DAAP Fire Evacuation

traditional training with slides and maps.

VR training with an immersive and interactive experience.

 

 

Thanks to the Institute’s Industrial Advisory Board (IAB) and industry patrons, including Siemens, Kinetic Vision, John Deere, Stress Engineering Services, Innovative Numberics, and Ethicon. 

Next Phase experiments

Multi-player test



 

Links

Ming Tang, Mikhail Nikolaenko, Evv Boerwinkle, Samuel Obafisoye, Aayush Kumar, Mohsen Rezayat, Sven Lehmann, and Tamara Lorenz. (2024). Evaluation of the Effectiveness of Traditional Training vs. Immersive Training. In: De Paolis, L.T., Arpaia, P., Sacco, M. (eds) Extended Reality. XR Salento 2024. Lecture Notes in Computer Science, vol 15030. Springer, Cham. https://doi.org/10.1007/978-3-031-71713-0_7

 2017 Virtual DAAP Fire Evacuation project.

 

At UC News

New UC institute looks ahead to ‘Industry 5.0’. UC will harness collective talent across campus to help companies solve new challenges. by Michael Miller.  December 8, 2022

 

 

Therapeutic Crisis Intervention Simulation. P1,P2

/in , , , , /by

VR-based Employee Safety Training. Therapeutic Crisis Intervention Simulation 

Grant:

  1. Virtual Reality for Employee Safety Training. Phase I. Sponsored research by the Cincinnati Children’s Hospital Medical Center. PI. Ming Tang. $16,631. Period: 6.2022- 09.2022.
  2. Virtual Reality for Employee Safety Training.Therapeutic Crisis Intervention Simulation-Phase II.  Sponsored research by the Cincinnati Children’s Hospital Medical Center. PI. Tang. $22,365. Period: 2.2023- 12.2023.

Under the leadership of Ming Tang, the XR-Lab is collaborating with the Cincinnati Children’s Hospital Medical Center (CCHMC) to develop a VR-based simulation to enhance employee safety training. This initiative involves creating a virtual hospital environment with interactive characters to facilitate research on diverse scenarios encountered during therapeutic crisis interventions. A vital feature of this simulation is the VR dialogue between a staff member and a teenage patient exhibiting aggressive behavior and mental illness. The primary objective is to equip staff members with the necessary skills to de-escalate tense situations effectively and adhere to appropriate protocols, thereby ensuring a safer and more controlled environment for staff and patients.

Team:

  • Ming Tang, Nancy Daraiseh, Maurizio Macaluso, Krista Keehn, Harley Davis, Aaron Vaughn, Katheryn Haller,  Joseph Staneck, Emily Oehler
  • Employee Safety Learning Lab, CCHMC
  • Extended Reality (XR) Lab, UC

Field of research: Virtual Reality, Safety Training, Therapeutic Crisis Intervention, Mental Health,  Human Behavior Simulation

screenshots from  Mobile VR Quest 2 headset.