SMAT: Scalable Multi-Agent AI for DT

SMAT: Scalable Multi-Agent Machine Learning and Collaborative AI for Digital Twin Platform of Infrastructure and Facility Operations.

Principal Investigators:

  • Prof. Sam Anand, Department of Mechanical Engineering, CEAS
  • Prof. Ming Tang, Extended Reality Lab, Digital Futures, DAAP

Students: Anuj Gautam, Manish Aryal, Aayush Kumar, Ahmad Alrefai, Rohit Ramesh, Mikhail Nikolaenko, Bozhi Peng

Grant: $40,000. UC Industry 4.0/5.0 Institute Consortium Research Project: 03.2025-01.2026

Read more

paper in JMS & NAMRC

 

Anuj Gautam, Manish Raj Aryal, Sourabh Deshpande, Shailesh Padalkar, Mikhail Nikolaenko, Ming Tang, Sam Anand, IIoT-enabled digital twin for legacy and smart factory machines with LLM integration, Journal of Manufacturing Systems, Volume 80, 2025, Pages 511-523, ISSN 0278-6125

The paper is also published in the NAMRC 2025 conference.

Anuj Gautam , Manish Raj Aryal, Sourabh Deshpande, Shailesh Padalkar, Mikhail Nikolaenko, Ming Tang, Sam Anand. IIoT-enabled Digital Twin for legacy and smart factory machines with LLM integration. 53rd SME North American Manufacturing Research Conference (NAMRC), Clemson Univ. 06/2025.

 

Abstract

The recent advancement in Large Language Models (LLMs) has significantly transformed the field of natural data interpretation, translation, and user training. However, a notable gap exists when LLMs are tasked to assist with real-time context-sensitive machine data. The paper presents a multi-agent LLM framework capable of accessing and interpreting real-time and historical data through an Industrial Internet of Things (IIoT) platform for evidence-based inferences. The real-time data is acquired from several legacy machine artifacts (such as seven-segment displays, toggle switches, and knobs), smart machines (such as 3D printers), and building data (such as sound sensors and temperature measurement devices) through MTConnect data streaming protocol. Further, a multi-agent LLM framework that consists of four specialized agents – a supervisor agent, a machine-expertise agent, a data visualization agent, and a fault-diagnostic agent is developed for context-specific manufacturing tasks. This LLM framework is then integrated into a digital twin to visualize the unstructured data in real time. The paper also explores how LLM-based digital twins can serve as real time virtual experts through an avatar, minimizing reliance on traditional manuals or supervisor-based expertise. To demonstrate the functionality and effectiveness of this framework, we present a case study consisting of legacy machine artifacts and modern machines. The results highlight the practical application of LLM to assist and infer real-time machine data in a digital twin environment.

NCBDS conference

Paper “Designing the Future of Retail: Cross-Disciplinary Collaboration in Industrial Design and Architecture Design” published at the 40th National Conference on Begining Design Students Conference proceedings.  North Carolina State University. Raleigh, NC. 2025.

Yong-Gyun Ghim, Ming Tang, University of Cincinnati

 

Abstract

The significance of design’s cross-disciplinary nature has increased alongside technological advancements as emerging technologies present new opportunities and challenges for complex socio-technical systems. Systems thinking has drawn attention to design as a holistic approach to tackling complex systems by examining the interrelationships between elements. This also necessitates cross-disciplinary collaboration to address the multifaceted nature of the problems comprehensively. These aspects of systems thinking further emphasize its importance in design education to help navigate the current era of technological innovation. The future of retail exemplifies this interconnected complexity in the context of emerging technologies because introducing them – such as robotics, artificial intelligence, and mixed reality – into retail environments requires a holistic consideration of the entire system, encompassing physical spaces, service processes, and human interactions.

This study examines a 15-week collaborative studio project between industrial design and architecture. By leveraging a systems thinking approach, the project facilitated cross-disciplinary collaboration to develop future retail concepts, enabling students to integrate their expertise and address the interconnectedness of artifacts, environments, and human interactions. Both disciplines followed a structured design process encompassing research, system design, space and robot design, visualization, and validation, while collaboration was organized around four key steps: planning, learning, prototyping, and communication. The project also involved collaboration with a supermarket chain, providing opportunities for onsite observations, employee interviews, and discussions with industry professionals. Students developed futuristic concepts for retail operations and customer experiences by leveraging the integration of mobile service robots, adaptive spaces, and mixed reality. Industrial design students focused on designing a product-service system of supermarket robots based on their redefinition of customer shopping experience and employee workflow, proposing an automated grocery order fulfillment system. Architecture students designed adaptive retail spaces that seamlessly blur the boundaries between physical and digital worlds, exploring how the Metaverse and mixed-reality interfaces can augment retail spaces and shopping experiences through dynamic, immersive interactions with digital avatars and robots. This cross-disciplinary collaboration resulted in holistic and integrative solutions for complex systems, presented through immersive VR experiences or animated scenarios.

This study’s contribution to design education is threefold. First, it proposes a systems thinking approach with cross-disciplinary collaboration for designing future retail experiences, demonstrating its effectiveness in addressing and designing complex socio-technical systems. Second, it offers insights into how industrial design and architecture can be integrated to create novel user experiences in digital transformation. Lastly, by examining the design and collaboration processes and reflecting on the opportunities and challenges, this study offers insights for its application to future studio courses. Given the increased complexity and dynamics between disciplines, thorough pre-planning and flexibility are critical for success.

Keywords:

Cross-disciplinary collaboration, Design education, Industrial design, Architecture, Future of retail

Project:  Future Service, Retail, Metaverse, and Robotics

 

AI and Emerging Technology Symposium

Ming Tang and Mikhail Nikolaenko presented “AI-Powered Digital Humans for Enhanced Interaction in Extended Reality” at the AI and Emerging Technology Symposium, University of Cincinnati.

The day-long event explored topics around AI and robotic process automation; smart campus innovation; and extended reality, virtual reality, and augmented reality. More on UC News.

AI-Powered Talking Avatars for Enhanced Interaction in Extended Reality

Presenter. Ming Tang, Mikhail Nikolaenko. Feb. 20, 2025 in Tangeman University Center. 

Read more

P&G VISM

Project VISM (Virtual Interactive Simulation for Material customization)
Interactive Visualization with User-Controlled, Procedural-Based, and Physical-Based Material Customization.

PI. Ming Tang.

P&G Team: Kim Jackson, Andrew Fite, Fei Wang, Allison Roman

UC Team: Ming Tang, Aayush Kumar, Yuki Hirota

Sponsor: P&G. 12/01/2024 – 5/31/2025

Amount: $28,350


Restricted Content
To view this protected content, enter the password below: