Posts

Poster in Creativity & Cognition

VR as a Creative Medium for Immersive Storytelling: An Autobiographic Design Exploration

Bozhi Peng, Heekyoung Jung, Ming Tang, and Yoshiko Burke. 2026. VR as a Creative Medium for Immersive Storytelling: An Autobiographic Design Exploration. In Proceedings of the  Creativity & Cognition 2026 conference (C&C ’26,  London. UK. 2026). Association for Computing Machinery, New York, NY, USA, 1363–1370.  https://doi.org/10.1145/3803784.3816833

This project explores virtual reality (VR) as a creative medium for expressing personal memories through space, movement, and interaction, while evoking shared experiences. Using autobiographic and research through design approaches, three VR scenes were developed through iterative prototyping and first-person inquiry to investigate immersive storytelling. Guided by the framework of immersion, presence, and interactivity, the project examines how design decisions shape immersive experience in VR through the alignment of perception, embodied movement, and responsive interaction. The work demonstrates how personal memory can function as a generative resource for experiential storytelling rather than representational narration. It also reflects on the potential and limitations of autobiographic design as a method of creative inquiry in VR and identifies future directions for engaging collective and cultural memory through audience participation.

Besides the proceeding book, the publication is also included in the ACM Digital Library. 

 

Paper: VR Training to De-escalate Patient Aggressive Behavior

Journal Paper: Virtual Reality Training to De-escalate Patient Aggressive Behavior: A Pilot Study

Daraiseh, N. M., Tang, M., Macaluso, M., Aeschbury, M., Bachtel, A., Nikolaenko, M., … Vaughn, A. (2025). Virtual Reality Training to De-escalate Patient Aggressive Behavior: A Pilot Study. International Journal of Human–Computer Interaction, 1–16. https://doi.org/10.1080/10447318.2025.2576635

Abstract
Despite intensive crisis de-escalation training, psychiatric staff continue to face high injury rates from aggressive patient interactions (APIs). New approaches are needed to enhance the application of effective strategies in managing APIs. This study explored the efficacy and feasibility of VR training for psychiatric staff in recognizing and selecting appropriate de-escalation interventions. A quasi-experimental design with psychiatric staff (N = 33) tested the effectiveness and feasibility of VR training depicting a common API interaction. Effectiveness was assessed through pre-post comparisons of the Confidence in Coping with Patient Aggression (CCPA) survey, correct answer percentages, response times, and attempt success rates. Feasibility was indicated by mean scores above ‘neutral’ on usability, presence, and learner satisfaction surveys. Results showed significant improvements in response times and confidence (p<.0001), with over 75% of participants rating the training positively. VR training is effective and feasible for enhancing de-escalation skills, offering a promising approach for psychiatric facilities.

More information on the project Therapeutic Crisis Intervention Simulation. P1,P2

Call for Papers: Architecture Journal Special Issue

Call for Papers: Architecture Journal Special Issue

Next-Generation Building Performance and Optimization

Dear Colleagues,

The Architecture, Engineering, and Construction (AEC) industry is undergoing rapid transformation driven by artificial intelligence (AI), computational design, and digital twin technologies. As the demand for high-performance, low-carbon buildings grows, research focus has shifted from isolated efficiency measures to integrated frameworks linking early-stage design with long-term operation. This Special Issue invites contributions that advance the discourse on next-generation building performance and optimization, exploring how computational intelligence, sustainable strategies, and smart systems can reshape the building lifecycle. By combining theoretical inquiry with applied research, this Special Issue seeks to illuminate how emerging technologies are redefining design workflows and operational performance, effectively bridging pre-construction simulations with post-construction realities.

The aim of this Special Issue is to advance both theoretical and practical knowledge on the ways in which emerging technologies can be embedded into the design and operation of buildings. Submissions may highlight methodological innovations, case studies of energy-efficient and low-carbon strategies, and cross-disciplinary collaborations connecting architecture, engineering, computer science, and environmental studies. By positioning building performance within the wider discourse of sustainability and cyber–physical systems, this Special Issue will provide a platform for envisioning next-generation approaches in which design intelligence and operational feedback converge to foster a more resilient and sustainable built environment. We welcome submissions that investigate, but are not limited to, the following thematic areas:

  • Integration of AI-driven generative designs for massing, orientation, and layout to improve daylighting, ventilation, thermal comfort, and carbon reduction;
  • Predictive modeling and energy simulation in early-stage design;
  • Use of AI and computer vision for occupancy analysis and behavioral insights;
  • Adaptive building automation for HVAC, lighting, and environmental control;
  • Machine learning applications for predictive maintenance and resource efficiency;
  • Digital twins and IoT sensor networks for real-time monitoring, feedback, and optimization;
  • Development of intelligent, data-driven, and responsive building systems;
  • Sustainable strategies at the urban scale;
  • Performance-based design approaches;
  • Building Information Modeling (BIM) for integrated workflows;
  • Building energy modeling for efficiency and carbon reduction.

Prof. Ming Tang
Guest Editor

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Architecture is an international peer-reviewed open access quarterly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 1200 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI’s English editing service prior to publication or during author revisions.

Keywords

  • building performance
  • energy simulation
  • digital twins
  • IoT data integration
  • green building
  • smart buildings
  • sustainable design
  • performance-based design
  • post-occupancy evaluation
  • artificial intelligence
  • data-driven prediction
  • generative design
  • computational design
  • parametric modeling
  • machine learning for built environments
  • predictive maintenance
  • real-time monitoring and control
  • human-centered design
  • climate-responsive architecture
  • net-zero energy buildings
  • resilient and adaptive design
  • lifecycle assessment (LCA)

Benefits of Publishing in a Special Issue

  • Ease of navigation: Grouping papers by topic helps scholars navigate broad scope journals more efficiently.
  • Greater discoverability: Special Issues support the reach and impact of scientific research. Articles in Special Issues are more discoverable and cited more frequently.
  • Expansion of research network: Special Issues facilitate connections among authors, fostering scientific collaborations.
  • External promotion: Articles in Special Issues are often promoted through the journal’s social media, increasing their visibility.
  • Reprint: MDPI Books provides the opportunity to republish successful Special Issues in book format, both online and in print.

Further information on MDPI’s Special Issue policies can be found here.

Published Papers

This special issue is now open for submission.

paper on AI, XR, Metaverse, Digital Twins

 

Metaverse and Digital Twins in the Age of AI and Extended Reality

Tang, Ming, Mikhail Nikolaenko, Ahmad Alrefai, and Aayush Kumar. 2025. “Metaverse and Digital Twins in the Age of AI and Extended Reality” Architecture 5, no. 2: 36. https://doi.org/10.3390/architecture5020036

 

This paper explores the evolving relationship between Digital Twins (DT) and the Metaverse, two foundational yet often conflated digital paradigms in digital architecture. While DTs function as mirrored models of real-world systems—integrating IoT, BIM, and real-time analytics to support decision-making—Metaverses are typically fictional, immersive, multi-user environments shaped by social, cultural, and speculative narratives. Through several research projects, the team investigate the divergence between DTs and Metaverses through the lens of their purpose, data structure, immersion, and interactivity, while highlighting areas of convergence driven by emerging technologies in Artificial Intelligence (AI) and Extended Reality (XR).This study aims to investigate the convergence of DTs and the Metaverse in digital architecture, examining how emerging technologies—such as AI, XR, and Large Language Models (LLMs)—are blurring their traditional boundaries. By analyzing their divergent purposes, data structures, and interactivity modes, as well as hybrid applications (e.g., data-integrated virtual environments and AI-driven collaboration), this study seeks to define the opportunities and challenges of this integration for architectural design, decision-making, and immersive user experiences. Our research spans multiple projects utilizing XR and AI to develop DT and the Metaverse. The team assess the capabilities of AI in DT environments, such as reality capture and smart building management. Concurrently, the team evaluates metaverse platforms for online collaboration and architectural education, focusing on features facilitating multi-user engagement. The paper presents evaluations of various virtual environment development pipelines, comparing traditional BIM+IoT workflows with novel approaches such as Gaussian Splatting and generative AI for content creation. The team further explores the integration of Large Language Models (LLMs) in both domains, such as virtual agents or LLM-powered Non-Player-Controlled Characters (NPC), enabling autonomous interaction and enhancing user engagement within spatial environments. Finally, the paper argues that DTs and Metaverse’s once-distinct boundaries are becoming increasingly porous. Hybrid digital spaces—such as virtual buildings with data-integrated twins and immersive, social metaverses—demonstrate this convergence. As digital environments mature, architects are uniquely positioned to shape these dual-purpose ecosystems, leveraging AI, XR, and spatial computing to fuse data-driven models with immersive and user-centered experiences.
 
Keywords:  metaverse; digital twin; extended reality; AI

The paper is features in the Architecture journal cover page.

paper in JMS & NAMRC

 

Anuj Gautam, Manish Raj Aryal, Sourabh Deshpande, Shailesh Padalkar, Mikhail Nikolaenko, Ming Tang, Sam Anand, IIoT-enabled digital twin for legacy and smart factory machines with LLM integration, Journal of Manufacturing Systems, Volume 80, 2025, Pages 511-523, ISSN 0278-6125

The paper is also published in the NAMRC 2025 conference.

Anuj Gautam , Manish Raj Aryal, Sourabh Deshpande, Shailesh Padalkar, Mikhail Nikolaenko, Ming Tang, Sam Anand. IIoT-enabled Digital Twin for legacy and smart factory machines with LLM integration. 53rd SME North American Manufacturing Research Conference (NAMRC), Clemson Univ. 06/2025.

 

Abstract

The recent advancement in Large Language Models (LLMs) has significantly transformed the field of natural data interpretation, translation, and user training. However, a notable gap exists when LLMs are tasked to assist with real-time context-sensitive machine data. The paper presents a multi-agent LLM framework capable of accessing and interpreting real-time and historical data through an Industrial Internet of Things (IIoT) platform for evidence-based inferences. The real-time data is acquired from several legacy machine artifacts (such as seven-segment displays, toggle switches, and knobs), smart machines (such as 3D printers), and building data (such as sound sensors and temperature measurement devices) through MTConnect data streaming protocol. Further, a multi-agent LLM framework that consists of four specialized agents – a supervisor agent, a machine-expertise agent, a data visualization agent, and a fault-diagnostic agent is developed for context-specific manufacturing tasks. This LLM framework is then integrated into a digital twin to visualize the unstructured data in real time. The paper also explores how LLM-based digital twins can serve as real time virtual experts through an avatar, minimizing reliance on traditional manuals or supervisor-based expertise. To demonstrate the functionality and effectiveness of this framework, we present a case study consisting of legacy machine artifacts and modern machines. The results highlight the practical application of LLM to assist and infer real-time machine data in a digital twin environment.