Posts

Digital Twin, LLM & IIOT

IIOT for legacy and intelligent factory machines with AR and LLM feedback with a Digital Twin demonstration of real-time IOT for architecture/building applications using Omniverse.

  • PI: Prof. Sam Anand (Director of Smart-Manufacturing Lab, Dept. of Mechanical Engineering, CEAS)
  • co-PI:  Prof. Ming Tang (Director of XR-Lab, School of Architecture & Interior Design, DAAP)

$40,000. UC Industry 4.0/5.0 Institute Consortium Research Project: 01.2024-01.2025

Primary Objective: To develop a conversational large language modeling system that acquires data from legacy machines, digital machines, environmental data, real-time data, and historical data within an IIoT environment to create a digital twin for assisting in real-time maintenance and assistance (Application Use Case: Digital Future’s Building) 

Student: Manish Raj Aryal, Sourabh Deshpande, Mikhail Nikolaenko.

AI in Architecture

Preface

In recent years, Artificial Intelligence (AI) has made significant inroads into the creative industries, notably in fields such as architecture design. The advent of AI-driven image creation tools and advanced language models has opened new frontiers in ideation, enabling a unique synergy between human creativity and generative AI imaging. This independent study aims to delve deep into the potential impacts of AI on the discipline of architecture and its integration into the design process. Three architecture students from the University of Cincinnati, Alex, Jackman, and Nathan, have embarked on a comprehensive investigation, utilizing cutting-edge AI tools to enhance their design capabilities and challenge and redefine traditional methodologies. Throughout their journey, three students have explored various dimensions of AI applications in architecture – from conceptualization and visualization to more complex aspects like representation and style. We invite you to explore the fruits of their experiments, which demonstrate their skillful application of AI in architecture and offer a glimpse into the future of architectural design, where human ingenuity and AI collaborate to create extraordinary works.

Ming Tang

AI in architecture design

by. Alex Hamilton, Jackman Land, Nathan Rosenberger

Year in review

Through various experiments and trials of multiple proprietary software, we have some findings and processes to aid students in how they navigate the new and ever-changing world of AI. The software we have primarily focused on are Open AI’s Midjourney and Chat GPT, Prome AI, and Adobe’s AI. Our goal in this semester of study was to identify how AI affects the design process, both currently and in the future. Our findings have also helped to identify whether or not AI can close the gap in design that a college education offers.

Our Experiment

Our primary experiment dove into whether or not a college education in architecture would yield better results when writing prompts for midjourney renderings. There is an in-depth report on this experiment, here is the summary of it. Three architecture students and three other people of varying ages and education each wrote 4 prompts aiming to achieve 4 different outcomes: an abstract piece of art, an outdoor rendering, an indoor rendering, and a con concept sketch of a building. The prompts would then be loaded into midjourney and they assembled randomly and given to volunteer survey subjects to see if they could pick which 3 images in each category were developed by the architecture students.

     

Our Findings

Here is what we found from this experiment: while the complexity and length of the prompt does not seem to have a drastic effect on the final production quality, you are able to achieve a much more targeted result by understanding design. The more specific you get, the more controlled the outcome. So while Midjourney closes the gap on quality as it relates to realism, graphic quality, or stylized approaches, the final production will vary drastically if not given very specific prompts driven by key design words.

We also found that, to the untrained eye, there is little to no discernable difference in final production between architecture students and anyone else who may have access to Midjourney. We found this through surveying over 30 subjects of varying ages and backgrounds to try and identify which images were produced by the design students. Even though some images were picked slightly more than others, the answers were not consistent, and when asked how confident the subject was in their answers on a scale of 1-5, nobody answered above a 3. This goes to show that most of the subjects were taking their best guess and could not tell a difference. Lastly, when considering the complexity of the prompt, it becomes much easier to identify design students. With almost every prompt doubling the length of their untrained counterpart and a vast vocabulary for design and architectural principles, the prompts, when laid out, are easily distinguishable as to who wrote them. This further proves the point that while the final production quality remains consistent across the test of whoever writes the prompts, the design students tend to narrow the window and achieve a specific goal with their tailored prompts.

So those results beg the question: what components create a strong architectural prompt? Here is what we found to be the important factors in crafting a prompt for AI generators such as Midjourney.

2D/3D
Firstly it is important to establish whether you want a 3d rendering or a 2d drawing. This will drastically change the outcome of your prompt.

Medium/Material
After you have chosen 2d or 3d, it is important to include how you want this image to be represented. If 2d, include material like paint, pencil, canvas, paper, realistic image, etc . . .

Architectural Element
This is where you define the subject of the image. This could be a single-family home, a bridge, a painting, or any other subject of the image.

In The Style Of…
This is where you establish a known designer and include “in the style of (designer)”. This is one of the most powerful parts of the prompt that will really pinpoint the style you are aiming to achieve. Finding a designer that has the style you want to emulate is arguably the most impactful portion of the prompt.

Architectural Style
This is similar to the last prompt but refers to a design style rather than a designer themself. This could include a design principle, time period, or architectural style.

Description of Design Elements
This portion of the prompt is where you can get into the details you want included in your design, whether this is windows, structure, colors, or other design elements, this is where to include them.

Example Prompt
Create a photorealistic 3D Rendering of a bridge in the style of Zaha Hadid. This should be a suspension bridge that is primarily white and utilizes unique lighting across a beautiful river. The bridge should be futuristic and kinetic. The season is Fall, the time of day is dusk, make it look dramatic.

Our Research

The research portion of our study was comprised of testing multiple software and how they can be used in the field of design. As AI continues to develop there are more and more that are specifically for design purposes. One of these software that has caught the eyes of many designers is called Prome AI, This software is specifically designed to create renders and drawings from sketches, or vice versa, create sketches from photos or renderings. This growing platform cuts the design visualization and production phase down to a fraction of the usual time needed to create a high-quality rendering. Prome has established itself as a design-driven AI software alongside others such as Maket AI and the Artistic Inspiration Renderer (AIR) for Sketchup. These software and others continue to bleed into other steps of the design phase beyond just visualization. Maket prides itself on the ability to create floor plans from a list of room types and dimensions. AIR for Sketchup is able to take simple blocking diagrams and transform them into beautifully designed buildings. This allows the visualization step to be achieved almost anywhere in the design process.

Prome AI Experiment

Thinking back to our first year in architecture, we wanted to see if this newly developed software would have been beneficial within the design process. Replicating a project done early in our academic careers, we took an abstract painting and extruded the shapes to form a design, using the geometrical painting as guidelines. While this was certainly an abridged version, we were curious to see what AI software could do with simple extruded shapes. We asked AI to create “urban skyscrapers” for our blocking design. Prome AI features a setting in which you can control the creative freedom the software has, meaning it can stick strictly to the geometry you provide or branch out for a more concept-driven piece. Below are the two experiments we conducted. First, the abstract piece of art followed by two views of the blocking diagram and extruded shapes as a result of the painting. Using Prome AI, we imputed these simple screenshots to see the result we were able to achieve using simple line drawings as a base. Attached are the three options for each view that we felt were the strongest ranging from being as precise the geometry as possible to giving the software more creative freedom. 

Our Findings

While the renderings produced were certainly not as strong as possible, we were shocked by the quality given there was no prompt provided and or post-production editing. We have continued research with this platform and simple changes could be made that would likely result in stronger results such as specifying a material or site or adding more detail in the initial sketch; however, for this experiment, we wanted to see what the software was able to produce with as little information as possible. We can see this being used in a number of different ways whether that be to generate a concept such as shown in the experiment above and/or produce final renderings by uploading a 2D floor plan or more developed screenshots from your modeling software. We believe this to be one of the stronger AI platforms and expect it to be utilized more frequently in the future given that it is one of the few free AI-based design websites.

How Can Architecture Students Learn Through AI

Those who are familiar with AI understand how quickly and relatively accurate these programs can be when given a prompt. Most students, regardless of their major, use ChatGPT to answer test questions and write their essays. It is a quick and easy software to obtain work that would take a student hours to achieve. Design students are familiar with DALL E and Midjourney. These are AI programs that work just like ChatGPT. A prompt is given that describes the intended image and the AI will produce a quality image in that liking. Most students use these programs to produce work or an idea. However, what if these AI programs were used to better understand the design of architecture? Architecture is a vast field of design with different styles and vocabulary. First-Year students are introduced to architecture and interior design by a series of exercises of hand-drawn work. They are then given some history about architecture and eventually they get to design their first structure. An issue about this is that they don’t get a taste of the vocabulary and styles of architecture that might interest them. A solution to this is to allow and promote the usage of AI programs like ChatGPT to assist in the learning of these architectural styles and terms. We believe an early introduction to AI for students could assist and benefit their design careers. Here is an example of what the combination of ChatGPT and Midjourney could achieve for students:

Exercises like this can allow the student to understand what makes a building follow the brutalist style of architecture. Instead of giving Midjourney a basic prompt like “Create a brutalist pavilion”, the student can learn and practice different architectural style and terms and start to understand different components of a building. Here is an example of how a student can start to understand the spatial layout of a building.

Questions like these might be common sense to those with experience in architecture. However, this could be a concept that first-year architecture students never get the grasp of until later in their education career. Personally, there are things that school didn’t teach me that I later learned by being in an architecture office. They seem like simple things, yet we were never taught them for whatever reason. Allowing a student to learn architecture through AI can boost their learning and prepare them for real work outside of the classroom. It will be interesting to see how AI can influence an eager student who is interested in design.

What Does the Future Hold

As AI becomes more advanced, and especially as it is developed with specific design intent, it will change the industry more and more. As far as we can tell, AI will never replace the designer completely; however, it can aid in the process and significantly speed it up. It is difficult to say how much AI will be adopted into the professional field of work, especially at this early stage where it is changing so quickly. We strongly believe, that at least one person in every office, regardless of size, should be keeping up to date on AI news in design and what changes are happening and coming in the future. This is a tool that is far too powerful and full of potential to go unused. We understand that this topic is uncomfortable for a lot of people who feel their jobs may be at risk down the line as AI expands. We urge you to not think of it as a replacement and use it as a tool. Take this as an example: Say you are in a firm and you specialize in conceptual design and design visualization. Without the use of AI, you are able to produce 3 conceptual designs and 3 renderings in a day. Now with the use of AI, you are able to 10x that amount and create 30 conceptual designs and 30 renderings in the same amount of time. Let’s say that half of the AI-generated or aided designs do not meet the standards you need them to, you have still increased your work by a multiple of 5. As you get better at crafting prompts and using new design-based AI you may be able to get that success rate closer to 100 which means you have increased work by a multiple of 10. You, as a worker who understands how to implement AI into certain parts of your work and manipulate and massage it into a design you envision, have become multiple times more valuable to the company you work for. This technology is unavoidable, so we recommend getting on the right side of it and understanding it from the beginning because no matter how many people say AI is going to take over the world, the much more likely outcome is that it revolutionizes the way we work and the output we can achieve.

Where Should I Start

There are plenty of social media pages, blogs, and news outlets that are constantly covering developments in artificial intelligence. We recommend finding one or two that you enjoy and following them to stay up to date on the latest news, this could be 5 minutes a day glancing over new articles.

Next is to experiment with larger, more fine-tuned, and available AI resources such as Chat-GPT, Bard (google), Dalle, SIri, and Midjourney. These more developed versions of AI are easy to use and understand and are largely free to the public.

Lastly, after you have familiarized yourself with some of the capabilities of the more mainstream intelligences, you should start to research your line or work or a niche that interests you and find what AI software are being developed for that specific purpose. In our case of architectural design, we recommend you begin experimenting with some of the more novel software we spoke of such as Prome AI, Maket, or AIR (just to name a few). Don’t just test these to test them, try to actively think of if and how it could be implemented into your workflow or design process.

Alex Hamilton

University of Cincinnati, DAAP | Bachelor of Architecture | 2024

Jackman Land

University of Cincinnati, DAAP | Bachelor of Architecture | 2024

Nathan Rosenberger

University of Cincinnati. DAAP | Bachelor of Architecture | 2024

 

More information on AI and Prof. Ming Tang’s research: 

Ming Tang has conducted previous AI investigations, including Building Safety Analysis with Machine Learning, AI-driven design automation, and human-machine symbiosis for creative arts.

Digital Human

Facial motion animation experiments at XR-Lab. The digital human project explores high-fidelity digital human modeling and animation powered by ChatGPT and Open AI. 

We are also developing full body motion capture through VR tracking system and Unreal’s meta human. 

 

 

Exhibition @ Reed Gallery

Reed Gallery featuring work by DAAP faculty and staff. 02.28.2022

Human and Machine Symbiosis. Portrait drawing with A.I and Robot. by Ming Tang.

Check out the  3D tour of the virtual gallery.

This work explores how Artificial Intelligence and Robots interact with humans and form a unique symbiotic relationship in painting. The co-creation follows steps:  1. Used Machine Learning and Deep Neural Network algorithm to translate a picture of a human face into a styled image. 2. Translate the face’s representation into a parametrically controlled toolpath for the KUKA robot 3. A KUKA robot executed the art-making by holding a paintbrush.

Please check out the full paper on the art-making process here, published at the 2022  HCI conference.

Drawing process:

paper @ 2022 HCI

Ming Tang’s paper Human and Machine Symbiosis. An experiment of human and robot co-creation of calligraphy-style drawing is published at the HCI INTERNATIONAL 2022, 24th International Conference on  Human-Computer Interaction. HCI International 2022 Proceeding. Communications in Computer and Information Science, vol 1580. Springer, Cham. https://doi.org/10.1007/978-3-031-06417-3_62

Tang, M. (2022). Human and Machine Symbiosis – An Experiment of Human and Robot Co-creation of Calligraphy-Style Drawing. In: Stephanidis, C., Antona, M., Ntoa, S. (eds) HCI International 2022 Posters. HCII 2022. Communications in Computer and Information Science, vol 1580. Springer, Cham. https://doi.org/10.1007/978-3-031-06417-3_62

Artificial Intelligence (AI) and robots impact creative jobs such as art-making. There have been many AI tools assisting average users in imitating the styles of renowned painters from the past. The Convolutional neural network (CNN) and generative adversarial network (GAN) emergence as a method to “hallucinate” and create expressions of styled drawings. This paper discussed an experiment to study how AI, Automation, and Robots (AAR) will interact with humans and form a unique symbiotic relationship in art-making. Our project, called “robot painter,” established a co-creation in calligraphy-style painting with the following steps:  (1) Use CNN tools to translate a raster image into a calligraphy-style image. (2) Develop an algorithm in Grasshopper and Rhino program for the Kuka robot. This generative tool allowed the artist to translate the image into a parametrically controlled 3D toolpath for a robotic arm. (3) A KUKA robot executed the art-making by holding a paintbrush and completing the painting with customized stoke, force, and angle on a canvas.

In conclusion, the paper discussed that AAR makes human intervention and co-creation possible. The ability of A. I and robots to mimic artists’ expressions have undoubtedly achieved a convincing level and will affect art-making in the years to come.

Project detail. Robotic Art