AI integrated Rendering

[twk]

Happy New Year All,

Heres to 2023 helping us forget 2022 😄

 

The folks at https://www.evolvelab.io/ have come out with an interesting approach to the rendering game in CAD.

https://www.evolvelab.io/veras

 

Would be cool if Vectorworks could get ahead in this space, if they haven't already 😉 

 

https://forum.evolvelab.io/t/rendering-exterior-cabin-the-rac-basic-model-sample-revit-model/493

 

One basic render > multiple text prompts > generated multiple versions in an instant.

For those unfamiliar, prompts are text that are provided by the user to the AI generator, to trigger a response that the algorithm will interpret. So the text prompts are crucial in some instances.

 

Base render.

 

Prompts that adjust background images, weather, and textures on building.

Prompt: modern minimal cabin, limestone, white walls, white ceiling, white cabinets, background forest during autumn

 

Prompt: modern minimal concrete cabin, background forest during autumn

 

Prompt: modern gray wood cabin, forest during summer

 

 

Prompt: modern gray wood cabin, forest during winter, snow, blizzard, white oak flooring interior

 

Prompt: muted colors, cabin in the woods, gray sky

 

AERIAL SHOTS:

 

 

 

 

[zoomer]

Great !

 

The ChatGPT for Renderings. I love it.

[elepp]

Am also all for it! Saw something similar for Blender with Stable Diffusion.

[Christiaan]

Even better: iterative render prompts. 

 

Prompt 1: modern minimal cabin, limestone, white walls, white ceiling, white cabinets, background forest during autumn

Prompt 2: add some clouds

[Matt Overton]

Been playing with MidJourney (free trail then $10/month for coupleGPU hours) over the break which can take feed images. So I've been able to feed it both a feed image from VW and then recycle the image it produces to add new prompts. There are a few around that seem good. 

 

It really seems promising. Seems like something VW cloud could add maybe with mood boards as part of the feed to let us quickly explore look and feel with clients. 

 

I think it would need to be cloud based so all VW users are training and retraining the same model. That would mean certain keywords produce different tweak to the look. We can create styles of image output that would quickly be geared towards the work VW users do instead of the more general public tuning of the open engines. 

 

Edit to add couple of samples these are pure text prompt images

Edited January 9, 2023 by Matt Overton
Edit to add couple of samples

[Christiaan]

 

[VIRTUALENVIRONS]

That is so cool.  How fast are the changes after a text prompt?

[Matt Overton]

9 hours ago, VIRTUALENVIRONS said:

That is so cool.  How fast are the changes after a text prompt?

MidJourney and most of the ones I've seen lock render time to 1 minute of GPU time for either a full image or a sample sheet of options. You can set it longer with self hosting or higher account levels but results are very good in that time for mood work. 

[LCarm]

I have been using midjourney for feasibility studies and has worked great, this needs to go on the requests forum page!

[matteoluigi]

 

Here another video about Revit, this time with "Veras"... Also heard about Archicad having sth similar

 

 

[Matt Overton]

Fake it till we make it....

Any SDK python gurus want to point us in the right direction?

 

Can I get a script that tiggers on completion of rendering of a viewport then grabs the produced output?

 

I suspect the hooks exist as Image Effects would need to do something similar?

Indeed they'd need to handle the return as well. If we could leavage that then we are part the way to making our own. 

[matteoluigi]

3 hours ago, Matt Overton said:

Any SDK python gurus want to point us in the right direction?

😄 that's more for SDK-guys, however, you won't get far without having access to an AI-platform like Veras. Creating the platform by yourself is a little bit more difficult...

 

so, whoever wants to implement sth similar like veras needs a connection to 2th similar like veras

than he needs to know, how to code the interface...

 

doesn't sound that easy. Sounds more like a job for a big company and not for a single person...

[twk]

If you're up for following along with some video tutorials here are some good ones, as I believe Veras uses a custom built ControlNet ontop of stable diffusion:

 

 

 

 

 

[Matt Overton]

Apple has been optimising one as a command line tool on MacOS with both AS and IntelS support. 
 

there are others that allow API access in python so to me we could proof of concept at least and get it to work with VW. build our own is way beyond that but
 

As I read it if you self host you can start doing dedicated training against your own image library start producing your own style. 

[matteoluigi]

does anybody have stable diffusion running on his machine? 1.5 or 2.x?

[Jeff Prince]

This is all very sad.

I don’t understand why people want to take one of the most enjoyable parts of design documentation, make it into a commodity, and automate themselves out of a job.

 

Never mind, I just figured it out 😉

 

[matteoluigi]

4 hours ago, twk said:

If you're up for following along with some video tutorials here are some good ones, as I believe Veras uses a custom built ControlNet ontop of stable diffusion:

there's no control model view in my text2img section, why? 😉 , maybe i am using the wrong weight? or the wrong stavble diffusion version?

[twk]

6 hours ago, matteoluigi said:

there's no control model view in my text2img section, why? 😉 , maybe i am using the wrong weight? or the wrong stavble diffusion version?

I should've prefaced my post, by saying I haven't actually tried any of this 😂😂. Ever since reading and liking a few posts on twitter, my feed has been flooded with videos and tweets like the ones I posted here.

 

I should also mention, that everything is moving incredibly fast to follow whats happening. The amount of offshoot diffusion models, gpt models, locally run gpt/diffusion models, being released daily is quite mind blowing. It's like seeing a beautiful avalanche; from a distance it looks beautiful, the sheer mass of it so very difficult to comprehend, but at the back of your mind you know alot is getting destroyed in the process.

 

[matteoluigi]

To get the tabs at the top of Automatic 1111 installation, go to Settings/User Interface, and in the Quick Settings list type the following: sd_model_checkpoint, CLIP_stop_at_last_layers, sd_lora, sd_vae

ControlNet Web-Ui URL Link: https://github.com/Mikubill/sd-webui-...

ControlNet Models Link: https://huggingface.co/webui/ControlN...

I hope You Guys Like this Video There are lots of things yet to explore in stable diffusion. Do let me know in the comment section below How should I make videos regarding Stable Diffusion.

 

[Poot]

Stable Diffusion w/ controlNET is what is most useful for our work, as you have control over how close it stays to the geometry of your input image.  Midjourney is cool if you want to generate random moodboards...but takes a lot of work to become good at prompting.

Stable Diffusion requires a lot of VRAM....or you will have to pay to use an online computing service.

I've been working with it a lot for the purposes of sketch renderings, not for zaha hadid style modern architecture realistic HD 4k renders.  LOTS of opportunity here to be super creative, without stealing other peoples style, and saving a ton of time for actual design.


 

 

[BartH]

Here I will mention the three C's again.

I think AI has great possibilities for creative work. 

 

I have been using InvokeAI. It is great fun and can lead to some very useful images.

However,  keep in mind that many (if not all) of the AI Models (Checkpoints, LORAs, Safetensor models, etc) have been built using stolen IP from other creatives.

I sometimes get the watermarks from iStock, or Getty images showing up in my tests, suggesting the model was built by scaping the internet for unlicensed images.

 

Here is one example generated using Invoke AI Stable Diffusion and a Vector Art style LORA:

 

We need a way to verify that the AI models have followed these rules before we begin using AI for our professional work

Have the creators that the models are built on

A) been given Credit, for their creations,

B) given Consent for the IP to be used

C) been Compensated for their efforts.

 

Bart

 

[Jeff Prince]

Have we become so lazy that we have outsourced our regurgitation of others designs to AI too?

I spent some time this past semester as a guest juror for a senior landscape studio.

It's sad how much time these student spend learning how to prompt AI instead of being able to solve basic design problems.

I think we are doomed.

[Christiaan]

6 hours ago, Your Name Here said:

I think we are doomed.

We don't need AI for that. The construction industry is responsible for nearly 40% of CO₂ emissions historically.

[Matt Overton]

2 hours ago, Christiaan said:

We don't need AI for that. The construction industry is responsible for nearly 40% of CO₂ emissions historically.

Yep the T rex lasted millions of years as the apex we aren’t going to clock up a 100,000 years at this rate

[Christiaan]

Nothing on the trilobites, which reigned for 300 million years, twice that of the dinosaurs.

[twk]

Ahh looks like Archicad beat VW to the punch..