In Horoma's Head 20

Hi everyone,

Back from holidays, feeling rested (_or not actually_), but still the brain full of new memories (_that I'll probably forget in a month if I don't write them down though_), and my camera full of new shots I'll need to sort out (_eventually.._).

This first week back at work has been quite intense already so I haven't had time to do much of the usual research I like to do routinely. My main focus for the few hours I managed to grab here and there was on AI art, because why be original right?

AI

The good thing with new technology, is that they make incredible leaps super fast.

Some months ago I was getting excited about the potential of AI text-to-image generators when playing around with Midjourney. Then got on the beta of some competitors like Dall-E and Stable Diffusion. Didn't quite get the time to experiment properly with them, but from what I saw on Twitter and other forums they seem to each have their strongsuits (except making human looking hands... this for now is out of reach it seems).

It's fun to see that for a while most people were more talking about Dall-E whereas now most of the breakthroughs (technology wise and business-model wise) are done by Stable Diffusion and the fact that it is now entirely open.

I haven't had time to try it, but you can now run Stable Diffusion locally on your own hardware (I'll be able to put my new RTX3090 to good use :D) within existing software like Photoshop or Gimp. They are also working on a standalone version with an entirely new interface that seems to be node-based, which is something that I generally enjoy using.

So what is pretty cool is that by the time I have time again to experiment with those tools (say October, fingers crossed), the tools will have caught up with what I was hoping of being able to do with them back when I started experimenting in May I think. Victory!

AI platforms

As for any new technology, new secondary platforms emerged in order to try to position themselves on emerging markets. So you can now browse and search directly for prompts to inspire yourself and/or copy on https://lexica.art or you can simply buy/sell prompts ideas on https://promptbase.com/marketplace

I can already hear the screams of half of the community on the Midjourney FB group saying this is bad and detrimental to art etc, etc. I have to admit I couldn't care less to that subtopic of AI, and am definitely fascinated with the many platforms emerging every day trying to leverage and democratize the power of text to image AI. This has only been a few months, these platforms are "logical extensions" or "adaptations" of existing ideas to AI, but I'm sure we'll see more original concepts emerge in a near future too that are only possible thanks to AI. And this is when it will start to get interesting on how this can be then integrated back into our workflow. I'm also wondering how this might later be integrated back into things like the metaverse, or web3. Fun times ahead.

Realism vs painting

It was interesting to see the latest beta version of Midjourney taking a strong turn towards realism by teaming up with Stable Diffusion. While realism is something we do strive for in archviz, having the painterly effect gone really felt odd considering Midjourney really differentiated itself by having a strong concept art looking output. Now the developers are aware of this fundamental mismatch and are currently working on finetuning that new "realistic" beta in order to better merge it with the strong artistic approach Midjourney had. The results are very promising and there are already parameters that can be appended in your prompt in order to choose the level of realism, which is absolutely amazing. An odd thing that happened with that sudden turn to realism is that the FB group of MJ has been flooded with images of half naked ladies, where some weeks ago it was more about political satire, or landscapes. While the devs of MJ definitely saw that type of use coming, they did seem to be a bit overwhelmed and let's say even disappointed to see how things were being used and are also working on how to keep their product PG-13.

AI integration in archviz workflow

Naked creatures with weird hands aside, this new shift towards realism actually opens the window of use cases for Midjourney inside archviz. And retaining the more artistic parameters while bringing additional tools (image prompts, image to image generator, etc) will ensure that Midjourney (or Stable Diffusion, or any other competitor that might emerge) can stay relevant throughout the different phases of image making. We also know that Midjourney is openly working towards text-to-3D support, meaning you could directly end up with a 3D model of whatever asset or scene you want. If they manage to pull it off, this would be another big game changer.

Now just for funnsies, here are some potential use I'd be interested in implementing over time in my own workflow :

1. 3D objects generation : as I just mentioned, the possibility of generating directly 3D model would be amazing as it would maximize the originality of assets used in scenes (Megascan is great, but the library has become so widely used it's very recognizable, like some assets from Archmodels)
2. 2D cutout generation : one of the thing that is extremely annoying is to get people cutouts doing exactly what you want in your images or videos. Only big studios have the resources to have bespoke cutouts for their images. This is unfortunately not within reach for freelancers. But with the ability to generate realistic people, or even slightly stylized people in very specific outfits, doing specific things, it will become possible to populate your images just by generating people. I can already imagine platforms that just focus on generating proper 2D cutouts doing absolutely anything we want.
3. Foreground/Background generation : in the same line of thought it will become more easy to generate backplates, or realistic foreground to composite inside your final images.
4. Image stylization : I mentioned this one already I think, but one thing I'd be interested in is to be able to run clay renders through the AI in order to get a first rendered draft with some rough textures, and nice lighting/color palette that is the signature from MJ.
5. Draft generation : This one is pretty basic and straightforward, but being able to generate images with some set elements (the project or context basically) would be really fun to explore as a tool.
6. References generation : One of the very first thing that I ended up doing after playing with MJ for a couple of weeks, is to save many of my favourite output directly into my reference folders for future use. Mainly for light and color palette ideas, but there could be other reasons I could save them, it would just depend on what topic I would be exploring with MJ.
7. Context generation : Some use cases I've seen of Dall-E and Stable Diffusion lets you expand the boundaries of an image and have the AI fill in the blank. A bit like content-aware from PS, but where it actually works :D This could be quite interesting for some case scenario where for instance your clients send you a bunch of photos and you don't have the exact framing you want and you could potentiall "expand" it to better fit what is necessary. Although I'm suspecting that the latest breakthroughs in terms of context captures (by Nvidia I think) will simply make it possible to recreate an entire 3d world from just a few photos, just like Reality Capture creates an accurate object from a large feed of high res photographs.

There's a lot of discussion going on regarding AI taking away the creativity of the artists or doing most of the heavy lifting. I haven't quite decided where I lean on on the subject, but for now I definitely see AI as a great way to maximize creativity, and if it can do more and more of the heavy lifting of what it is to make an image, then great. As I keep saying during my talks, on my YT channel or on my course, the time where you could differentiate yourself on a technical level is long gone. AI will make sure that the last barriers are completely lifted, and only creativity and originality will matter in a near future. When it happens there will be a big reshuffling of what clients are after, and what archviz artists are really valuable in that new era.

Also the democratization of the technology is still in its infancy, so we'll see the usual steps of innovation where something looks original, then very common, only to then see new uses of the technology that helps in bringing back the originality and creativity. Once we're in that phase, things will be basically back to normal, just one (huge) step further.

Back to work

In the meantime, I'm working on the final recordings and finetunings of my masterclass. This is taking forever, but we're getting there. I seriously can't wait to finally put it out there and enjoy again the online workshops that will come with it. And also be able to get back into my experimentation and discovery phase that I've been longing for for too long.

Now I hope you all managed to get some sort of break during summer and are eager to start fresh this second part of the year.

Good luck to all of you ;)

If you find this utterly boring and useless, please let me know before unsubscribing 🎀. If on the other hand you do like it, then I'm glad, and feel free to share it with your archviz pals that would maybe like it too by sending them here : https://www.horoma.net/newsletter-subscription

Cheers

Simo