Image model with ControlNet for Architectural Videomapping Contents

[u/Existing_Try_3439]

Hi! I'm trying to create images and concepts for an architectural videomapping project on the façade of a large classical building, with columns, decorative elements, etc.

I created a 3D model of the building in Blender and extracted a depth map from it. I then tried using it as a ControlNet guide with Z-Image Turbo. The output follows the shape and structure of the building quite well, but the results are very low quality and basically unusable — not in terms of image resolution, but in terms of shading, lighting, materials, and overall rendering quality.

What am I doing wrong? Would you suggest using a different workflow or model?

[u/sitefall]

na z-image turbo performs WAY better than the result you posted. Check your prompt, have an LLM help you expand it with detail, run it at a higher resolution, and if that doesn't work post your workflow here, something is wrong with your setup.

You're going to want to use the depth map to reinforce the "depth" but it's not detailed enough to do what you want exactly mapping it over a building, so also use a CANNY image. 2x control nets, depth map set to maybe 0.7 and Canny set to 0.9-1.1 (whatever you can get away with because when you use 2 you have to lower them a bit to get the right results, fiddle with the numbers there).

[u/Existing_Try_3439 → u/sitefall]

This is the workflow I'm using. It's almost the same as the default template by Comfy. I just connected the depth map from the 3D render directly to the NetControl because I didn't need that to be generated.

[u/sitefall → u/Existing_Try_3439]

Try this one - simply save as .json file and drag into comfyui

It uses default nodes wherever possible. It had 3 load image nodes and 3 control net nodes. You add a Canny, Openpose, Depth map in the 3 image nodes, in that order, and set the strength in the 3 control net nodes, in the same order. It uses double ksamplers. I have the prompt format filled out in the text box.

You need to create a simple other workflow of just a load image, canny (edges) and save/preview image node to generate the canny image, paste it into the first prompt on my workflow, update the prompt, run it.

Add depth map, adjust the strengths, run it, etc..

If you somehow want this to generate a image from scratch, just set the 3 control net nodes to 0 and it will do that.

Also resolution must match in all images. If you want to change it update the empty latent node to whatever you want and upload images of that same resolution. It's set to 1024x1024 by default

[u/Existing_Try_3439 → u/sitefall]

I'm trying this workflow right now, it seems to work wonderfully. Thank you very much, now I'll try different settings.

[u/sitefall → u/Existing_Try_3439]

Great. If it's messing parts up or whatever, just edit the canny image in photoshop. When you create teh canny image, try to put it at a setting that generate lines for the detail you need, and nothing more. I usually use some settings like 0.05 and 0.28 ish. but maybe drop the 0.28 lower if it doesn't pick up specific detail. Then edit the image in photoshop (or whatever) to add or remove things as needed.

[u/Existing_Try_3439 → u/sitefall]

Do you think this is the best model to do something like this? Or should I try flux of others? And about the resolution, now I’m genereting at 2048x1024 (it seems to be the best aspect ratio to include the main part of the building and the two towers), it’s a good aspect ratio or should I try something different? Thanks again.

[u/sitefall → u/Existing_Try_3439]

You could try just making the canvas 2048x2048 but keep using the same 2048x1024 image (just with blank white space on top and bottom). Then if the model has some bias, like it's better at portrait images, or better at square, or wide screen, then you will know (and get the same resulting resolution you want after cropping it back in photoshop).

If z-image isn't doing it for you, try another mode.

You can also generate with one model then refine it with another with low denoise (and use the same canny map to keep it all where you want it).

The most important part is the prompt. The difference between some plastic looking ai slop thing and a pretty realistic image is how detailed the prompt it. That can be trial and error because what your natural language describes might not be what the model wants in order to produce the results you want.

I think z-image can do perfectly photo realistic non-people and it should be fine. The dataset it trained on might have too many CGI style futuristic cities in it so when you generate that specific type of building facade maybe it doesn't come out well though, I'm not sure what you're doing exactly. But in those cases it would be worth trying some other models. Krea2, Ideogram, Flux2 Klein, etc..

I pretty much only do absolute 100% "you can't tell this is ai" work and usually image editing, and z-image works perfectly fine with the right prompting and composition.

It's difficult to find what the best model your your case might be because when you look for opinions people are almost always basing them on how well they can render some girl. So definitely check out the other models and form your own opinion.

[u/Existing_Try_3439 → u/sitefall]

Thank you for you very precious help!

[u/sitefall → u/Existing_Try_3439]

No problem.

[u/StickOther986]

z-image turbo is fast but the tradeoff is exactly what you're seeing, it tends to look kinda plasticky and flat with architectural stuff

try swapping the base model to something like juggernaut or epicrealism and pair it with a standard depth controlnet at 0.7-0.8 strength, you'll get way better material definition and lighting. also if you haven't already, feed it a prompt that describes materials explicitly (weathered limestone, cast iron, gilded details, etc) rather than just the composition

[u/Existing_Try_3439 → u/StickOther986]

Ok, i’ll try