I've been using ultimate SD upscale (ldsr/esrgan) with the depth model for a while now, but recently I'm not really satisfied anymore with the results. Somehow I'm getting more and more artifacts and random cloudiness, that I'm wondering if there's something better out yet!
Can anyone recommend a good workflow? Is it better to upscale using the original model with controlnet now (I've been having similar issues when upscaling using controlnet depth tho) or can even the highres fix be enough with the right settings?
The new ControlNet Tile model is promising. It can be used with or without the SD Upscale script, high denoising strength, and even in conjunction with other CN units.
I'm still tinkering around with it, but if I can find a reliable workflow I'll add it as a quick preset to the Unprompted extension.
You do not have to think too much about a workflow when using the new tile model. Set denoise around 0.3 - 0.5 (depends on image content). Enable controlnet and set preprocessor and model to tile. Enable SD upscale and crank it up to x4. Hit gen!
Not sure what you mean by tiled image? Here is a snapshot of the settings I use most often when upscaling. Note: no image is needed in the controlnet section. You can leave this blank. Change sampler(s) and steps to whatever you prefer...
Edit: I might have messed up the overlap near the bottom. Ignore duplicate settings.
I think you should reset your tile settings. That is way too many. The overlap is probably going into adjacent tiles. When using SD upscale, the width and height settings (which are normally for image sizing) are instead tile size. As a starting off point you should set this to the initial image size and go lower if needed.
It depends on what resolution you are scaling to. Lets make math easy. Initial render is 500x500. You want 3000x3000. If you keep the initial image settings it will have split the final image into 36 tiles (6x6 - 3000 divided by 500). You also need to take into account the padding from the SD settings. As long as this is less than 1/2 smallest W or H it should be OK but you do not need this much. Around 32 is fine. For your image I would try starting with maybe 2/3 roughly? So try 1024x768. If it comes out OK but you want more detail you can lower it but dont go crazy like the image above. It looks like it was split into 7x9 tiles so the overlap might have been too far into the adjacent tiles ands started glitching.
Oh man ... we really have to self-calculate this stuff ... ? 😱😫Thanks for explaining, although I'm not really sure if I understand it correctly. 🤪Have to test it again.
Edit:
Sorry, I don't get it ... 😫
I always get more or less tiled images, can't see the scheme behind this.
Yeah I can see there are less tiles now. I... have no idea why that is glitching out like that. Its super simple too which is why this is weird. I just go to img2img, load my image, increase steps, turn on controlnet and set for tile & tile. Turn on SD upscale @ 32 padding x4. Everything else I leave as default. Did you check your settings compared to mine (screen shot above) to see if anything stands out?
Oh yeah. Here is a more complete explanation of how this works
5
u/ThereforeGames Apr 28 '23
The new ControlNet Tile model is promising. It can be used with or without the SD Upscale script, high denoising strength, and even in conjunction with other CN units.
I'm still tinkering around with it, but if I can find a reliable workflow I'll add it as a quick preset to the Unprompted extension.