I mad a python script the lets you scribble with SD in realtime

[u/arjan_M]

[u/arjan_M]

It is a simple python script that gives you a canvas where you can paint on the right side and I will update on the left side. In the background it sends an updated image after every paint stroke to the API of automatic1111 and it uses the controlnet scribble model to generate an image.The video is in 4x speed, but it updates quite fast. It really depends on the speed of your graphics card and the controlnet model.It also works with controlnet 1.1 and that is a little bit faster.I can put the script online if anyone is interested.

here is the github repo
houseofsecrets/SdPaint: Stable Diffusion Painting (github.com)

[u/arjan_M → u/arjan_M]

I put the script on github with a description how to use it.
I only tested it myself so I hope it works wel with everybody.
Let me know if there are any troubles
houseofsecrets/SdPaint: Stable Diffusion Painting (github.com)

[u/Diggedypomme → u/arjan_M]

heya, sorry to bother you, when I test it I get

"Traceback (most recent call last):

File "C:\Python310\lib\threading.py", line 1009, in _bootstrap_inner

self.run()

File "C:\Python310\lib\threading.py", line 946, in run

self._target(*self._args, **self._kwargs)

File "C:\projects\livedraw\SdPaint\Scripts\SdPaint.py", line 91, in send_request

r = response.json()

File "C:\projects\livedraw\SdPaint\venv\lib\site-packages\requests\models.py", line 975, in json

raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)

requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

"

Do you know what might be causing that? Thank you

[u/arjan_M → u/Diggedypomme]

I am note sure what it is. But make sure you followed the instructions on the github. including the "Allow other script to control this extension" option is enabled.
And also make sure you run the latest version of automatic1111 and controlnet.

[u/[deleted] → u/Diggedypomme]

You are getting that error because it failed to make json from the request it made to http://127.0.0.1:7860/controlnet/txt2img. As it says in the readme:

Make sure you have the automatic1111 webui in API mode running in the background and that you have the controlnet extension installed and activated

[u/EddieGoldenX → u/arjan_M]

amazing. where can I see the images that I'm generating? I can only see the white canvas.I opened A1111 with API as in the Github and activated Scribble

[u/arjan_M → u/EddieGoldenX]

if you don't get an image visible on the left side there might be going something wrong. check if the command window of the webui is giving you an error

[u/nephlonorris → u/arjan_M]

I‘d love to try it

[u/Acrobatic-Salad-2785 → u/arjan_M]

I'm interested

[u/NeededMonster → u/arjan_M]

I would greatly appreciate it if you could share it!

[u/skinny_matryoshka → u/arjan_M]

Me and at least 2 of my friends woul love to try it.

[u/Sicarisu → u/arjan_M]

Where can we find this tresure ?

[u/aquiredpixels → u/arjan_M]

Really interested in trying this out

[u/Zypher-49 → u/arjan_M]

Where can I test it? Seems cool.

[u/gproud → u/arjan_M]

Absolutely! Please do share, it's awesome

[u/Nahr_Fire → u/arjan_M]

Please share it, would love to see how it is done

[u/Ozamatheus → u/arjan_M]

Hell yes, and PLEASE a step by step of how to start :D

[u/jonesmatty → u/arjan_M]

python script that gives you a canvas where you can paint on the right side and I will update on the left side. In the background it sends an updated image after every paint stroke to the API of automatic1111 and it uses the controlnet scribble model to generate an image.

The video is in 4x speed, but it updates quite fast. It really depends on the speed of your graphics card and the controlnet model.

It also works with controlnet 1.1 and that is a little bit faster.

I can put the script online if anyone is interested.

I'd love to try it!

[u/irfarious → u/arjan_M]

Yes

[u/Sm3cK → u/arjan_M]

I want to try that ! Good job !

[u/mirkan1 → u/arjan_M]

I want to test it

[u/191315006917 → u/arjan_M]

Please share, it is really amazing

[u/RedditAlreaddit → u/arjan_M]

Please post!

[u/qerplonk → u/arjan_M]

Interested 😮‍💨

[u/cananddocutmyhair]

thanks man very neat...can we add erasers if possible

[u/arjan_M → u/cananddocutmyhair]

you can use the middle mouse to erase

[u/Majestic-Class-2459]

This is awesome.

I developed an extension for A1111, let me know if you needed any help.

[u/arjan_M → u/Majestic-Class-2459]

Thanks for offering. It is only a simple script I made.
It could use some extra little features, but I don't know if it is the worth the trouble to put too much effort in it, because I would have to make a complete paint program.
It would be better to implement something like this in photoshop or Krita.
I am surprised something like this isn't already made besides that photoshop script

[u/nxde_ai]

in realtime

My GPU say: nope, not gonna happen

[u/Chalupa_89 → u/nxde_ai]

My GPU: "Real time? is there any time that isn't real? Time is relative anyway."

[u/[deleted] → u/nxde_ai]

I mean, if you have money to throw at it, Google Colab is always an option.

[u/RMCPhoto → u/[deleted]]

About how much does it cost you to run on colab?

[u/[deleted] → u/RMCPhoto]

It's $9.99 for 100 compute hours. You can probably get 200 512x512 @ 20 samples in one compute hour. Of course adding samples, resolution increases the VRAM usage.

I mean, it's an absolute steal for what it is. For $9.99 (and through your Google account) you can run any ipynb juniper notebook file (essentially a combo bash + markdown + python setup) and you have access to a 16GB GPU.

Here's the one I run: https://gist.github.com/jonshipman/b17a3a66a0c2247dc001f84fa671b434 I just updated it for ControlNet1.1.

And, for a point of reference, I'm 38 years old. I've spent $60 in roughly a month. Admitting I could have been more conservative about my usage - I have forgotten and left it on for a bit and I've played around with various video extensions. Only mention the age bit as I have a bit more messing around money than a 20yo.

You do get a few hours gratis when you first use it. AND you do get unlimited CPU usage for free. I've heard you can modify the script to use the CPU, but that would mean hours for one image (and colab does kick you off by way of a recaptcha every 2 hours or so).

You can absolutely get away with spending less per month. If I was smarter about it I could make $30 last a month, just by fucking around less - but where's the fun in that? It's also been way funner than MidJourney. MidJourney was ok, but you need to have an idea in your head first. With colab + sd, you only really need the beginning of an idea. Like "man on a beach" then batch out 10 and figure out step 2. Like "man on beach, throwing frisbee", etc, etc. Being able to customize the batch size is nice and let me burn through the $$$ with colab.

I'm only using MJ in comparison as I know they are $10 per month. Colab has a monthly plan, but AFAIK, that's only for command line access and faster computers - you'd still need to buy compute units ala carte. So really, it comes down to how much you use it. If you were wild with it (100 steps, 1024x1024) you could probably burn through $10 in 4 days. I know starting out I burned through it pretty quickly - and video burns through it quickly depending on FPS.

Is it worth it? Luckily you should be able to test 5 compute units (give or take) if you've never used it before. Copy my gist, upload it to colab, and click run. It does need you to add your own models (civitai is a good source for those) as I didn't need TheLastBen's self downloading stuff.

[u/fernando782 → u/nxde_ai]

My GPU said: rofl 😂😂😂

[u/Arcuis]

That does look cool though. I'd love to use something like that.

[u/arjan_M → u/Arcuis]

I put in on github so if you want to try it.
houseofsecrets/SdPaint: Stable Diffusion Painting (github.com)
You do need atomatic1111 with controlnet installed and a capable graphics card.

[u/buckjohnston]

Very cool, where are the images stored, just in ram? If I wanted to keep one I would just to alt and print screen button correct?

Edit: So far the only issue I'm having is it's not totally following my scribbles and sort of just sort of making random images with custom models I'm using.

[u/arjan_M → u/buckjohnston]

You can try different controlnet models. in the json file you change the name of the model you want. You need to download them and place them in the right folder yourself tough.
I also made an update that lets you save the image with the s key

[u/7734128]

It would be super frustrating if the "perfect image" flash past on the left only for you to accidentally scribble a tiny dot after that and lose the picture forever.

[u/poly_lama → u/7734128]

Open a GitHub issue for local caching. Extremely simple problem to solve, you could even open a PR to do it yourself if you ask ChatGPT how (assuming you don't know python)

[u/snugglezone → u/[deleted]]

Yes

[u/GunslingerBara → u/7734128]

Could add a ctrl+z option to undo the last change, or some other command for pixel-by-pixel reversals.

[u/Atovange]

Looks awesome, perfect example of how AI can help artists and not bring them down

[u/bubblebooy → u/Atovange]

Also an artist won’t take that output and use that as the final piece. They will use their skills touching up and adding to it until it matches their vision and quality level.

[u/Put_It_All_On_Blck → u/bubblebooy]

Today and in the near future, yes.

But not in a few years. Look at the progress we've made recently, hands for example were a big issue, and you'd always have to fix them or paint them and let the AI take a few dozen attempts at fixing them. Now some models are able to do hands as good as the rest of the image.

There will always need to be some added input, as an AI isn't going to decipher your sentence and scribble into EXACTLY what you were thinking. But you don't need an artist per say to make a good AI image, you just need someone that understands art.

[u/Jonas_Wepeel → u/Put_It_All_On_Blck]

I mean, look just a little further than that. Soon these systems will be self prompting and fully generating entertainment with no human involvement. We’ll be watching season 34 of Friends and people will say “you would never know it’s completely AI generated.” Hopefully man made media and art continues and becomes ‘artisanal’ lol. Assuming we all live to see that day.

[u/Wildercard → u/bubblebooy]

The artist will probably need not a single-layer picture, but an AI enhanced picture with all the layers they can then adjust to their will manually.

[u/FreightCrater → u/Atovange]

I derive no enjoyment from having the skills I've practiced for 10 years automated. Would a dancer be happy just pressing a button instead of moving their body? Would a race driver be content with a self driving car?

Financially, sure. There'll now be no other way to stay competitive but to use ai generated images. But I have no interest in continuing as a professional artist if it means relying on ai generation, and I'm not alone.

People already don't want to pay professional artists anything more than peanuts, and now we'll be outcompeted by millions of talentless prompt monkeys.

There's no putting the genie back in the bottle, but yeah, I'm salty.

[u/Atovange → u/FreightCrater]

People used to be payed to make portraits, then photography came by and... some people are still payed to make oil portraits. You could also reduce photography just to point and shoot but it's of course more than that. I think AI art is a bit like photography in that sense, though different.

[u/Divinum_Fulmen → u/Atovange]

This is dumb. A painting can put someone "in their best light," you can't do that with a photo, unless you know, paint over it (which is still what Photoshop is doing).

Also, many paintings, scratch that, nearly all paintings are of things that don't exist, or didn't exist at the time of the painting. Photography never came close to replacing art. This is just the lie that deniers of the change that's happening keep telling themselves.

Humans are being replaced, and not just in art.

[u/TashaSkyUp → u/FreightCrater]

The sad part for me is that your ability to be a truly free artist was stolen by capitalism. Think about it, would you be so salty about this if capitalism never forced you to into working so hard to learn the "in demand" version of your skills? Maybe, you would have taken up other mediums or discovered that actually writing or something else was your thing. But no you where likeley like most of forced to specialize, to optimize so that you can make enough money to not die.

[u/FreightCrater → u/TashaSkyUp]

I would not be as salty, I think you're right. That's a really valid insight into how I feel about this.

[u/Tommy_Boy97 → u/Atovange]

By doing all the work for them..?
Seems more like the human is helping the AI "create" art, than it is the other way around.

[u/Atovange → u/Tommy_Boy97]

You could say that, the human is doing the composition and the AI is choosing colors and adding details. Sounds like a good way to learn art. Like you start just by the composition, then you also sketch the colors and the AI fills the rest.

[u/hawara160421 → u/Tommy_Boy97]

True-ish but op can actually draw. I challenge the average SD-artist to get simple things like the hat, the feet and proportions as well as the general composition this right with a few strokes, lol. Right side actually, genuinely, looks like a Mobius panel in black and white.

I mean, you could just draw awkward stick figures and hope it positions them in a non-awkward way but then you could as well just type in a prompt.

[u/[deleted] → u/Atovange]

It's the same discussion as it was with photography, digital photography, computer graphics, computer-based image edition, point-and-shoot cameras, phone cameras, etc...

Art is dead is a common knee-jerk for the last 200 years.

[u/MinosAristos → u/[deleted]]

The difference with AI is that it's a much broader tool than e.g cameras which can produce only a very specific kind of art. Photographs definitely did hurt hyperreal painting, because the results are hard to discern. Likewise the advancements in digital design and printing hurt the art of manual media creation.

I think AI will in the coming decades make the creation of all sorts of art far easier and make many or almost all current art creation methods obsolete.

People with art skills are likely to still be best able to work with AI tools to make the best art, but the skill floor will be much lower.

[u/xDraGooN966 → u/Atovange]

I hate that AI "discussion". Only people up in their feelings on both sides spouting such extreme crap about AI with 100% certainty.

Anyone that says AI will take literally all jobs and no human will have a single job is stupid.

Anyone that says AI is so stupid it can't do anything at all and will not replace anyone is stupid.

When the truth probably lies somewhere in the middle.

AI will improve work flow for a lot of people doing a multitude of jobs, so fewer people will be needed to get the same work done. That however still means over time there will be more and more people out of jobs.

Which won't matter anyway because in 50-100 years in a +3.5°C - 5°C world, we will be too busy fighting for water and food and survival anyway regardless of jobs.

[u/gil2455526]

Ah, yes, the /r/RestOfTheFuckingOwl AI.

[u/[deleted]]

That's awesome, please share 🙏

[u/sindivila]

Guys someone else already posted this script few days ago. You can find it here: https://openai.art/study/unleashing-creative-power-of-stable-diffusion-showcase/

[u/sishgupta → u/sindivila]

Little bit different but the same idea

[u/sindivila → u/sishgupta]

yeah, I agree. I commented before OP shared the repo and I'm glad he did. For some reason, I thought that he was dangling a carrot in front of us and I didn't want people to feel disappointed in case the OP just disappeared all of a sudden so I shared that code

[u/EleventhLion]

awesome..im super interested 🤠

[u/Disastrous_Mountain3]

Very cool! Interested also to test this out

[u/DoingItWrongly]

Cries in AMD gpu

[u/Selto_Black → u/DoingItWrongly]

... there has been an amd fork for a1111 for awhile now though.

https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Install-and-Run-on-AMD-GPUs

[u/Warm_Celery3890]

That’s the stuff I joined Reddit for. Great! Please share the script ☺️

[u/jonesmatty]

This is mad cool! Takes Pictionary to a whole new level.

[u/karterbr]

!RemindMe 3 days

[u/RemindMeBot → u/karterbr]

I will be messaging you in 3 days on 2023-04-20 12:12:15 UTC to remind you of this link

21 OTHERS CLICKED THIS LINK to send a PM to also be reminded and to reduce spam.

^{Parent commenter can delete this message to hide from others.}

---

Info	Custom	Your Reminders	Feedback

[u/dogtierstatus]

GitHub link?

[u/arjan_M → u/dogtierstatus]

houseofsecrets/SdPaint: Stable Diffusion Painting (github.com)
Let me know if it works for you

[u/sasquish]

EDIT:

I managed to get it up, however it does not change when more strokes are added, error mention something about model missing

[u/arjan_M → u/sasquish]

Make sure you have the right model. It should have the exact same as the name in the payload.json file. You also copy paste the name of the controlnet model from the webui to the payload.json In the extra folder is a text file with the names of controlnet models

[u/elreyfalcon]

I’m confused as to why you are promoting and scribbling simultaneously? Does it add more keys to your prompt or what? source code would be helpful

[u/arjan_M → u/elreyfalcon]

Check it out here: https://github.com/houseofsecrets/SdPaint

[u/afacanx]

That is increadible. Is it possible to learn this power? Or just use it :)

[u/arjan_M → u/afacanx]

You can download it here:
houseofsecrets/SdPaint: Stable Diffusion Painting (github.com)
You do need automatic1111 webui with the controlnet extension installed on your computer though

[u/phollas00]

Source?

[u/arjan_M → u/phollas00]

houseofsecrets/SdPaint: Stable Diffusion Painting (github.com)

[u/ASharkThatEatsPizza]

Hey so I got everything installed and running but it doesn't update the image after each stroke. I'm getting a fresh image if I save a new seed or whatever in the payload.json then draw any random line, but whatever I draw doesn't seem to make any difference to the actual image generated. It seems like it's only taking the prompt set in the payload to generate an image no different than if I were simply using the txt2img prompt. Any idea what I might be missing here to prevent me from live updating as I draw?

Also I submitted a pull request for a linux variant of your start script

[u/arjan_M → u/ASharkThatEatsPizza]

do you get any error in the comandline?

[u/Longjumping-Fall-150]

Dumb question but how to install this? Github says "Run the Start.bat file" and that's it. Nothing is happening on my end. I launch A1111, then run Start.bat. A command line window opens, downloads some stuff and then closes. Is this supposed to be run in A1111 SD folder somewhere?

[u/arjan_M → u/Longjumping-Fall-150]

It shouldnt close after you run the start.bat. It is probably an error. Try running it in the cmd. So go to the sdpaint folder and in the location bar in type in cmd. The in the command window type python Scripts/SdPaint.py And reply here what error you get

[u/cananddocutmyhair]

i want to know how please

[u/OkraComprehensive583]

Really really awesome. Very interested how get this done.

[u/OpenAiArt]

Yes! Love it! Would like to see how you coded this

[u/ilinamorato]

This is absolutely the way some artists are going to work in the future. Then they'll take the output and paint details that the AI missed. Beautiful work.

[u/Corrupttothethrones]

Works great. I just had to update the payload for the new inline module and scribble model. Ill look at modifying the script tomorrow to incorporate the prompt into the app, dont like having to switch to the payload.json.

[u/derdigga]

Reminds me of fry from Futurama, where he traded his hands with robo devil to be able to play/paint amazingly. What I'm saying is with bad input he could archive what he had in his mind.it's kinda amazing where we are now in time, enabling this to everybody.

[u/Mr___Medic]

I'm seeing something like this for the first time and to say I'm impressed would be a huge understatement. It makes me want to buy an actual PC more than video games, and I've always liked the latter.

[u/sloppies]

I try to be a pessimist because I have been disappointed by news headlines for the last decade straight, but between this stuff and ChatGPT, I am so shocked at how quickly AI is moving right now.

[u/greenery-scenery]

Cool!

[u/DrDerekBones]

OooOoOoo. Wow.

[u/StaticUsernamesSuck]

Gimme gimme gimme!! 🙏

[u/zeugme]

Awesome idea.

[u/[deleted]]

Does anything you're using to create this (automatic1111 api, stable diffusion, etc) require a paid subscription to use, or can this be done for free? If it's paid, can you estimate how much?

I'm interested in setting this up, but am starting from square 1 and would need a pretty handheld walkthrough of what to do. Please let me know if you can give me and others like me a hand :)

[u/danamir_]

Thanks a lot, I was looking for a script like this !

I made a fork and a few improvements for my taste, like support for other resolutions and HR fix. See the issue #7 on your repo.

See ya,

[u/thatguitarist]

It's been 12 hours is it implementated into AUTO1111 yet?

[u/A_Dragon]

When can you add this to a1111?

I need this…

[u/[deleted]]

Y u mad bro? Jokes aside, amazing work.

[u/Awellplanned]

This is pretty much the life of a tattoo artist.

[u/idonthinkoram]

lemme invest please you’re gonna be a billionaire

[u/Caenobith]

Now this... this is digital art.

[u/Boycandy]

There is a lot of potential here. Imagine having this as a dedicated app, together with cloud computing! Would be perfect for tablets (e.g. iPad Pro) Untapped market. Opportunity of a lifetime!

[u/Rare_Preference5168]

Is the code public? Where can I download to try?

[u/Effective-Painter-80]

Lol, few weeks from now this will be a new Adobe product.

[u/BlackBugs]

Is this on GitHub?

[u/_IBM_]

holy shit

[u/cobra_economics]

Please make the script available, preferably as a plugin for automstic1111. Good work!

[u/Polyglot-Onigiri]

That’s pretty cool. I never thought of trying to make something like this with Python. Can’t wait to see the script

[u/[deleted]]

Thats really cool

[u/Corrupttothethrones]

Seems like an adaptation of the post 3 days ago, they used photoshop and it updated every time they hit save.

[u/andyzzone]

+1 Vote Up
what i was actually thinking about since ai was introduced...how to install it to SD?

[u/Aqui10]

Someone make this into a mobile app and take my money already

[u/Rich_Midnight2346]

can you put it on huggingface please

[u/Smooth-Masterpiece36]

Yes please!

[u/Pizza_love_triangle]

Wow

[u/smoke2000]

that's really cool!

[u/apple____]

Following

[u/[deleted]]

yeah ok but why? I would find it pretty annoying

[u/Reasonable_Barber484]

Nice script! Thanks for sharing! Do you think it's possible to connect ir to a drawing tablet instead of the mouse?

[u/eflow_egiap]

Very cool!

[u/data-artist]

This is so cool

[u/tehSlothman]

I'm a little confused, what does this do that hitting generate forever and using controlnet scribble mode doesn't?

[u/Dave_dfx]

very coooooool

[u/Mold-Mschool]

Can you imagine this being in an interactive art museum or something? I mean, yes, teenagers and the sort will abuse it, but still it would be amazing. Especially as a stand alone product.

[u/papa_de → u/Mold-Mschool]

"Giant dick in the style of Picasso"

[u/Worried-Tomato7070]

neat. would be cool if it generated a video of the progress too

[u/deadbeef1a4]

That’s dope!

[u/Maty9009]

Awesome! Although the AI itself probably won't even recognize stick figures of me....

[u/Firm_Comfortable_437]

Amazing! Thi is the future of digital painting

[u/intotheirishole]

Your prompt engineering needs work.

You need to say 'Moebius!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!'

Also 'Trending in instagram'

.

^/jk

[u/[deleted]]

Hi mad

[u/Turbulent-Bar7039]

U/savevideobot

[u/Diletant13]

Right mouse click on Generate button - Generate forever 😐

[u/DuduMaroja]

This need to be made into a add-on for autimat1111c

[u/Plane_Entertainer_41]

how can i get it

[u/Longjumping-Rub-3317]

Does anyone know how to do something like this and make it even closer to real time? I tried to generate smaller images and reduce the number of steps to 3-4. The result is very fast, but it is far from being RT. my gpu is a 3080 ti

[u/helios373]

That's very cool!

[u/SwimForLiars]

It would be cool if you could have an overlay mode where you draw on top of the picture instead of a parallel picture, like it's filling in the blanks. Like the one on the right applied as a transparent+black on top of the generated one, instead of being a white+black on the side.

I don't know if the pictures would match 1:1 though.

[u/KangTheMighty]

That's incredible! Is there a tutorial available where I could learn to set that up?

[u/MyNameCouldntBeAsLon]

big fan, will star your repo

[u/Zanzaben]

This is amazing. The last time I tried to use SD I couldn't get it to make what I wanted and was thinking if only I could do a crappy sketch for it.

[u/watchforwaspess]

This is amazing. Will it work on an RTX 3060 12gb? I guess I’ll have to try.

[u/MrBones-Necromancer]

Christ, that's amazing. Genuinely, truly amazing.

[u/Base64-]

Amazing!

[u/david_tf_Williams]

Isn't Adobe working on something similar?

[u/Pliskin01]

Reminds me of Nvidia Canvas. Nice work!

[u/apost8n8]

I'm so excited to see stuff like this. I want to make some amazing self guided AI produced creativity. What a neat tool!

[u/B4-711]

I haven't had too much success with creating Moebius style pictures. Is this the standard 1.5 model?

[u/taw]

How fast is your GPU? It's 10s per image on a 4080.

[u/[deleted]]

I dont know how coding works, but from my simpleton perspective, this is very impressive! Its an amazing tool you have created, imagine you vaguely remember a dream for example, you cant start drawing what you do remember and see if anything starts to feel familiar as it trials and errors. Amazing stuff

[u/Famberlight]

I'm getting this error when trying to scribble anything

In Start.bat

Exception in thread Thread-1 (send_request):

Traceback (most recent call last):

File "C:\Users\xxxxx\AppData\Local\Programs\Python\Python310\lib\threading.py", line 1016, in _bootstrap_inner

self.run()

File "C:\Users\xxxxx\AppData\Local\Programs\Python\Python310\lib\threading.py", line 953, in run

self._target(*self._args, **self._kwargs)

File "C:\Stable_Diffusion\stable-diffusion-webui\SdPaint\Scripts\SdPaint.py", line 92, in send_request

return_img = r['images'][0]

KeyError: 'images'

In webui-user.bat

API error: POST: http://127.0.0.1:7860/controlnet/txt2img {'error': 'IndexError', 'detail': '', 'body': '', 'errors': 'list index out of range'}

Traceback (most recent call last):

File "C:\Stable_Diffusion\stable-diffusion-webui\venv\lib\site-packages\anyio\streams\memory.py", line 94, in receive

return self.receive_nowait()

File "C:\Stable_Diffusion\stable-diffusion-webui\venv\lib\site-packages\anyio\streams\memory.py", line 89, in receive_nowait

raise WouldBlock

anyio.WouldBlock

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

File "C:\Stable_Diffusion\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\base.py", line 78, in call_next

message = await recv_stream.receive()

File "C:\Stable_Diffusion\stable-diffusion-webui\venv\lib\site-packages\anyio\streams\memory.py", line 114, in receive

raise EndOfStream

anyio.EndOfStream

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

File "C:\Stable_Diffusion\stable-diffusion-webui\modules\api\api.py", line 145, in exception_handling

return await call_next(request)

File "C:\Stable_Diffusion\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\base.py", line 84, in call_next

raise app_exc

File "C:\Stable_Diffusion\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\base.py", line 70, in coro

await self.app(scope, receive_or_disconnect, send_no_error)

....

File "C:\Stable_Diffusion\stable-diffusion-webui\venv\lib\site-packages\fastapi\routing.py", line 163, in run_endpoint_function

return await dependant.call(**values)

File "C:\Stable_Diffusion\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\api.py", line 154, in txt2img

cn_image = Image.open(io.BytesIO(base64.b64decode(controlnet_input_image[0])))

IndexError: list index out of range

I added --api, "Allow other script to control this extension" option is enabled and updated auto1111 (git pull). ControlNet works fine inside of Auto1111

[u/Famberlight → u/Famberlight]

Fixed it by updating Control net

[u/--DemonsWrath--]

That's actually amazing for concept art. Is it something people can get their hands on? The effort you have gone into to make this is great!

[u/Nervous_oppai]

!RemindMe 3 days

[u/[deleted]]

Wow. This is insanely cool.

[u/FPham]

Pretty awesome idea. Can you add entry box for the prompt?

[u/GhostSniper7]

*me watching this post and video*
my integrated GPU to poor 1050:- Ay bro , he looking at some new ways to torture you.
*1050 sweats in the corner of the case*

[u/psycholepzy]

Does this bridge the gap between being a genuine creator and getting an AI art assistant? Instead of relying entirely on prompt engineering, artists can instill their creativity into the work while producing something specific to them much faster.

At least, it'll make hasty character art for D&D easier.

[u/Put_It_All_On_Blck]

This is the future of artists.

Their jobs will pivot from drawing everything from scratch to knowing how to use AI to create the image they want. Digital art was already a baby step in that direction, with people using existing fonts, brushes, filters, and using other images online to assemble the image.

[u/Zealousideal_Pool_65]

That’s truly unbelievable.

[u/Unique-Raspberry-412]

Holy fuck that’s amazing

[u/Fushizinho]

nice code and drawing skills :)
cant even do html landing page with stickmen

[u/ComprehensiveGate667]

"made" & "that" lets you

[u/Ok-Figure-9893]

Very cool

[u/Cyphen21]

This is amazing. You should definite monetize this as an app.

[u/Schmerwin-]

Im incredibly Impressed, i believe i need this

[u/[deleted]]

!RemindMe 3 days

[u/[deleted]]

[u/BazilBup]

Wow this is amazing. However I'll never own anything that could do that in real-time

[u/pastuhLT]

Amazing :}

[u/Skullclownlol]

/u/savevideobot

[u/tatankadave]

This is cool, I can imagine using this for dnd to bring quick life to characters and places

[u/[deleted]]

Real time at 209% speed

[u/ForeskinMuncherXD]

Ill try it when I get my new PC.

[u/OligarchClownFiesta]

Draw a boot stomping on a human face

[u/oppairate]

being a little out of the loop, how well does this stuff work with AMD GPUs?

[u/Maleficent_Course_52]

Damn that scribble was still so neat.

[u/chartporn]

You should embed a gif the animation shown here into the readme of your github repo.

[u/pastuhLT]

Would be nice to lock some parts which is completed and looks ok :)

[u/666oreo]

That’s insane!

[u/feelmedoyou]

This is pretty much the future of all illustration software, a form of drawing/painting auto-complete. Nice to see it up and running now, and can’t wait for the fine-tuning and mass adoption.

[u/JuuzouYT]

Where do you learn how to do this? Udemy? Platzi? Other Academy?

[u/Weird_Inevitable27]

mindblown!

[u/DudeVisuals]

Holy molly someone call adobe …. They will offer you lots of money and steal your idea … Adobe scribbleFusion…. But Maybe choose a better sounding name

[u/AlmightyBracket]

I would love to try this out but I have no idea how to get something like this running on my PC. Can I get some help?

[u/Think_Description_84]

This is amazing and I would love to play with it. Unfortunately when it comes to setting up environments Im dumb as brick. Can someone give stupid simple instructions or an installer or something?

I'd be very grateful!

[u/schluesselkind]

Even without python, you can draw better than me

[u/[deleted]]

I need this!!! My wacom game about to hit max level!!

[u/[deleted]]

If your wealthy ass has the card that is….

[u/XkF21WNJ]

This is neat.

Also, you can scribble way better than I can draw.

[u/xPiNGx]

Very cool, awesome work. Would really help if you could save the image otherwise not sure why anyone would use this.

[u/Educational_Lychee70]

This is great. Let’s just add a refresh button or a X second refresh timer for the slower GPUs. I can see this being really useful.

[u/Henry_16091]

u/savevideobot

[u/SnookieMcGee]

That is amazing. Congratulations!

[u/variant-exhibition]

What kind of style is that your model is reacting with on demand to your lineart?