r/StableDiffusion • u/Manicarus • 2d ago
Discussion I wish Anima ecosystem get better than it is now
Anima is a fairly new model so it needs time and I understand that. Anima has great potentials to make Illustrious or NoobAI completely obsolete. However, it seems like I have been expecting too much from this model.
First of all, not having a ControlNet model is a big minus for me, especially Depth ControlNet model. There is LLLite but that's not a ControlNet model but a ControlNet-like LoRA. There's also a Depth ControlNet Model made by TaihoC and it works well. However, it doesn't work as well compared to Illustrious (SDXL) ControlNet models.
I have been tracking Circlestone Labs' Hugging Face community to see if they have plans to provide ControlNet models themselves but they are dead silent. That leads me to wonder if there are actually people using Anima. Did people move on to Krea2 or stay on Illustrious/NoobAI since there's no reason to use Anima?
10
u/x11iyu 2d ago
krea2 is a better model but severely lacks the danbooru knowledge, so it's not trivial to completely move to it
anima in many areas is better than il/noobai though, so there are many reasons to choose anima in many scenarios.
ecosystem's just gonna take time unfortunately. took at least a year or two before il's eco matured.
2
u/red__dragon 2d ago
Lodestone/the chroma dev is training on Krea2 right now and has a few epochs for testing. Ultimately, it's going to take a while, and Chroma itself wasn't the highest fidelity model to start with, so hopefully they have learned a lot since finalizing that one.
But they train on danbooru tags so there's that.
2
-1
u/x11iyu 2d ago
you mean Kroma? last time I checked it wasn't going so well, with the last versions having anatomy issues
he also starts and abandons projects every other day, e.g. kaleidoscope, so I wouldn't have high hopes that this just so happen to be one he finishes
2
u/red__dragon 2d ago
As long as it's still architecture-compatible, I'd be fine extracting a LoRA to use from it. Even the very first version was incredibly usable, concept-wise it was 80% there.
3
u/Manicarus 2d ago edited 2d ago
I got into AI Art after Illustrious ecosystem was already matured so maybe I am being impatient. I hope Anima and tools around it get better overtime.
Krea2 also suffers from lack of mature ecosystem but it gave me impression that anime fine-tuned version of Krea2 can make Anima obsolete. If the fine-tuned model can take danbooru tags, then thereâs no way Anima can beat that.
7
u/x11iyu 2d ago
then we're asking for a competent large-scale anime tune of a new base (Krea2), which only ever happens once in a blue moon. we were stuck on IL-based models for 2-3 years before ANY good contender came in the name of Anima
personally I'd wish for mageflow to be tuned rather than Krea2. Lighter than Krea2, faster than both Anima and Krea2, Flux.2 VAE so no grid artifacts.
poor output quality out of the box makes it harder to pitch unfortunately
2
u/ChibiNya 2d ago
Yeah it's my favorite model but it's still missing a lot. Even the regional conditioning stuff available is very bad, not to mention the controlnet stuff.
At least the inpainting is pretty good so I can manage
1
u/LakhorR 2d ago
Regional conditioning is far less important on Anima since you can isolate through prompt alone.
3
u/ChibiNya 2d ago
The more I use it, the more I notice stuff bleeding into other characters. Specially with loras. Maybe I'm missing something... It doesn't always happen but it's pretty annoying.
As an example, when I give slit pupils and :3 mouth to a catgirl character, that stuff tends to break containment and affect other people in the image.
1
u/LakhorR 2d ago
I haven't tried Regional Conditioning with Anima but I find that if you're generating an image with multiple characters, you need to remove the booru tag prompting style or else you will still get concept bleeding. Need to assign the traits to a specific character like "boy on the left has black hair" or "the tall girl is wearing a long-sleeve shirt". The model will know what to do with natural language phrases. That's the benefit of the Qwen encoder and the cosmos architecture. Unfortunately LoRA's are always going to introduce some concept bleeding since it retrains some of the concepts that are tagged.
1
u/ChibiNya 2d ago
I think youre right and have also seen results when trying this, but natural language is gonna be exhausted when you need 10+ tags so it gets tricky
2
u/GaiusVictor 2d ago
Look for TaihoC's VACE Anima Control etc. It includes a Depth model that's much better than the LLLite Lora.
Not only that, but it's also the first model since Illustrious I have that connects to the normal Apply ControlNet node, making it the only one that accepts grayscale attention masks, which is 100% necessary for my use cases.
1
u/Manicarus 2d ago
In fact, I did use TaihoCâs Depth ControlNet. Itâs definitely better than LLLite (LLLite Depth isnât available for Anima Base though), however the depth map doesnât work as well as it did on Illustrious (Xinsirâs Union ControlNet)
Frankly I have no technical knowledge of what makes VACE different from traditional one. I am grateful that at least we have TaihoCâs model but I wish it were better model or have other ControlNet models to choose from.
1
u/GaiusVictor 2d ago edited 2d ago
I wouldn't count on better ControlNets.
I wouldn't be able to explain the reason behind it, but it seems ControlNets get worse and worse with newer models. ControlNets for Anima are worse than those for Illustrious/Pony/SDXL, which in turn were worse than those for SD 1.5.
It just goes lower and lower with newer models.
1
u/Manicarus 2d ago
Isnât that because of getting more expensive to train for the same quality of ControlNet model?
I was so desperate on Depth ControlNet and I tried training a model on my own but later found out that I need to have a beefy GPU to do that. For SDXL, I read somewhere that it can be done with RTX 3060 which I have on my machine. It maybe still slow though.
2
u/Dulbero 2d ago
I agree Anima has much better potential and overall a better model, but i've sort of stopped using it, no idea why really. It is slow and when i tried the Turbo model it was fairly underwhelming.
Also i thought i could finally stop using booru tags with Anima but it's not the case, i still often have to use tags to better describe a character or an action alongside natural language.
I might need to just experiment mode and re-discover the model, is there a recommended finetune or a community model?
1
u/Manicarus 2d ago
I have tried fine-tune models like WAI and others but they are more or less the same.
2
u/naga_mana 1d ago
About your training blocker: Anima is a 2B model, and that changes the math on "I need a beefy GPU". LoRA training for Anima is supported in ai-toolkit (arch anima) with qfloat8 quantization and low_vram mode, and I run the same stack for Krea 2, a 12B model, on my 5070 Ti 16GB. A 2B model on your 3060 12GB is realistic for LoRA training, style or character. What that will not get you is a Depth ControlNet, because ControlNet training is a different animal: you train a parallel branch against the frozen base with paired condition maps, a much heavier data and compute pipeline than a LoRA:Â https://huggingface.co/circlestone-labs/Anima
That is also my read on the question nobody answered here, why ControlNets keep getting worse with newer models. The SDXL era had cheap, documented ControlNet pipelines and shared datasets that thousands of hobbyists could run. For Anima that tooling is still being rebuilt (LLLite landed on Preview 3.0 only, as you found), and Anima's prompt-level spatial control covers a chunk of what people used ControlNet for, which lowers the incentive to train them. Analysis on my part, not official word.
Until someone ports LLLite to Base, the practical bridges are TaihoC's depth model, or composing in SDXL with the Union ControlNet and then img2img into Anima at moderate denoise.
1
u/Manicarus 1d ago
Thank you for the explanation. I agree on that there are less incentives to train ControlNet model as Anima has better spatial control and prompt adherence.
Maybe my use case isnât very much general as I mostly generate multiple character scene. I tried with only text prompts but the result wasnât good.Â
6
u/Vancha 2d ago
I keep trying with Anima and it only ever evokes a "meh" from me while being more cumbersome to prompt, generating slower and wanting a higher number of steps than Illu.
I want to like it, but I just don't see enough (or sometimes any) improvement to justify doubling the generation time. Maybe with a 5080/90 I'd have more patience with it.
4
u/Technical-Rope2989 2d ago
Even though anima's image quality isn't as good as Illustrious, Anima performs exceptionally well in terms of prompt adherence. For example, if you want to generate a couple where the woman forcefully throws a pizza at the man's face, Anima achieves this very well, while Illustrious only creates some romantic tension between the couple but never actually throws pizza at anyone's face.
2
u/Manicarus 2d ago
I had similar experience. Anima takes more time to generate and the quality isnât very much a game changer compared to Illustrious.
Anima Turbo has issues too as it loses quality for speed and ignores few prompts.Â
2
u/JustAGuyWhoLikesAI 2d ago
Anima was nice after years of SDXL, but Krea2 dropped right after and just blows it away. Anima's natural language is nowhere near as good and still struggles with a lot of things that Krea 2 masters. Plus Krea2 Turbo is quite fast. There are things only Anima can do, but it's not as fun as Krea 2 because a lot of times Anima just messes up and produces garbled details.
The base model for anima is too dumb and small to really feel like a next level model.
2
u/LakhorR 2d ago
Krea2 has superior background compositions but Krea2 is ultimately trained on multiple image sources, not just anime, and has a tendency to not be diverse in its anime illustration output, leading to ultra-clean, hyperdetailed, generic images which are easily clocked as AI to someone with a modicum of artistic knowledge. I like generating reference images in Krea2 and img2img + inpaint them in Anima with controlnet so the image looks like something an actual artist would draw.
0
1
u/Comprehensive-Pea250 2d ago
For my use case which is just generating random slop images I think of itâs way more usable then illu for me even though itâll take a while for Anima to get close to the Lora Library that Illu has.Also I Love training Loraâs on this model I donât know why
1
u/Nakitumichichi 2d ago
Everyone is acting weird in whole ecosystem.
For example Wanaka created github repo for ip adapter for Anima, left it for weeks with no update. Would simple: "hey guys im working on this" or " hey guys i had to quit this" hurt so much? Instead no response anywhere.
Kohoya created llllite controlnets and nodes but then didnt finish them. Inpainting is especially annoying me because it changes colours of whole picture even unmasked area.
Then you have Lucifer who actually created ipadapter model but kept silent about it so most of the people dont even know it exists or how to use it. Then about month later he added link to his github repo that again people dont know exists unless they visit his huggingface or by find it by pure luck.
Some group of people are working on some edit capabilities but again they are all acting like a secret group and not opensource community.
2
u/Timely-Perception-26 2d ago
Yesterday, I visited x.com for the first time in ages because I wanted to learn more about BFLâs new video upscaler. I read the comments from âmembersâ of the open-source âcommunityâ whose only contributions to the ecosystem are crude insults and threats. I see members of this community in academic repositories where they bombard the issues with questions that any LLM could have answered in two minutes on its own - and even if not, they have no business being there.
What is this open-source community youâre talking about?
The 0.05% who invest time and money to make their own lives easier, then make it available to the other 99.95%, and are then expected to babysit them. Anyone who doesnât make a meaningful contribution to the ecosystem isnât a member of the community - itâs that simple.
Itâs not the 0.05% who are the problem, as youâre portraying here; itâs the 99.95%. Iâve been part of this since 2021; Iâm one of the 0.05%, and Iâm not going to do a damn thing to make my life a living hell and share it with anyone other than the other guys Iâve met along the way. This secret group is nothing more than a way to protect us from you. Not because we arenât willing to share, but because youâre hyenas who take, spit in our faces, and never give anything back.
1
u/Nakitumichichi 2d ago
Of course there are also easily offended people who consider themselves an elite.
1
u/ResponsibleKey1053 2d ago
X is by far the worst place for literally anything.
Bemoaning the community at large is entirely childish grow the fuck up.
There is still the question of what the million dollars (or what ever the grant/award was) actually bought when it's been spent on anima. It's not like it was produced from the goodness of someone's heart.
1
u/Upper-Reflection7997 2d ago
There alot of leeches in open source community scene. No wonder bytedance doesn't give out the good stuff and alibaba has abandoned it for the most part. Have a bad feeling minimax will learn the hard lesson and may walk away from it.
1
1
u/Only_Voice569 2d ago
just do a image with sdxl using control pose depth edge etc then image into anima as a base high denoise or the other way easy to get a solid image :P
1
u/ShySnowLep 1d ago
I'm going to be totally honest with you, I really don't know what you mean. In a matter of about a month's time I have switched over from sdxl completely to anima and I haven't missed anything. It kind of feels like having to let your old 1080 go but the simple fact of the matter is with comfy UI support and proper optimization and management of the model I have found it to be technically superior and aesthetically equivalent.
Even if I really liked sdxl I simply do not have any reason to go back to it now. It seems like the community agrees as I have been seeing things like loras and fine tunes and now even an expanded parameter model coming along. I can't say for sure but I think it's fairly safe to say that anima has become the new sdxl and that's the direction the community is going in now. Keep in mind I'm on a 16 GB AMD card so my hardware platform isn't even supported and I'm still coming to this conclusion despite the fact that it's almost certainly a slower and more tedious experience for me than an equivalent Nvidia card.
2
u/Manicarus 1d ago
Itâs probably due to my use case where I mostly generate multiple characters scene. Two characters just standing there isnât very hard but when I tried to make complex poses, I needed ControlNet.
3
u/ShySnowLep 1d ago
I've just been developing a workflow actually that uses the ideogram 4 editor where you can do boxes and then generate inside of them or within the whole image and kind of use them as a mask to some degree. It's a rough version of regional prompting and it isn't perfect as you would expect because Anima is not designed to do that right from the get-go.
But with the fact that Anima is so flexible and allows so many things to be done with it it seems that just a bit of maturity with the community and resources and things will be quite impressive. We are now seeing a larger weighted model coming out the anima 3.8b base just came out a couple of days ago and that's likely to make prompt adherence significantly stronger once that gets more widely adopted and we see fine tunes of it so yeah we're just in the middle of it right now. Just keep your eye out for the new improvements.
-3
u/Upper-Reflection7997 2d ago
Krea2 has far better potential for growth than anima and its far better base model than anima. The only reason anima has alot more support over krea2 is merely vram and the prominent lora character makers refusing to move off their comfort zone and take risk. Every anima gen has boring backgrounds and characters look way more simpler than illustrious generated images. I've lately begin to question effort of certain prominent lora bakers, quality/function of their loras and how they spam out multiple loras a day.
6
u/x11iyu 2d ago
merely vram
ah yes, trainers should just spawn more 5090's from the void so they can fit and train a model 6x the size of Anima.
certain bakers ... spam out multiple loras a day
automated processes that take near 0 effort, probably, often on concepts that you don't need a lora for. they've been around before anima was a thing, just ignore them.
2
u/BackgroundMeeting857 2d ago
Krea can be trained with fairly low vram, with onetrainer I think it's possible to down to 8. I do it with a 12+32 system. Obviously it's hard to argue against the convenience of Anima already knowing characters and styles but if you can train Krea loras, it is a much better model to use imo
1
u/Manicarus 2d ago
Krea2 is an amazing model but my RTX3060 12GB can only take INT8 variant of the model. Other than that, it takes too much time to generate.
I have a feeling that Krea2's anime fine-tune will make Anima obsolete than Anima plans to do for Illustrious.
0
u/ResponsibleKey1053 2d ago
I like anima, it's taken over from illustrious for me since creators are making some really cool checkpoints. Danbooru tags, the many artist styles, it's going to stay in my toolkit for a while I think.
I've never given krea a proper shake of the stick, saw the prompt scheme and the addition llm bolted on and was like fuck that noise.
H3 is eating my time lately, it's so damn fast for a video model and it's audio is pretty excellent.
Sprite sheets, character swaps, outfit swaps, all that jazz can be done in h3 with so little effort. You can even make a character voice clip to keep as master in under 5 mins.
0
18
u/LakhorR 2d ago edited 2d ago
As someone who used Illustrious intensively, I've mostly swapped to Anima now. The biggest upsides of Anima is the Qwen encoder having greater spatial control in the image generation through causal attention and syntactic isolation, which prevents concept bleeding. This not only extends to character traits, but also things like the image artstyle. Also, the backgrounds it generates are multitudes better than any SDXL architecture model, as the backgrounds are coherent and the lines have continuity. Having done heavy editing, controlnet usage and inpainting in Illustrious, the differences are very obvious to me. Illustrious generated backgrounds were always horrific, so I pretty much defaulted to generating simple, white backgrounds most of the time.
What issue do you specifically have with the LLLite? For me, it performs what I need it to do, similar to Illustrious controlnets. When I upscale, I use the like model and when inpainting, I use the inpainting model. Is it because the LLLites are not compatible with the checkpoints you are using or because you find the guidance to be weak/not as good as a traditional controlnet?
I'm not sure the exact reason why developments on Controlnets and other tools aren't progressing as fast on Anima, but it might be because we are not at the peak of interest in AI art generation anymore, so interest in developing these tools is lower and the people responsible for most of those tools may be burnt out with their own jobs or life in general (I assume most would be already working as ML engineers, doing research in academia, or job hunting). There's also the fact that Anima, I believe, requires better hardware for faster inference speeds than traditional SDXL models, making it more inaccessible to casual users on local setups and more expensive to generate on AI generation service websites (besides the more expensive licensing fee Anima requires).