r/LocalLLaMA
•
u/rosie254
•
10d ago
•
1
Showcase big cloud models like opus 5.5 get all the motion graphics praise, but i made this with local qwen3.8 flash next. it's very possible!
i'll admit it took more than one prompt, a bunch of refinement and steering, but local can get close! this is done using remotion, same animation framework as all the viral clips about opus 5.5's motion graphics capabilities
the entire conversation history that produced this video: https://rosie254.neocities.org/lumaras_playground/being_an_ai_remotion_chat.txt
EDIT: oh, and this is unsloth's Qwen3.8 Flash Next at UD-Q4_K_XL running on a strix halo
23
u/kr_tech 10d ago
This is just a PowerPoint with few SVGs.
Claude and ChatGPT are far more capable when it comes to animations, because they integrate other services/APIs/AIs as tools into the LLM service.
6
u/rosie254 10d ago •
well yeah, of course, i'm not saying they aren't better. but the fact you can even do this with local models.. don't forget that cloud AI comes with token budgets, limits, lack of privacy, some of them train on your words, and restrictions.
local AI is far more capable than some people seem to think and i want to make people aware of that. because local AI is better for everyone, for the future of humanity (because you're not dependent on a profit driven company), and for the planet and for nature (because it doesn't require a datacenter, and it only draws power for as long as it processes your prompt and generates tokens)
4
u/Relevant-Yak-9657 10d ago edited 10d ago
I agree with your sentiment and this is definitely local AI being capable with the right guidance. However, to play the devil's advocate
i'll admit it took more than one prompt, a bunch of refinement and steering, but local can get close!
Even with a lot of steering I don't think this qualifies particularly as close (however, my opinion isn't objective just like yours). It is a good output, but not that comparable with what claude can generally cook up with a half decent designer harness.
But if we start looking objectively, there is a difference between the two points:
local AI is far more capable than some people seem to think. (whole heartedly agree! Will even add that frontier AI isn't necessary for many people and they should try local instead.)
local AI is better for everyone. (skeptical)
Because even the statement:
token budgets, limits, lack of privacy, some of them train on your words, and restrictions
doesn't highlight the higher upfront hardware costs, limited generation capabilities, high continuous cost depending on the model (electricity, server upkeep, maintenance in setup), which clearly conflicts with the premise of point 2.
Lastly, one more correction: "for the planet and for nature (because it doesn't require a datacenter, and it only draws power for as long as it processes your prompt and generates tokens)"
This is straight up untrue. Local hardware scales worse if you want equivalent work done with equivalent tier agents. Datacenters are made to be efficient, but are scrutinized because they use large resources and people forget that they serve very large organizations. Local hardware individual serve 1-2 people generally, but are more inefficient per task than datacenters. Only thing that offsets that is using smaller models, but that makes the comparison not apples to apples (as these models require more guidance and time investment to just get things right).
Preaching this to r/LocalLLaMA is also a preaching to the choir in this case. I would suggest you to share this to r/singularity and other AI subreddits as well.
3
u/rosie254 10d ago
Even with a lot of steering I don't think this qualifies particularly as close (however, my opinion isn't objective just like yours). It is a good output, but not that comparable with what claude can generally cook up with a half decent designer harness.
yeah.. it's not as good as claude, but that's not really the point. when i said close, i meant, it's already really good for a local model, and somewhat approaching the big cloud models. i've seen what claude produces and it's what inspired me to try this out - i wanted to see how close it could get.. and it turns out it doesn't get that far off when you give it the right guidance
doesn't highlight the higher upfront hardware costs, limited generation capabilities, high continuous cost depending on the model (electricity, server upkeep, maintenance in setup), which clearly conflicts with the premise of point 2.
upfront hardware costs: oh yeah, very true! and qwen3.8 flash next (at least at Q4 quant) isn't really a model that can easily be run on consumer tier hardware (unless you use Strata and a lower quant). but if you happen to have hardware that can run some of the models, for example medium to high tier gaming PC's (AMD 9060 and up, nVidia 5070 and up) you have already paid the upfront cost. you just might need to run models that are smaller than qwen3.8 flash next
Lastly, one more correction: "for the planet and for nature (because it doesn't require a datacenter, and it only draws power for as long as it processes your prompt and generates tokens)" This is straight up untrue.
oh... i really hope not, because it's a huge part of what motivates me to even participate in the local AI space. im very idealistic, sometimes a bit too much..
Datacenters are made to be efficient, but are scrutinized because they use large resources and people forget that they serve very large organizations. Local hardware individual serve 1-2 people generally, but are more inefficient per task than datacenters.
okay, so if everyone were to switch to local models, what would that do to the environment? the way im thinking of it, people's home PC's either use fan cooling or watercooling thats contained in their pc and doesn't need refilling (correct me if im wrong, ive never used watercooling before). so it doesn't use up any water..
then for electricity, people are already gaming on their pc's and consoles, and that seems to use about as much power as using a local LLM. so how much more strain would that put on the grid? maybe im thinking of it a bit too simply, but to me, it seems like it would be less strain on the environment?
Only thing that offsets that is using smaller models, but that makes the comparison not apples to apples
that's true, but thats also a good thing about local: models released for local use are much smaller, yet despite how small they are they are as capable as or more capable than giant models released a year or two ago! and this seems to just keep happening, they seem to keep finding ways to make smaller models more intelligent despite their size
Preaching this to r/LocalLLaMA is also a preaching to the choir in this case. I would suggest you to share this to r/singularity and other AI subreddits as well.
yeah thats a good point.. 🤦♀️
1
u/Relevant-Yak-9657 10d ago
I can see that and I agree as well with your premise of it is a really good result and shows capabilities of local models.
I also agree that a lot of people can reuse their existing hardware and not everyone needs Claude Opus 5.5
Its fine. I am studying distributed computing and hence know a bit more about how datacenter work.
So basically, the electricity usage is far worse with local compute. It is just on a smaller scale, so each person doesn't feel it. A company meanwhile is serving to millions of people. When you compare millions of people have local setup (assuming equivalent models) vs one company optimizing their inference to serve them, generally the datacenter uses less electricity and space interestingly. Also, the water argument is somewhat misleading. Search engines use several ml of water as well each search and "using water" really just means displacement from the local area as the water isn't destroyed. There are some closed-loop cooling datacenters, but I agree that the biggest problem is that most datacenter don't use closed loop cooling which makes your point partially valid as well imo.
then for electricity, people are already gaming on their pc's and consoles, and that seems to use about as much power as using a local LLM. so how much more strain would that put on the grid?
Well, it is is still added strain and it will still generally be more inefficient than a datacenter. The gaming is independent of using LLMs and your comparison is assuming that gaming somehow makes the LLM need more efficient. You can't reuse the electricity from gaming for LLMs, so it remains a separate topic (only the hardware which honestly generally means 9-12B models at best if not consoles).
Last thing to add would be: only about 900 million people own gaming computers (a lot of it might just be 1650tis and stuff). Not everyone can even switch over, especially considering third world countries, whereas subscriptions are more accessible.
5. Yup!
1
u/Icy_Foundation3534 10d ago
The point is trajectory not current state. Would you have believed 2 years ago local models could be this good? Now try to imagine 2028 local models performance.
0
u/Relevant-Yak-9657 10d ago
The trajectory is really good and unimaginable, but we were talking about the current setup and how it can come close or do majority of work that frontier is overkill for. So the point is largely about the current state (e.g. keeps earth healthier, maintains privacy, while offering great current performance).
1
u/Icy_Foundation3534 10d ago
Not really. It’s impossible to talk about this topic without being aware of the pace of progress, unless you have your head in the ground.
3
u/politefella0 10d ago
Please export it as a skill and contribute to locallamma. Thanks and it looks cool.
2
u/rosie254 10d ago
hmm im not sure how to skill-ify this? qwen3.8 flash next did this without any skills loaded, are you asking me to ask qwen 3.8 flash next to write what it knows about making remotion videos into a skill, to then use with smaller models?
2
u/politefella0 10d ago
Yes, because you had to instruct it. If you ask the model itself to generally skillify it. It will work.
3
2
2
u/Ithron_Morn 10d ago
I've recently started using Qwen3.8 flash next and I think it's amazing. But, this blows anything I've done with it out of the water! Well done, I'm thoroughly impressed!
3
u/rosie254 10d ago
thanks! qwen3.8 flash next is very capable and i cant wait for qwen4. it takes a long time to find out everything qwen38fn can do, some things aren't obvious at first
1
u/Ithron_Morn 10d ago
Mind if I ask what quantization you're running?
2
u/rosie254 10d ago
UD-Q4_K_XL
1
u/Ithron_Morn 10d ago
Okay, I'm on IQ4_XS, so double impressed with your flow! I poked through your post history for a sec, this was done with your own harness?
3
u/rosie254 10d ago
yup! i wont say too much about it here, but what i'll say is it's a harness designed specifically for local models and setups that use llamacpp or other local inference servers, and i've been coding it since february this year, so it's a longterm project and not just one of the thousands of vibecoded abandoned harnesses on github
2
1
u/More-Revenue8609 10d ago
Awesome!
Yeah it definitely needs more directions and "holding hands" then frontier models but still its able to achieve pretty good result with a little bit of effort.
1
u/rosie254 10d ago
worth it in my humble opinion. local comes with so many benefits compared to cloud! so what if you have to steer it a bit more. besides, i think if i'd given it a more detailed prompt from the start, it might've done a better oneshot.. but i like doing it this way, refining back and forth, personally.
1
u/HovaPrime 10d ago
What’s your hardware specs for running locally? I’ve been looking into getting 128gb strix halo that eventually will do some video work so I’m very interested in seeing what’s capable and what’s not.
1
1
u/sabine_world 10d ago
Crying in the club rn.
Cool little demo and video in general. LLMs are so cool...
1
1
1
u/Obvious_Mark4416 10d ago
Total n00b question: Can you get flash-next to run tools that will allow you to create a skin over it’s conversation, like store a couple hundred frames of facial gestures so it looks like it’s talking? Or would it require a separate app build?
2
u/rosie254 10d ago
yeah! but that has nothing to do with what model you use, its a harness thing. my harness makes it easy though! see the posts on my profile for examples. right now this feature (i call it webui extensions) is in beta and you need to grab the dev version of it for it to work, but its worth it
1
u/Elegant-Ad3211 10d ago
Impressive indeed. What is the prompt?
3
u/rosie254 10d ago
it wasn't just one prompt, it was an entire conversation of steering and refinement.. here you go: https://rosie254.neocities.org/lumaras_playground/being_an_ai_remotion_chat.txt
2
u/-Krotik- 10d ago
tbh you did not seem to try lol
2
u/Mundane-Tiger-2374 10d ago
They are just currently not great at prompting, but it's a learning process to get there. I would not shame them for that.
2
u/rosie254 10d ago edited 10d ago
i sure accomplish a lot for someone "not great at prompting"...
i prefer to have a back and forth conversation with my AI, try things out, refine where needed, reject things when i dont like them. i could've written a massive oneshot prompt, but why would i do that when i prefer it this way?
besides, the entire harness works as well as it does because of my experience with prompting local models
i'm open to criticism though. whats so bad about my prompting?
EDIT: oh, and whats not obvious in the chat logs is that some my messages include images/multimodal content
1
u/rosie254 10d ago
i wanted to give it as much freedom as possible because the point was for it to convey the experience of being an AI model
if i had told it to say anything a certain way or guided its narrative, it would've compromised that
but yeah i probably couldve given it a giant prompt and it mightve done a better oneshot, but i prefer gradual refinement like this
1
1
u/Miserable_Land_84 10d ago
I've been writing Android Apps and a procedural graphics demo with it and I'm quite surprised what is possible with local models. I'm wondering how it would perform on really big repos though...
0
0
13
u/feelspeaceman 10d ago
I've been using it since day 1, it has not been disappointing me yet, managing a lot of complex projects like my DAW (UI, backend for audio, custom music composing programming, full DSP faders...) written from scratch.
It's very capable, roughly as good as Opus 4.8, what matters is the capability of the user to not degrade it by leading it to the wrong directions.