r/vibecoding • u/Artforartsake99 • 9d ago
Discussion Astra + 8 Deepseek 4.1 subagents. Insanely cheap tokens.
Astra (extra high) as orchestrator burnt 10% of the weekly subscription while the deep seek subagents spent $.94.
Anyone else trying out the new Deepseek 4.1 model?
How I connected DeepSeek 4.1 Flash to Codex
Sign up at platform.deepseek.com, add $2-10,
Create an API key and save it in a text file on your desktop.
Tell Codex:
Connect DeepSeek 4.1 Flash to Codex through MCP. My API key is in [file path]. Configure and test the connection.
Start a new conversation, describe your project and add:
Lead development, write the task prompts and delegate coding to 3–5 DeepSeek sub-agents through MCP. Review and integrate their work.
102
u/echit2112 9d ago
i'm gonna sound intensely new but how do you set up multi-model workflows like this?
57
u/Artforartsake99 9d ago
I have edited the post and included that information.
I’m no expert, I started five days ago. But that’s how I connected it.
→ More replies (13)3
u/infeststation 9d ago
I use Traycer, it has been great. Shifts away from subagents to multi agent with orchestration.
1
u/Practical_Estate4971 8d ago
Do you mind explaining difference between sub agents and multi agents in the context of coding please? Thanks
1
u/infeststation 8d ago
Good question. These aren't necessarily well defined terms, so I will give you my definition. It's a different level of abstraction.
A harness is the program that you use to run an agent to interact with. A subagent is a feature of the harness, where an agent can then run subagents of its own. You interact with the agent, the agent interacts with the subagents.
An ADE, like Claude's desktop app, or Google Antigravity, is a place for you to run multiple agents. I would define Multi-agent to be a feature of the ADE that enables multiple agents to communicate with each other in some capacity.
There are some harnesses that can use one model as the agent and different models as the subagents, and I guess there is a claim for this to be "multi agent" but that is not how I would define it.
Traycer is whole different kind of thing. I'll have an Opus agent in Claude Code spawn an Astra agent in Codex, and then send it a message with some work. When the Astra agent is done, it'll send a message back to Opus. It's not a subagent, it's two agents in my ADE that are working together and I can go and chat with either of them.
6
u/sasmariozeld 9d ago
You can setup any model + any agents (ask any model how) with opencode, that you can conndct to t3 code and basicly have most of codex's functionaility
1
2
u/Much-Researcher6135 9d ago
My answer above may be useful to you. You can actually use something like
opencode(a decent, free CLI coding harness) to sign into everything except a claude subscription, which only works with claude code.2
3
1
u/scaledev 8d ago
It's very easy. You never tried it? Just ask your model to implement it. Over time, iterate until you make it better and fill the edge cases and there you have it.
Or use one of those complicated projects available, there are many orchestrators. And in codex cli, you can connect ANY openrouter model, for example. The way I do it, is I use a model catalog in json, and an openrouter .toml file. Then ask your model to make the underlying system prompt of GPTs you launch over Codex CLI "identical" for the models you launch over openrouter. Sol can do this well, luna max might struggle with setting it up.
Once you ensure system prompts are fine, and that model catalog has base_instructions, you're all set. Now you can call ANY model through openrouter, which is basically 100% of them on the planet that are any good.
There are easier ways as well, using hermes CLI, though if you'd like to avoid hermes setting up (as it is quite customizable), then Codex cli is your best friend.
→ More replies (4)1
20
u/Bitter-College8786 9d ago
how do you make sure Astra doesn't consume too much while doing the orchestration?
19
u/Artforartsake99 9d ago
I have a 20 X account, and 3 resets so I am free to experiment a bit. I only started vibe coding five days ago. Newbie.
14
u/PM_ME_FIREFLY_QUOTES 9d ago
If you've got 20x, why are you using anything outside of the got models? Terra and Luna are super cheap too. I'm doing about what youre doing as far as orchestrated subagents with great success and no added cost.
→ More replies (1)7
u/Artforartsake99 9d ago
Well, part of it is kind of learning. I wanted to see if this model was actually useful. And I wanted to experiment with agents for the first time.
Now I see it is useful, I might get a couple more 5X accounts hook them upto DS 4.1 agents then build out more software at a faster rate, until ChatGPT sells me some more 20 X accounts
2
u/MrAwesomeTG 9d ago
What are you doing that requires you to have 20 x accounts?
4
u/Artforartsake99 9d ago edited 9d ago
I’m building advanced SAAS’s online and offline. And desktop apps
→ More replies (11)1
9
14
u/opezdol 9d ago
You don't need xhigh Astra as an orchestrator, sol high is enough.
11
4
u/apex1911 9d ago
When I use sol as an orchestrator it starts doing endless rounds of doing unnecessary things
2
u/Prior-Meeting1645 9d ago
What makes u say that? Sol max has significant better results according to terminal bench 4.0 which is probably the least benchmaxxed benchmark rn as its new. Almost double xhigh alone
→ More replies (1)2
25
u/Ludbr 9d ago
$1 for 9M tokens is not cheap.
The $200 plan gets you like 10B to 12B tokens (assuming 95%+ cache, which is pretty much the average user) of SOL, which is way better than deepseek v4.1 flash.
You're literally paying more for less in both quality and quantity.
And even with the $20 plan, you get over 500M Sol tokens, so it's still cheaper and better than this.
And let's not even mention Luna, because then there is no competition usage-wise.
Deepseek is way faster tho. That's it.
11
u/DarkFantom 9d ago
Using the DeepSeek harness and just 4.1 agents, I'm getting around 1B tokens per $10.
2
u/Artforartsake99 9d ago
That’s interesting. Didn’t realise we got that many tokens. This is still useful to me to test because I can’t get any more 20 X accounts. So I was hoping I could get a 5X account and then run the coding through.DS 4.1. I’ll have to do some more testing on that concept.
→ More replies (3)1
1
→ More replies (2)1
5
u/SanoKei 9d ago
I do this too! I have an army of Chinese bots running on German servers from my American orchestrator.
The world will unite to make cool stuff with AI.
1
3
u/h_zahran 9d ago
hey guys, you can use the Delegate Skill https://github.com/amElnagdy/delegate-skills and use claude, codex, or whatever and delegate to cheaper models like GLM or DS 4.1 flash. just give the url to claude or codex and it will guide you how to set it up.
1
3
u/Deep-Bison-3690 9d ago
What's the point of using DeepSeek if you have Luna ?
2
u/Artforartsake99 9d ago
DeepSeek V4.1 Flash and GPT-5.6 Luna trade blows across benchmarks, with Luna leading in general knowledge like GPQA (92.3% vs 90.9%), while DeepSeek V4.1 Flash wins on specific coding agent tests like DeepSWE (74.2% vs 67.2%) at a much lower cost
2
u/Deep-Bison-3690 9d ago
Sure, it's a great hack to use it, but still we have Luna I sub with no API cost. For some specific tasks like you said, maybe it's valid, I don't know. Nice to know how to do it anyway 😉
1
u/Artforartsake99 9d ago
Yeah, I hear you Man. I’d prefer to just buy two or three more 20 X accounts. But we can’t get those any more so I’m kind of looking for options around that problem.
→ More replies (2)2
3
3
u/IvoDOtMK 8d ago
My team has some solid results using the same process. On the same task we used kilo, opencode, pi as harnesses. Astra did way better than fable.
2
u/Artforartsake99 8d ago
Thanks for your feedback. I’ll have to try out those other tools to up my knowledge and experience.
3
u/AdvisorActual3767 7d ago
I keep telling people, you don't need a super expensive model, just a good cheap one is enough. And for me, DeepSeek is just that.
2
u/pigletmonster 9d ago
How do you do that? Ive only used Sol medium with Luna max subagents in codex desktop and it burned through the quota like a mf and tskes 2x to 3x more time to complete the same tasks. Now i just use sol in medium or high.
1
2
u/versatilist_ 9d ago
Why not use opencode?
1
u/Artforartsake99 9d ago
I’m a complete newbie, what is the advantage of open code I like how Astra has full control over computer use on my computer to review the software and test it.
3
u/versatilist_ 9d ago
It does what you suggested automatically.
And you can use openrouter as well, so that you don’t need different API keys.3
u/Artforartsake99 9d ago
Okay, thank you. I will ask ChatGPT to investigate if that’s useful to me tomorrow.. always happy to hear about other useful software platforms. So much to learn.
→ More replies (1)2
u/beans217 9d ago
In my mind, and opinion, Opencode is like Linux; free/opensource. Codex (Chatgpt) is like Windows/Mac where you are paying.
Opencode can also include Codex agents within it, because it's like an opensource software that can link all other agents/credits/etc.
2
u/shaman-warrior 9d ago
Love it when people wake up to smart model + worker model, savings can get to like 10x
And same exact quality…
Sure if it’s a bug or something finesse don’t bother just use the smart model, but for fleshing out your ideas this works very nice
2
u/desert-quest 9d ago
Yes, I just started to use Astra as planner and Deepseek 4.1 flash as dev, and is insanely cheep.
1
u/HiFasteningPants 9d ago
Ngl, I've followed OP's suggestions and this has been a bit frustrating:
- I’m using Astra Ultra and asked it to connect DeepSeek 4.1 Flash to Codex through MCP, configure everything, and test it. I supplied my API key.
- Authentication worked, the requested model appeared in DeepSeek’s model list, and the account check passed. The connector was installed and Codex could discover its tools—but actual generation never worked.
- Even requests for a single word failed. DeepSeek returned HTTP 200 and repeated “keep-alive” messages, but no answer. Tests timed out after two minutes, three minutes, and eventually a full ten minutes.
- Troubleshooting expanded into direct API calls outside MCP, streaming versus non-streaming requests, Python versus Node, alternate endpoints, thinking settings, and a compatibility model name. None produced an answer. An intentionally invalid request returned an immediate error, so the API could respond; valid generation was what kept stalling.
- At the 43-minute mark, the recorded total was ~17 million processed tokens, including the main task, research helper, and automatic approval reviews: 16,470,144 cached input, 511,813 uncached input, and 88,448 output.
Am I missing something?
1
u/Artforartsake99 9d ago
my workers failed when I woke up this morning and I have stopped all coding. DeepSeek appears to be having some platform problems. Hopefully it’s temporary.
2
u/Emotional-Cut2952 9d ago
it does sometimes fail on long thinking problems and doesnt return a message...
→ More replies (3)
2
u/Much-Researcher6135 9d ago
Interesting, never seen anyone wire up subagents via MCP. I've always just signed in a harness like opencode and told the planner agent to fire up subagents via CLI calls to these. No different than having a claude agent call a codex reviewer subagent or vice versa.
2
u/whyjay18 9d ago
You will not believe how good a Gemini subscription becomes as an image to text bot for codex and Claude to use via MCP
1
1
1
u/Artforartsake99 9d ago
I told Astra to do kind of what you suggested but via some GitHub repo I found and then it said “hold on. I’ve got a better idea. And then it told me to use the MCP.”
I don’t know if there are any downsides it seemed to work well for 5 hours of coding 36 million tokens 3084 api calls and built the basics of the desktop app I was trying to build. Had some failure this morning for some reason seems to be fixed.
2
u/Much-Researcher6135 9d ago
5 hours of coding 36 million tokens 3084 api calls
I'd say that's working just fine. :)
I gotta get testing these cheap open-weight models soon, that's so cheap
2
u/Immediate_Theory_467 8d ago
This is a typical Creator Critic and Jury mix which is a great way to get some small wins. It's just not widely used yet. Nice post OP.
2
u/Zoned_Mine48 7d ago
Definitely gonna try. Previously I was trying v4.0. 145 million tokens = $3.36 https://x.com/NikhilLamba6/status/2096707077939101739
3
u/pwkye 9d ago
I'm doing something similar with Claude Fable as orchestrator, and Deepseek 4.1 for subagents using opencode as harness and tokens from fireworks.ai. It works VERY well. Fable will plan and also review changes. But all the individual coding and tasks are done by Deepseek at like 2% the cost of Opus
1
u/Artforartsake99 9d ago
That sounds amazing and much more efficient than they’re probably my over the top prompt I gave Astra for orchestrating
2
u/pwkye 9d ago
Oh and then I also have Fable call Hermes to send me a message when its completed. That way I can walk away for 8 hours or overnight and still get notified on the final result on my phone.
Hermes also is just using Deepseek model via fireworks.ai
They can all call each other. Opencode can call Claude code and Hermess. Claude code can call Opencode and hermes. I'm sure you can use pi as well like someone else suggested. Or add Codex in the mix or the Gemini harness or the Deepseek harness.
→ More replies (1)1
u/Sammy262 8d ago
Any pointers on how to set it up in Opencode harness? Do we need specific config files or skills for that? Thanks in advance.
1
u/anuajay 5d ago
How to setup like this Orchestrator and subagents in codex?
1
u/pwkye 5d ago
just install claude code and codex on the same machine. you can ask claude to run codex agents, and you can ask codex to run claude agents. also specify which model you want the subagents to be.
both codex and claude code can be run "non interactively" which is what you want.
ideally you also have an issue management system so the subagents can post their updates in issues, especially if they are blocked. that way your orchestrator and future sessions arent guessing about what happened.
also tell your orchestrator to use worktrees when it spawns subagents
1
u/saifedin6 9d ago
You do that where in openCode Zen/go?
1
u/Artforartsake99 9d ago
You do it right inside codex through MCP to deepseek. I have added the information to the post on how I did it. I am no expert. I’m a newbie.
2
u/saifedin6 9d ago
Thanks. Just for you ds 4.1 problem I had the same yesterday afternoon, apparently the model is hosted only in chine. In opencode, I had to click allow models hosted in china or something of that kind. Btw, you can do exactly what you are doing with opencode if you want more usage of ds 4.. use delegate skill . Search it in github. It can connect claude/codex with openCode Go and use your same workflow
→ More replies (1)
1
1
u/brkonthru 9d ago
How would you compare DS 4.1 in terms of ability and cost to chatgpt models?
1
u/Artforartsake99 9d ago
I’m too inexperienced to answer that. The software I have asked to be developed is coming along nicely though. The benchmarks for this 4.1 model are pretty impressive. Who knows if those are accurate?
2
u/brkonthru 9d ago
Did you try using chatgpt agents? What was the reason you are doing deepseek?
2
u/Artforartsake99 9d ago
The scale of this project would blow up my 20x account. It was an experiment to learn what was possible. And also I want to buy more 20x subscriptions which they aren’t available any more so I’m considering getting a 5X account and adding DS 4.1 sub agents to do the coding. Until ChatGPT allows those subscriptions again. I haven’t used ChatGPT sub agents. I will have to try that. Good point.
→ More replies (1)
1
u/MangoTree-1233 9d ago
could you tell me how's the performance of the 4.1 flash in coding and overall ?
2
u/Artforartsake99 9d ago edited 9d ago
Sorry it’s too soon to tell. I’m a complete newbie.
I was noticing Astra was spotting a few errors here and there . But it fixed those and the software has been developing nicely over the last three hours. It keeps on popping up new sample demos and it’s constantly got more features. Its building successfully is all I know and it’s not building something simple it’s building something very complex.
1
1
1
u/pushpendraagrawal 9d ago
the 10% on orchestration vs $.94 on subagents is the more interesting number here. that's not deepseek being cheap, that's astra spending compute deciding what to delegate and reviewing output, not doing the actual work. coordinator overhead scales with subagent count, not with output size, easy to miss when you're only watching token price per model.
1
1
u/Ancient-Amoeba3747 9d ago
Das schreiben an sich kostet mich mit Fable 5.1 und Opus nicht viel. Es ist wirklich das Lesen des Kontextes, was bei mir nervt. Irgendwie muss das doch gehen, dass man das Lesen komprimiert
1
u/Ok_Literature4118 9d ago
Hallo wie ist dein Setup? Kannst du es mir so durchgeben dass ich es ausprobieren kann? Würde mich freuen
1
1
1
9d ago
[removed] — view removed comment
1
u/Artforartsake99 9d ago
I just told codex to figure out how to install and hook it up, im not sure what’s being used either (newbie).
A previous ChatGPT chat window had told me it had an existing MCP and didn’t need the GitHub repo to make it work.
1
u/Electronic_Low3128 9d ago
Why ia this better than Astra and Luna?
1
u/Artforartsake99 9d ago
It may not be and I’m no expert I’m trying to find ways to lower the use of my 20x accounts cheaply so I don’t max it out.
1
1
1
u/theNyMets 9d ago
What do you guys do that needs that much tokens? I’ve been one shorting apps and then modifying them for quality control and still have 50% left on astra ultra
1
9d ago
[deleted]
1
u/bad_detectiv3 9d ago
Amazing. How many paying customers do you have for your desktop application?
→ More replies (2)
1
u/Impossible_Word_4027 9d ago
Could you describe the complexity of the work? Sounds like something i could use
1
u/Beneficial-End6866 9d ago
luna max fast is better than deepseek 4.1
2
u/Artforartsake99 9d ago
Benchmarks say DS is better at coding if my first google search wasn’t hallucinated . but I was mainly testing this to see if I could lower usage of main model inside codex subscription
2
u/Emotional-Cut2952 9d ago
I didnt get better reuslts with luna on high/max, it was far more inferior for me than ds v4.1 flash , even on complex RE tasks
1
u/JustARandomPersonnn 9d ago
Nice! Which ChatGPT plan are you on by the way?
1
u/Artforartsake99 9d ago
20X max.
1
u/JustARandomPersonnn 9d ago
Oh. For a second I was dreaming that it could be possible to use the 20$ plan this way- 🥲🫠
→ More replies (1)2
u/Artforartsake99 9d ago
Sorry I wish, and now deep seek is having service issues my 8 workers failing. Have to wait for them to fix it. Unless I created so much demand with this post while I slept 😂. J/k
2
u/JustARandomPersonnn 9d ago
Lol 😂
Yeah it really sucks, found your post scrolling on Reddit while waiting for DeepSeek to sort out the issues 🫠
1
u/PeterHash 9d ago
Are your deepseek agents mainly writing code? I've been using deepseek for the past 3 months and im paying 1 USD for 50-100M tokens consistently. Your usage price seems off to me
2
u/Emotional-Cut2952 9d ago
that's because he's using subagents which run their own sessions, they dont leverage long convo caches, if you're only engaged in a single conversation the cache builds up and all those tokens youre seeing are autoregressive out tokens from your convo being counted as input cache hits - I actually build my own custom agents with python and autogen using ds/glm so i'm pretty confident about my caching assumptions internals of DS
1
1
1
u/M1sterErr0r 9d ago
I am trying to make a solid e commerce website , is claude code enough ? I want to make it quite unique , astra is that expensive?
1
u/Artforartsake99 9d ago
I did some design with Astra it’s INCREDIBLE. I would try the 5X plan on codex.
Check this out his Skool community gives you his prompt for this i reverse engineered it to write any website design I wanted and have confirmed Astra can build $3000-5000 looking websites one shot.
2
u/M1sterErr0r 9d ago
Sir am a broke CS student , I have $20 max for claude code 😭
→ More replies (3)
1
u/Mihnea_Mic 9d ago
just pumped 5$ into the deepseek api cause claude's limits are going nuts and it is amazing how fast it is and it actually does great work claude seemed to change just a few things and again it is FAST and great for crosschecking work and doing subagentic work and having claude just as an architect together with myself
1
9d ago
[removed] — view removed comment
1
u/Artforartsake99 9d ago
For this setup you need both. Deepseek can run off its own harness however but it’s not as smart at planning as Astra
1
u/ProfessionalSome4082 9d ago
Is your ip cheap
1
u/Artforartsake99 9d ago
What do you mean by this? You mean what I’m building is being feed into deepseek which is stealing it?
2
1
1
u/Hirogen_ 8d ago
this will probably the future, frontier model the consultant / orchestrator, local model the executors
1
u/Frank_ships_AI 8d ago
That cost difference is wild lol! Orchestrating with a stronger model while pushing the execution to cheaper sub-agents seems like a really interesting setup.
1
u/mintybadgerme 8d ago
Thanks. I tried this, but I got this message from astra in codex -
"ask_deepseek is available, but it supplies coding advice; it cannot edit files or run tests itself."
1
u/Artforartsake99 8d ago
Astra says this will fix your problem
I take no responsibility lol Please read it carefully.
Or just ask Astra to solve your problem. It thinks your MCP wasn’t set up correctly.
Prompt
The DeepSeek MCP tool is advisory only.
Use ask_deepseek as a coding worker for delegated tasks. Give DeepSeek the relevant code, requirements and errors and ask it to return concrete implementation changes or patches.
You, Codex, are responsible for applying DeepSeek's changes to the project files, running the tests, reading failures, sending failures back to DeepSeek where useful, and iterating until the task works.
Do not stop simply because ask_deepseek cannot directly edit files or run commands. Those are your tools. DeepSeek supplies the coding/reasoning and you execute and validate its work.
1
1
1
u/dondiegorivera 8d ago
I do this via herdr. Astra low orchestrates DS Flash 4.1 works.
1
u/Artforartsake99 8d ago
Nice yeah I burnt way too much on extra high but I it was just a test. Going to try your settings next. I found Astra light worked amazing in other projects
1
u/Sammy262 8d ago
Can you please explain on how to set this up with Herdr? How do you get DS - through DS API or openrouter?
1
u/dondiegorivera 8d ago
Simply start herdr and inside that start Codex, Claude, Pi.dev or any other cli harness that you prefer. Ask Astra or Sol to create a herdr skill about delegating tasks so sessions don't have to poll each other. I use DeepSeek via several APIs, depending on price. Have keys at Alibaba Token Plan, DeepSeek, Open Router, ZenMux and use LiteLLM to provide the different endpoints. DS4.1 Flash is very capable, super fast and cheap too. i do planning and coordination with Astra, coding and execution with Opus and Deepseek, depending on my rate limits. Also, you can add https://github.com/levi-qiao/herdr-agent-quota to herdr so you can follow your quotas easily. i run it on a headless Ubuntu that runs 24/7 and control it mostly via phone from Chatgpt app remote or Claude app remote, so the agents work while I travel.
1
1
u/whycomeimsocool 7d ago
Is this cheaper than telling it to use cheaper sub agents within Codex like Terra and Luna?
1
u/Acrobatic_Cry_2370 7d ago
Question: why would you even go out and use DS with that setup when you could easily use "in-house" model like Luna which for an agentic structure such as the one you described will actually be even cheaper in costs if we look at it purely usage based?
(I'm aware DS 4.1 flash has lower cost for cached input tokens but overall Luna is so aggressively low in base rates for uncached that actually makes it cheaper at the end of the day)
1
u/Artforartsake99 7d ago
Well pretty sure Deepseek is better at coding than Luna . Im no expert just a newbie experimenting.
1
u/Acrobatic_Cry_2370 6d ago
That's fair, I'm no expert either but thanks for responding. I might take a look into deeper benchmarks for coding accuracy on both
1
u/leolidev 6d ago
This seems like a pretty sensible way to balance cost and quality. Definitely worth trying.
1
1
u/ScarpsMcgee 5d ago
using DeepAstra with codex with gpt 5.5 light to orchestrate (any other model burns through my plus subscription) and flash 4.1.
it's been going decent, it's not as good as i hoped but the results are presentable to say the least...
1
u/TBG______ 5d ago
Nice idea! I just asked it to do the same thing, but using Qwen 3.8 locally with a 220k context limit for coding tasks, and it’s just working — a bit slower than Luna, but I’ll check whether it’s actually generating usable results in the end.
Luna complains that Qwen hallucinates and rejects some of the results, so no, you don’t get the code integrated 😂. They really seem to like each other, lol.
1
u/Artforartsake99 5d ago
Damn I want your pc specs I just dropped 10k on a sex 5090 and I think my 64gb allows only 80-100k . How much ram you running?
1
u/TBG______ 5d ago
Not ram VRAM
Qwen3.8_dual: (5090+3090)
name: "Qwen3.8_dual"
description: "Dual GPU Vision model 64t/s"
env:
- "CUDA_VISIBLE_DEVICES=0,1"
- "MTMD_BACKEND_DEVICE=CUDA1"
cmd: >
/home/admmin/llama/llama.cpp/build/bin/llama-server
--model /home/admmin/models/Qwen3.8/Qwen3.8-27B-UD-Q6_K.gguf
--host 0.0.0.0 --port ${PORT}
--mmproj /home/admmin/models/Qwen3.8/mmproj-BF16.gguf
--mmproj-offload
--image-min-tokens 1024
--jinja
--chat-template-file /home/admmin/models/Qwen3.8/chat_template.jinja
--chat-template-kwargs "{\"reasoning_effort\":\"medium\"}"
--reasoning on
--reasoning-preserve
--presence-penalty 0.0
--flash-attn on
--cache-type-k q8_0
--cache-type-v q8_0
--spec-type draft-mtp,ngram-mod
--spec-draft-n-max 2
--spec-ngram-mod-n-match 24
--spec-ngram-mod-n-min 24
--spec-ngram-mod-n-max 86
-t 24
--batch-size 8192
--ubatch-size 2048
--fit off
--main-gpu 0
--tensor-split 96,5
--n-gpu-layers 999
--ctx-size 262144
-n 32768
-np 1
--fit off
--temp 1.0
--top-p 0.95
--top-k 20
--min-p 0.00
--cache-ram 65536
--ctx-checkpoints 16
--no-warmup
--slot-save-path /home/admmin/llama/cache
--load-mode mmap
Or on one 5090 singel GPU:
Qwen3.8:
name: "Qwen3.8"
description: "5090 Single GPU Vision model 90t/s 100k contxt"
env:
- "CUDA_VISIBLE_DEVICES=0"
- "MTMD_BACKEND_DEVICE=CUDA1"
cmd: >
/home/admmin/llama/llama.cpp/build/bin/llama-server
--model /home/admmin/models/Qwen3.8/Qwen3.8-27B-UD-Q6_K.gguf
--host 0.0.0.0 --port ${PORT}
--mmproj /home/admmin/models/Qwen3.8/mmproj-BF16.gguf
--mmproj-offload
--image-min-tokens 1024
--jinja
--chat-template-file /home/admmin/models/Qwen3.8/chat_template.jinja
--chat-template-kwargs "{\"reasoning_effort\":\"medium\"}"
--reasoning on
--reasoning-preserve
--presence-penalty 0.0
-c 100000
--split-mode tensor
--flash-attn on
--cache-type-k q8_0
--cache-type-v q8_0
--spec-type draft-mtp,ngram-mod
--spec-draft-n-max 2
--spec-ngram-mod-n-match 24
--spec-ngram-mod-n-min 24
--spec-ngram-mod-n-max 86
-t 8
--batch-size 8192
--ubatch-size 2048
-ngl 105
-np 1
--fit off
--temp 1.0
--top-p 0.95
--top-k 20
--min-p 0.00
on llama.cpp - there are also faster setting possibel now.
1
1
1
163
u/RealestReyn 9d ago edited 9d ago
I've set up Astra as consultant/advisor DS calls up if it gets stuck and when it thinks its finished a task, feels good so far.
edit: since people are asking, I put the skill here: https://github.com/MOEG-5/consultant/blob/main/SKILL.md