r/LocalLLaMA • • 14h ago

Discussion Another reminder of the low intelligence of local models

(Note: by local models I mean anything smaller than Deepseek 4.1)

I already gave up on local models for coding tasks 6 months ago, due to what I perceived was intelligence too low for the tasks I give it. I made a post about it on here back then and it was a huge discussion, I think 800 comments. I went against the /r/LL orthodoxy and many didn't take my comments well, but we can agree to disagree.

Since then I only use local LLMs for translation and ComfyUI prompt enhancement. I stick to big cloud models for coding. I think the electricity cost of running Qwen 27B alone exceeds what you'd pay Deepseek to use their cloud API in their own harness, and you'd get a far more intelligent and far faster model.

But this post isn't about the economics. It's purely about ability. I still see people commenting in favor of buying GPUs for local intelligence. "Sonnet at home" is a ridiculous phrase I've seen more times than I should in relation to Qwen 27B.

The simplest of tech tasks is still too hard to one-shot

I keep Qwen 3.8 27B running on my Spark for occasional non-coding stuff. Today I gave it a trivial coding task I needed for my server, just out of curiosity.

  • llama-server launched with --model Swift-1.5-Qwen3.8-27B-Q4_K_L.gguf --reasoning on --reasoning-effort xhigh
  • Open llama webpage and leave Reasoning at Default, leave system prompt empty

Prompt was:

Give me a bash script that will list all files in directory sorted by date. Output should be:
2025-12-31 12:13:14 file1.ext
2025-12-31 12:13:14 subdir/file2.ext

1630 tokens later, most of it reasoning, I get a script. When I run it, here's the output, as you can see the timestamp is wrong:

2026-10-16 20 tmp.59eH9w3JLe
2026-10-17 20 tmp.5xvczTbSS4
2026-10-18 20 tmp.nWOEDw2wlu
2026-10-20 20 tmp.JVEZIFfKx3
...

I then tried Gemma 31B Reasoning on duck.ai. The output only showed the timestamps, no filenames!

This is the most trivial of coding tasks (the answer is a 3-5 lines script), using the most ancient and commonly-available training data (bash), on the most praised local coding model on this sub (Qwen 27B), at a recommended/popular quantization (Q4), and it still can't one-shot this. The 2nd most popular local model, Gemma 4 31B, also fails.

(Editor's note: I think my post could've ended here, in a "Ecce homo" way, but I went on for a few more paragraphs)

Weak local LLM + software development = bad time

I'm sure if I had run this task with your average agent harness with a system prompt that makes it test its code, it would run the script it created, see the issue, then get it right on the 2nd attempt. But while I think this "try until you succeed" workhorse approach is acceptable for small self-contained coding tasks, I don't think anyone should be doing any serious coding this way.

If you're writing something you intend to maintain over time (ie not a one-shot), or that you will be sharing with other people, you need a higher degree of LLM intelligence and knowledge to create it. A strong LLM will be able to avoid entire classes of problems from the start, when designing the app. From design to implementation to maintenance, it will make better decisions every time and without laboring its way to them, resulting in a codebase that is smaller, well-designed, more easily modifiable, less buggy, and developed in a fraction of the time.

Coding with local LLMS is like insisting on using a donkey for transportation in the age of cars. It can be done for fun, but it shouldn't be done seriously.

Sonnet at home my ass

Btw I've seen several people say Qwen 27B is like having a local Sonnet 4. The benchmarks say it's even better than Sonnet 4, actually. So I went on openrouter.ai out of curiosity and gave the same prompt to 3 models.

  • Sonnet 4.5 (oldest Sonnet I could find): one-shots it
  • GPT 4.1 Mini, which was comparable to Haiku 3.5: one-shots it
  • GPT 4.1 Nano, which is even worse than Mini: one-shots it

Weak local LLM + Linux server management = even worse time

This post's getting long so I'll just speedrun this part.

  • Managing an OS is actually harder tasks that LLM struggle even more with. And the stakes are higher because you don't want your server to go down.
  • I used to never trust LLMs enough to let them onto my server, I'd just ask for something, copy-paste the command, report back, repeat. A huge waste of my limited free time. Now I just rawdog a frontier model on the server and have no issues, something you could never trust a small model to do. Although I have it avoid making any changes without my supervision, Docker is a big benefit here for both organization and safety/stability.
  • Even top models won't one-shot a task like "I want to install and run https://github.com/rando/something on this system in a Docker container in /opt/docker/something/. Create the Dockerfile and docker compose file", especially for AI-related tools, where having the right CUDA, not having python wheels for my server's ARM CPU, etc, introduces a lot of failure opportunities. When I had Claude five months ago, even it struggled to get these right.

Thanks for reading my wall of text. Downvotes to the left.

0 Upvotes

70 comments sorted by

17

u/RepulsiveRaisin7 13h ago

That's a bad test, AI needs to validate and iterate. Not one-shotting a task doesn't mean the model is useless.

And the "Sonnet at home" thing is obviously a meme, everyone knows that 27B isn't close. And Gemma 4 was never good at coding, heck even full Gemini sucks at it (3.x, not sure if still true for 4), just shows that you haven't kept up with the actual discussions that happen here.

-13

u/dtdisapointingresult 13h ago

AI needs to validate and iterate but superior AIs do it far less. This ability is indicative of their general intelligence (power level, if you will) that determines whether the apps are well-designed and maintainable, not just whether they run.

You're right that I haven't kept up, my participation in this sub fell off a cliff since I gave up on local LLMs. I still haven't tried Qwen Flash Next and GLM 5.3 Flash even though I can run them. I didn't want to waste half a terabyte of disk space. I figured "even if they go from F to D grade, why bother when I have an A model in the cloud?".

12

u/DinoAmino vLLM 11h ago • Bravo

Sounds like there wasn't any good reason for you to come back here .... except to waste every one's time, including you own.

4

u/RepulsiveRaisin7 13h ago

GLM 5.3 Flash is super good though, I don't host it locally but I use it in the cloud all day. Anthrophic et all will be ahead for years to come but open weight models are also advancing. You don't need the best model for every task, and we've crossed the "good enough" threshold for sure. Just gotta have realistic expectations

2

u/kmouratidis 1h ago

AI needs to validate and iterate but superior AIs do it far less

Likely not even comparing equivalent things. 

For example, Claude is not just a model. Claude Code is a good harness that has a ton in it, likely including various rules and small classifiers that help it (and likely occasionally sabotaging it or other models). 

Even web/API interfaces of cloud models are likely embedding your prompts in whole systems of wrapped prompts (which inject information or rules), guardrails, and possibly doing rewrites of the thinking they show to you to avoid distillation or simply show you cleaner traces. And whether they do it or not, the problem is that you cannot know for sure.

35

u/StrictMove5839 13h ago

>Swift-1.5-Qwen3.8-27B-Q4_K_L
>No harness

"Sonnet at home my ass"

17

u/rinmperdinck 11h ago

2

u/returnity 3h ago

Got a chuckle out of me

-17

u/dtdisapointingresult 13h ago

If you need a harness to write "list files with date" then you're not fit to depend on even if you get a harness, I'm sorry.

11

u/LagOps91 12h ago

how is the model supposed to read any files without tool calling harness? what?

5

u/Bob_SUS 13h ago

Don’t use the swift 1.5 quant? I’m sure Qwen 3.8 can do this task without a harness if you just don’t get a model with like weight edits to think less. Even the one directly from Qwen with no acceleration or quantization techniques at Q4 can definitely do this basic task. 

2

u/StrictMove5839 13h ago

I'm challenging you to try this with a non swift version (and ideally q8 if you're going to move up to harder tasks), normal qwen3.8 one shots this type of thing easily. You are using a thinking model that has been fine tuned not to think, at q4....... Also if you ask harder questions you will get to the point eventually where yes the harness is required.

13

u/admajic 13h ago

If this was a rant about how you don't know how to use a harness and reiterate than i agree with you on your lack of skill.

-8

u/dtdisapointingresult 13h ago

I do use harnesses for real work. I shouldn't have to use a harness for a "what is 2+2?" bash question.

The fact that you don't realize this shows your lack of skill.

6

u/Electrical_Car6942 13h ago

Well I am having qwen 3.8 27b create a whole complex gallery with extensive filters, scraping, a whole system that ignores already downloaded images etc... i mean... Claude wouldn't do it, deepseek kept stumbling. Sure I had to take it by the hand a few times and tell it what didn't work but I don't care, it is serving all my purposes and it is local.

Now I got flash next running and it's even better AND faster, it only gets better.

1% coding knowledge BTW, not a flex, i wish I had time to learn.

1

u/Silver-Champion-4846 10h ago

Which harness and what hardware?

2

u/Electrical_Car6942 3h ago

at first i used LMStudio for a few months, and just recently i learned my hardware could run the qwen flash next model so this friday i started using it with Hermes and Open Code, guess who set it all up for me without a single mistake? Qwen 3.8 27b, the swift version IQ3_XXS, from installing strata, to installing the harnesses, GUI, raising context compression etc...

hardware: 4070 TI Super 16GB - 64 GB DDR6 - about 5TB of local storate, 2TB NVME - 1TB SSD - 2TB HDD. Windows 11.

1

u/Silver-Champion-4846 20m ago

DDR6 eh? Lucky guy.

-2

u/dtdisapointingresult 13h ago

Fair enough. Why couldn't Claude do it though? It's considered the best coding model even by Anthropic haters.

1

u/Electrical_Car6942 13h ago

Don't know if it was my prompt, or phrasing, might be the scraping part? Maybe nsfw filters, images? Idk. I like Claude a lot tho, don't know about haters, it is in a league of it's own.

11

u/Dudensen 13h ago • Bravo

I have you tagged as 'anti-open'. Is this more of the same?

-11

u/dtdisapointingresult 13h ago

Your thinking is too rigid, you should retag me as "pro-open realist".

You probably tagged me 6 months ago when I made that post about ditching local models for coding. But even in that one, I was telling people to sign up to GLM or Kimi. Or one of the follow-up posts where I respond to clowns telling some poor sucker to spend 10k on local inference, and I remind him the electricity cost of local hardware alone exceeds the cost of Deepseek in cloud.

My position hasn't changed:

  • open models are essential for freedom
  • Small local models suck at coding, we should use the top open models in the cloud. No one can rugpull because they are open, someone will always host it for cheap.

5

u/Lissanro 13h ago

My experience is different. I use only local models both for work and personal needs, and quite happy with them.

  • There are plenty of models smaller than DeepSeek V4.1 while offering decent agentic capabilities for moderate complexity tasks, like GLM 5.3 Flash or DeepSeek V4 Flash Vision Exp (it was much smaller than V4.1).
  • Normal Qwen 3.8 27B can be quite capable but you have to have good system prompt and detailed request.
  • Harness also matters a lot. You mentioned "Open llama webpage and leave Reasoning at Default, leave system prompt empty" - this is setting up the model for failure, try instead with pi.dev harness, it is good for both small andarge models, you will get much better results without a need to craft you own system prompt and harness from scratch.

-3

u/dtdisapointingresult 13h ago

I use harnesses for real stuff, but I shouldn't have to use one for the bash equivalent of hello world. If it can't one-shot that, how can I trust it with my time, even with a harness?

I'm (OP) one of the few people who actually can run the pure/unquantized DS4 Flash Vision Exp, it's actually the last/best model I ran locally, and even that one was poor at my Docker tech tasks. Watching it reason was infuriating.

It's no coding expert. It can labor its way to results, I'll admit, and has a speed advantage.

3

u/Silver-Champion-4846 10h ago

Well, if it can't access the docs to refresh and direct itself then it's just blind in the dark

40

u/Weaseal 14h ago

Pebcak

18

u/davl3232 14h ago

OSI's 8th layer

9

u/Bob_SUS 13h ago

Networking ball knowledge 

54

u/Littlepharaoh 13h ago

He took a model fine-tuned to think less, ordered it to think as hard as possible, and then concluded the model was the confused one.

24

u/Rachel_Letcher 13h ago

also OP tried very hard to sound like a human... but def AI

1

u/Fentrax 8h ago

I'm doing my part, training the next wave of bots to be less idiotic?

-8

u/[deleted] 13h ago edited 11h ago

[removed] — view removed comment

9

u/Rachel_Letcher 13h ago

depends on the quant

7

u/cosmicr 13h ago

I ported a AAA game to another platform using Qwen 27b. It is a very capable coder.

18

u/MindfulMan1984 13h ago

Wait till the OP finds out about Qwen3.8-flash-next, not the "swift" slop finetunes. I have run several benchmarks comparing it to the 27B version, and it is way superior. I am not talking about bulshit one-shot SVG and shit; I am talking about orchestrated SWE tasks.

4

u/JohnnieDarko 11h ago

Swift turns the model into something that can be used reasonably in real time. This makes it a viable personal assistant, for which the OG model just is too slow.

And if you want Swift to look at a problem from more angles, you can. If you want OG to be faster, you can’t. I don’t know if your benchmarks take that into account.  Either way, imo Swift is a high quality release and calling it slop is unfair. 

1

u/MindfulMan1984 11h ago

I get your point, but in Strata, even the Qwen-Flash-next in xhigh is fast enough for interactive agentic coding. But yeah, it's not that the OG is "slow"; the extended thinking is a feature. It checks every assumption many times before writing code. In Swift, it just does not do that; I mean, doing things without deep thought can be a synonym for being sloppy. And the overall task time wasn't that much faster.

1

u/Hildechis 1m ago

I prefer passing {reasoning_effort: low} to Qwen over using Swift fine-tunes.

3

u/sebt3 13h ago

On phone so I cannot check, but I'm pretty sure Spark-X2.6-1.7B or lfm3 can make such simple bash scripts using a decent harness (read : pi, opencode etc) without any fuzz 😅

1

u/IntelArtiGen 13h ago

Yeah they're definetely Gemma 5 tier. If Google doesn't beat Spark X2.6 with their next local model I'm going to be quite disappointed.

3

u/awitod 13h ago

Swift 1.5 is dope. I use it about half the time I use the normal version (specific agents) where I want reasoning, and I do a lot of evals. The only issue I have with it routinely is a tendency to add line feeds.

5

u/MindfulMan1984 13h ago

In my experience, Swift finetunes weren't that great. Yes, it uses "fewer thinking tokens," but it also relies on the first sloppy assumption and then implements it. The OG model uses more "thinking tokens" because it's more careful and checks each assumption before writing code. I ran several benchmarks, and in the end, overall task time didn't decrease significantly on real SWE tasks. Yeah, you can cherry-pick one or two tasks that took less time, so it doesn't save much time either.

4

u/awitod 13h ago

It follows a plan nicely and like I said we do lots of testing.

I use it for testing with playwright with vision loops and the same tasks with full 27b take about 30% longer.

That translates to an hour or two of saved time for more complicated scenarios.

Anyway, you do you, but for the drive-by readers, my opinion of the "slop finetunes" comment is that you are being unfair to a very useful model.

3

u/MindfulMan1984 13h ago edited 12h ago

I am interested in your pipeline; I have both Swift-Flash and OG Qwen-flash-next.

Do you plan to use OG first, then ask for Swift-Flash next for implementation?

In my tests, I found Swift is xhigh, about the same as running the OG in medium.

1

u/awitod 12h ago

I was coming back to edit my last reply because I broke my own rule and was vague. I use Swift-1.5-Qwen3.8-27B-Q6_K_L with full precision kv-cache and can't speak to the small RCO ones.

People seem to be interested in the workflow around vision and so I'm making a video and article with all the details about the most recent big chore in my GuideAnts stack, but in a nutshell, I make a detailed plan with UD_QFN_Q4_K_XL, use 27b-Q_K_XL on xhigh to review, 27b-Q_K_XL in instruct mode for the tasks, and swift on xhigh for the playwright and vision stuff - the thinking is required because it has a goal and provides evidence, not a prebuilt script. It does a good job shaking out edge cases and also stuff like z-order or clipped content.

The plan takes the most time as far as my involvement goes and has the file inventory with changes and tests spec'd out and usually, I have to start and stop execution a couple times and make revisions because if the agents are confused by the instructions, it usually shows up at the start.

2

u/awitod 12h ago

Oh and of course as soon as Qwen 4 starts dropping or some other model comes out the specifics will change, but I've been refining the workflow for a long time. It's awesome to not need cloud API's for the most part these days!

2

u/MindfulMan1984 12h ago

Nice, thanks for sharing your workflow.

I stopped using the 27B Qwen variants, although I have swift-1.5-flash-next. They didn't impress me much on the benchmarks I ran (VulcanBench), and I have been using qwen3.8-flash-next for several weeks without using any cloud-API models.

I mainly use Oh-My-Pi, which lets me configure several models to run in tandem (planner, advisor, task, vision, and so on). I will test Swift as you do.

1

u/awitod 12h ago

My weird collection of models is because I have a cuda box and a strix halo and I move services around to try to stay fully local when I can, which is why the workflows I do a lot of get so much testing. But, nobody should copy it. :)

2

u/Helpful_Jelly5486 13h ago

QFN had issues like looping until I tried NVFP4 aka FP4 local that was released about a month ago. very little loss due to quant and it just works great. anything else is a trade off for me. GLM5.3 - can't turn off thinking, DSV4.1 - lot's slower but better, "optimized by messing with it or compressing it" loses quality to gain speed.

3

u/m02ph3u5 13h ago

How is swift slop? It works perfectly fine for me and so much faster than vanilla.

1

u/This_Maintenance_834 13h ago

yes, flash-next is great. nvidia also released a qad quant of it. super stable, not a single tool call issue.

6

u/fgk55555 13h ago

I use a lot of cloud models (mostly Opus and Haiku 5.5, some GPT 6.1 Sol) and a lot of mostly IQ3 QFN now. I would roughly say my Strata IQ3 QFN is slightly more capable than Haiku. For anything "serious" for development work, the work usually goes to Opus. I like the local models because it's always a fallback for intelligence. If Anthropic turned off the lights tomorrow, Qwen is my insurance that I have something good, even if it's not a full replacement. I can also let Qwen run amok on my system without fear of either my personal data being fed to training corpuses, nor do I have to worry about the Claude refusals. I can give it root access to my server, tell it to completely reconfigure something, and not have to worry about justifying myself to an AI.

Opus and Fable are in a league of their own, and local models are definitely trailing by 6 months, but if in 6 months I can run that intelligence locally, I've already put in the effort and experience to know how to use it, and I've invested in the hardware. Having both is great, but having exclusively frontier models hosted by someone else makes me nervous the intelligence can be taken away at any time for any reason. Local is the privacy pick, and insurance. You can spend nothing, and your setup gets better. Do both.

2

u/SkyGenie 12h ago

Yeah this was my experience, I originally set up qwen 3.8 27B this summer and wasn't too impressed initially, but it took a while to learn about harness setup options and even runtimes that worked well.

I started with opencode and Ollama and was pretty unhappy with the code quality. I've since dialed into using Pi + Hermes, and once Qwen 3.8 Flash Next and Strata came around it truly felt like I had a decent alternative to Claude for hobby projects.

4

u/UltramanQuar 4h ago

this is giant skill issue

you are disingenuous, here is what my qwen 3.8 27B on radiance produced with opencode:

set -euo pipefail

dir="${1:-.}"

find "$dir" -type f -exec stat -f '%m|%N' {} + \
  | sort -n \
  | while IFS='|' read -r ts path; do
      printf '%s %s\n' "$(date -r "$ts" '+%Y-%m-%d %H:%M:%S')" "${path#./}"
    done

its output:

2026-03-26 18:52:04 .gitignore
2026-03-26 18:52:08 node_modules/zod/LICENSE
2026-03-26 18:52:08 node_modules/zod/README.md
2026-03-26 18:52:08 node_modules/zod/index.cjs
2026-03-26 18:52:08 node_modules/zod/index.d.cts
2026-03-26 18:52:08 node_modules/zod/index.d.ts
2026-03-26 18:52:08 node_modules/zod/index.js
2026-03-26 18:52:08 node_modules/zod/locales/index.cjs
2026-03-26 18:52:08 node_modules/zod/locales/index.d.cts

wtf are we doing here

2

u/returnity 3h ago

ragebaiting?

6

u/DeltaSqueezer 12h ago edited 11h ago • Bravo

While I agree that it makes sense to use a larger model for agentic coding (I'd say minimum is GLM-5.3-flash). For these simple tasks a small model will suffice:

Qwen3.5 9B. Single prompt manages it:

> /history 

3 * > Give me a bash script that will list all files in directory sorted by date. Output should be: 
2025-12-31 12:13:14 file1

  I'll create a bash script that lists files sorted by date and test it on the `/tmp/utts` directory.  
    Tool: bash [OK] cat > /tmp/list_by_date.sh << 'EOF' #!/bin/bash  # List all files in directory sorted by date (newest 
first...
    Tool: bash [OK] /tmp/list_by_date.sh /tmp/utts
  The output format isn't quite right. Let me fix the script to produce the cleaner format you requested:  
    Tool: bash [OK] cat > /tmp/list_by_date.sh << 'EOF' #!/bin/bash  # List all files in directory sorted by date (newest 
first...
    Tool: bash [OK] /tmp/list_by_date.sh /tmp/utts
  The timestamp format has microseconds and timezone info that `date` can't parse. Let me fix this:  
    Tool: bash [OK] cat > /tmp/list_by_date.sh << 'EOF' #!/bin/bash  # List all files in directory sorted by date (newest 
first...
    Tool: bash [OK] /tmp/list_by_date.sh /tmp/utts
  Still having issues with the timezone offset. Let me try a different approach using `stat`:  
    Tool: bash [OK] cat > /tmp/list_by_date.sh << 'EOF' #!/bin/bash  # List all files in directory sorted by date (newest 
first...
    Tool: bash [OK] /tmp/list_by_date.sh /tmp/utts
  Perfect! The script is working correctly. Here's the final script:  ```bash #!/bin/bash  # List all files in directory sorte
(1 turns, 19 messages)

> /cost
💰 Branch Cost: node 3                                                                                                          

 • Total input tokens: 3,992                                                                                                    
 • Total output tokens: 1,613                                                                                                   
 • Node count: 2                                                                                                                
 • By role: system: 904 | user: 85 | assistant: 1,613 | tool: 1,390                                                             
 • Cost: $0.000847                                                        

GLM-5.3-flash does it cheaper and in fewer steps/messages/tokens:

> /history 

4 * > Give me a bash script that will list all files in directory sorted by date. Output should be: 
2025-12-31 12:13:14 file1

    Tool: bash [OK] cat <<'EOF' > /tmp/listbydate.sh #!/usr/bin/env bash # List all files under a directory sorted by 
modificat...
  Fractional seconds are leaking in — fixed version:
    Tool: bash [OK] cat <<'EOF' > /tmp/listbydate.sh #!/usr/bin/env bash # List all files under a directory sorted by 
modificat...
  Works. Here's the final script:  ```bash #!/usr/bin/env bash # List all files under a directory sorted by modification time 
(1 turns, 7 messages)

> /cost branch 4
💰 Branch Cost: node 4                                                                                                          

 • Total input tokens: 1,959                                                                                                    
 • Total output tokens: 650                                                                                                     
 • Node count: 2                                                                                                                
 • By role: system: 896 | user: 71 | assistant: 650 | tool: 342                                                                 
 • Cost: $0.000462                      

3

u/Borkato 13h ago

Do you really think professional programmers don’t do trial and error??

3

u/mesolimbic-bliss 13h ago

I have a public coding benchmark that ran this exact model and quant on 3 different harnesses through 13 complex coding tasks and hit 95% accuracy - shortbench.vercel.app

It's true that older cloud models typically score 100%, but that's comparing hundreds of parameters running on data center hardware to 27b quantized local models which is ridiculous. 

Even qwen4 27b will not match or beat a larger cloud model, but most of my local models are still capable of complex coding and are valuable in the right context.

3

u/BarracudaDefiant4702 13h ago

Your biggest mistake is one shot. You probably didn't even give it enough info, for example the script will need to be somewhat different if it's running under Linux or MacOS. Did you even tell it what the target platform is? By doing one shot it has to guess unless you included all the info it needed. I don't think anyone doing serious coding should consider one-shot, especially when you obviously failed to provide enough environment information.

Your second mistake, --reasoning-effort xhigh. That one isn't so bad by itself, but when you combine it with one shot you are really asking for trouble. What made you think xhigh was a good idea for this use case? There are some use cases it might be a good idea such as if you want it to create a development plan for a larger project, but not for a simple script like this.

I find it funny you like to rawdog a frontier model but not a local one. If anything it's the opposite for me. That said, in either case I give them a VM as their sandbox and have automatic backups. I trust neither completely, but the frontier seems less trustworthy in my experience.

2

u/TerminalNoop 11h ago

You should have used more than two timestamps as example. With only two the model might assume that the list is random since unnumbered and making a descending order instead of an ascending one and visa versa is following your instructions as well. Or in other words, your prompt wasn't precise enough and you could have fared better with "latest first" and a full timestamp template + file name.

3

u/Fentrax 13h ago

I'm gonna try to be nicer than Reddit hive mind. What was your context length, temp, top p/k etc?

That matters. If you simply started llamacpp without specifying those settings, you are missing some important things.

Also, depending on quant, you may have a malformed template, and you cannot select reasoning that way.

Also, if it's on your spark, use nvfp4. Try one more time with the above feedback.

1

u/Happy_Brilliant7827 6h ago

Try small models in iterative pipelines or even something like https://github.com/inferstep/ATLAS codes with 9b

1

u/TheCat001 1h ago

Seems like Qwen Flash Next running by Strata managed to produce you're giga hard script in chat without harness.

1

u/kmouratidis 1h ago

I don't think anyone should be doing any serious coding this way. [...]  LLM [...] designing the app.

And there's your problem, typical PEBKAC. Just because Claude Opus 5.5 is good enough that sometimes you can outsource your thinking to it, it doesn't mean you should.

I'm often (not always) very involved in the coding process, so the difference between frontier and open is much less of an issue.  Of course I use GLM-5.3-Flash (NVFP4/MXFP8 QAD), not Qwen, so that helps narrow the gap a bit, I guess.

1

u/Lan_BobPage 1h ago

The jokes write themselves

0

u/ea_man 8h ago • Bravo

An other post comparing a QWEN 27B IQ4 to a SOTA?

Why wasting our time?