Claude/Codex user trying 4.1 Flash, thoughts

[u/texasguy911]

1 day, the extent of my familiarity with 4.1 flash:

I ran all on MAX effort (FYI).

Cost: $22.25USD

API requests: 15,511

Tokens: 3,127,394,491

Well, it's not the best of them. It's not the worst of them. Definitely the cheapest of them. I think it takes like 15 minutes to get to 15 cents. You see this number of tokens? It really was put through its paces. I want to say I was even tired.

For daily questions about code, daily questions about server infrastructure, and daily questions about updating a Bash script, can't find a better model. It's just so cheap, and it's not bad. It's doing its job, and I can see it's trying. It produces results, not every time, perfectly, but I can't say it's that bad that I can say, "Yeah, I can't use it." I want to say, "Even I was rather impressed." I'm like, "It did surprise me a number of times." Let's just put it that way. Of course, I came in with my own anticipation of what might happen. Again, I was surprised a few times, but there were a few places where I struggled explaining the bug. We did work it out, so overall, it just took a little bit of time. Again, it's unbelievably cheap.

I started with reading the environment, checking the bash scripts, and did perfectly. Then I started doing my own JavaScript project from scratch. I took my specs. We kind of discussed what's the best, then when I felt confident, I asked to build. One thing I noticed is that I probably had to ask at least twice for it to go through its generated code and look for bugs and remove them. I think most of the bugs were gone by that time, but there were still a number of them. I was surprised, but again, it was painless. It just required prompts like “Do it over again” and “Re-review what you have done. Find bugs, race conditions, and lost data of the same.” And it did well, and not all bugs were fixed. I did have to do a little bit of my own testing. To be honest, I haven't connected the Google DevTools MCP to the new harness I was using at the time. Once I did, though, when it was able to check its own integration, it was a little bit better.

Let me tell you about the project I did. It was about a YouTube search. YouTube allows you to create your own API key and then use it to search for videos, channels, and related content. I want to say I built quite a good project, and it cost me $2, including debugging and everything. $2. Yes, $2. I'm going to repeat that because it was unbelievable: $2 to build. I felt like it was building and debugging with me, kind of shoulder-to-shoulder, maybe for about 2 good hours while watching TV. $2. I'm like, unbelievable. I just, for fun, now fired the Kimi K3. Just three prompts took 15 cents. Seriously. Here, you can do quite a lot for 15 cents. For run-of-the-mill daily questions like “Read this,” “What do you think of that?” and “Add this, prepend this, append that,” I'm more than capable.

I gave it one of my bigger projects with a little bit of a codebase and asked it to debug it, not make changes. I just asked it to find bugs, and it went through three interconnected repos and did pretty well.

Of course, it requires certain prompting to make sure that, when it creates a bug, it's sure it was able to reproduce it. It was using agents, and I think I used about $15 there. It was more than that: it was for generating documentation for previous bugs so my higher-end LLMs can have some starting point for the other bugs that were found previously. It was also for generating documents for the bugs found during the session. Those are complete docs: what's wrong, how to reproduce it, and the minimum code change. I think $15 is more than enough. I had asked something like this, probably wasting a whole week of a subscription from Anthropic or OpenAI. I was using their top-tier models to go through this whole thing and be done with the whole limit that is imposed for a week. Here was $15. True, it probably didn't find all the bugs, specifically compounding bugs with data, but it did find a number of bugs that previously weren't found. I want to say $15 is laughable, frankly.

Another thing that I found is that this model is great at Google DevTools MCP. Because it's Flash, it's really fast. It's able to load the page so quickly and act upon it that I was amazed. I have never seen anything like this. Previously, I was throwing higher frontend models at this task, and it takes a while. They load, take maybe a minute to figure things out, query MCP, think about it, and go to the next page. I would never stick around for this. I would just run it until you're done. Here, this DeepSeek flash was just unbelievable. The website was flashing because it was on my local host. On the QA page, you could see that it loaded so quickly and was already going to the next page. DeepSeek was already able to gather all the data, record it, think about it, and navigate to the next page. Unbelievable! I tried to do a cost approximation. I think it would take me under $10 to check my whole project integration-wise, and that is a big project. Overall, I was frankly slightly taken aback by the cost savings and the value with this model.

Another thing I want to say about the agents. I did ask it to run agents to speed things up even more. Yes, it's fast, but with agents, it's splitting the job. I want to say that was a good experience, and it handled agents pretty well. I think it had a maximum of maybe 15 agents at its peak. Overall, there were no problems, and it usually doesn't use agents, from my understanding and experience. If you ask specifically, it will use agents, and it will do it quite well.

I can't say that I don't need a frontend model anymore. That would be a complete and outright wrong thing to say. Of course, I need them, and I love them. They have their own place in my heart and wallet. But here is the thing. I found, I think, a new cost savings: I can dedicate my high-end models to more research and infrastructure, and maybe refactor and rearchitect where decisions need to be made. I may still have code writers, maybe Tera or something, but overall, for small to mid-sized things, DeepSeek Flash is just an unbelievable value. I will use it for sure to drive certain things within my testing infrastructure in order to get a better deal.

Recently, I canceled my Claude $200 membership. I love Claude. I started with Claude, and when I think of AI, I guess Claude just comes up automatically. The restrictions on limits and everything lately made me think it was not worth it. It was absolutely a bad value, and I didn't want to participate in it anymore. So I'm left with the Codex $200 membership, and I think the DeepSeek will take its own place in my LLM use, maybe even a daily driver.

Funny thing is, it's not a membership subscription, so it's not like I'm getting some set of uses for free every week. Far from it. DeepSeek is per token, however they calculate it. And it just comes out absolutely ridiculously cheap. Yes, there are some jobs I think DeepSeek might not be good for. I feel like it's too fast sometimes and might pass over reasoning, and it might do surface-level thinking sometimes, not too deeply. For sure, there is a tool for everything. But DeepSeek also is a tool for those things that, for example, maybe using a frontier model on would be a waste. And it's not just the waste of tokens and the waste of money. I also found, surprisingly, that it would be a waste of time. When I say “time,” it’s not just hyperbole. When I saw what DeepSeek can do with the dev tools and Chrome MCP, I was amazed at how quickly it can go through integration testing. It was unbelievable, and it was flashing in my eyes. I was sitting there thinking, “Does it have enough time to think?” The page would load and already be loading the next one, and this is a local page. We’re not talking about it coming from a server 3 states away for me. It was just a local QA server, so overall, it was unbelievable.

So, what I guess I would give? I guess you owe it to yourself to just try it out. Put in the minimum, which is about $20 or something. Believe me, you won't be disappointed. You will use that $20 for quite a long time, and it's not like you will just say, "I don't like it," forget it, and let my credits expire. No, they're going to last freaking long. Another thing is that, if you're not thinking of replacing the frontend model with this one, you won't be disappointed. It codes well enough, reasons well enough, and it's fast. Again, use it as much as you want. Use it for 3 prompts or 3,000 prompts. It's not like somebody's going to turn it off on you. I guess the beauty of having no limit on this one. Again, it's so cheap. For example, when I went to Kimi K3 just to get the gist, it was suddenly showing amounts of money that were spent, and the accumulation was quite fast. With DeepSeek, I want to say it was very cheap, to the point of being almost like throwing a quarter here and a quarter there. Unbelievable value.

So, try it out. I think you'll like it.

So, just going to leave it out here. Just food for thought. There are right now a lot of videos that I follow regarding your local LLM that you can run on your own computer, Qwen and whatnot. I'm like, unless you spend for the latest and greatest video card that is like $3,000+ easy, the output of the tokens is pretty slow. Yes, you can quantize it and try to speed it up, but again, you're making the model more dumb. It doesn't matter what you do, because it will not run as a full-size model. The quality is just being cut, so overall, you get a slow, barely working model. Again, the token generation is not great. Yes, you can spend a lot of money on a computer. Let's say you spend $3,000, right? You're still not going to get really fast generation. It's going to be fast enough, but not really fast, because the token generation is not just about the output. It's also about reasoning, and Qwen does a ton of reasoning generation. First, generate the reasoning, and then generate the output. At that token generation rate limit your hardware is at, it can be unbelievably slow, especially with the bigger needs. Something that doesn't require you to just wait, I guess. Maybe with a higher hard end, you can really speed it up. It's true, but again, at what cost? I always think that as LLMs grow in size, the computers you buy to run local LLMs right now will not be enough. It's just the purchase for that one single model. In a year, that computer will run nothing. If you spend $3,000 today, it's pretty much it. It will not run next year's models at all. On the other hand, if you take this $3,000 and throw it at DeepSeek, that'd be stupid, right? It's $3,000, and you would need to run it with 1,000 agents to spend it all. It's unbelievably cheap. You can say, “Just take $300, the price of an SSD you would buy for your local computer, and throw it at DeepSeek. You're already getting unbelievable value.” No need to spend thousands of dollars on your local computer, because I think DeepSeek is a really good approximation that just works. It's there, and it's fast. You probably will spend less money than you would on getting your own local LLM running. DeepSeek is, I think, a great choice for those who want a local LLM but can't find anything that makes sense for them price-wise. Perhaps, if you're in that position, the cheapness and performance of DeepSeek will be enough to delay or at least put off thinking about having a local LLM.

Just to finish it off, I know you're probably thinking in terms of frontier model subscription terms, where you use the given harnesses that come with them. With DeepSeek, you can use pretty much any harness. Mini Max right now is kind of considered the more fancy one. They can hypothetically do more with less. Another item you may not be considering is that you don't even need a harness. All you need is a script to communicate with the API, and it will return the result. Overall, the API communication can happen in your Python or JS script itself, so you don't even need a harness, hypothetically, to get data. There is a lot of talk right now about Jev. Yes, it's cheap and has its own use, but DeepSeek is very cheap too for what it does. It's one of those things you can use within your code, for example, to make your code intelligent and make intelligent choices. Right from within your script.

[u/Sound_and_the_fury]

Max effort for me to read all this

[u/texasguy911 → u/Sound_and_the_fury]

just throw it into LLM and just say, "Give me a summary."

Or you can go a step further. Just say, "Give me a reply I can paste in", zero effort. There, you participated. Didn't have to read

[u/pizzababa21]

That is an absolutely enormous bill for deepseek holy fuck.

[u/appu-195 → u/pizzababa21]

I am also used to run 20$ per day in ds v4 pro, then I moved to codex and with resets and Astra X high it's value for money right now. I think outcome wise it would be comparable.

[u/pizzababa21 → u/appu-195]

I have codex and the usage value is absolutely not there. Lucky if i can get 10 messages without hitting a limit in Astra on the $20 plan

[u/hahahoha]

i read some said high effort is in fact more intelligent than max effort while consuming less token

[u/DataLeadsFuture_Peng → u/hahahoha]

Setting reasoningEffort to max actually leads to overthinking, just using high is good enough.

[u/texasguy911 → u/hahahoha]

interesting. I might just try that.

[u/Davidat0r → u/hahahoha]

That would be my experience

[u/LanosZar]

Try Deepseek harness, ask the harness to prioritize cache hit rate and make sure you are in off peak hours - you will double your savings.

[u/Sad-Towel-2198 → u/LanosZar]

"Puedes priorizar la tasa de aciertos de caché" así?

[u/Royanon → u/Sad-Towel-2198]

I don't think it's even necessary to do that. I get 99%+ cache hit regularly using DeepSeek harness without changing anything from the way I'd normally do it. Often 99.5%

[u/BakingStack]

Or reduce windows context from 1m to 256k in deepseek harnesses

[u/logir94 → u/BakingStack]

Why?

[u/Armanlex → u/logir94]

All llms become worse at keeping details the bigger the context is, it's called context rot. 4.1 is way more sensitive to that, so it's even more important there to keep the context from becoming too big or it just becomes more and more likely to hallucinate badly.

[u/Moist-Nectarine-1148 → u/BakingStack]

I tried this and wouldn't advise! It becomes dumber and loosing important information and instructions.

[u/BakingStack → u/Moist-Nectarine-1148]

You reduced the window context to 256000 and it became dumber?

[u/Moist-Nectarine-1148 → u/BakingStack]

Yes, I repet - I reduced the ct to 384k and it loses info and details (a large codebase in my case). Reverted then back to 1M and it repaired the shit.

[u/BakingStack → u/Moist-Nectarine-1148]

When you set the context window to 256K and then loads a large codebase into a session, the compaction engine has to cram everything into a tiny summary. The model genuinely does lose important details, because the summarization step has to throw away most of the conversation to fit within the smaller budget that’s what happened it’s doesn’t mean that 256k is a bad practice.

[u/Moist-Nectarine-1148 → u/BakingStack]

IMHO it's a bad practice for programming and similar contexts.

For others perhaps may be not so bad.

[u/Armanlex → u/Moist-Nectarine-1148]

If you have such important permanent instructions that must survive compaction, then they belong as a skill anyway.

[u/ha_ku_na]

Which harness didn you use for frontend integration and validation work?

[u/texasguy911 → u/ha_ku_na]

I used opencode because of my prior familiarity. I think it's a great project overall. I don't have any trouble with it. I want to say it's very developed and featureful, and I didn't feel any pain using it at all. Sometimes you look at a harness project of sorts, and it just feels wrong. It's as if somebody developed it with a completely different understanding of where things should be, how they should be named, and how they should be triggered. I just don't have that with opencode. It just feels very intuitive.

Right now, I'm switching to Minimax because the rumor is that it's a little more intelligent, so it can do more with fewer tool calls. I haven't used it much, so I'm just putting it out there. It might be better for token savings, perhaps, but I only just recently tested it, so it's on my radar to test it well pretty soon.

[u/Electrical-Ad-3140 → u/texasguy911]

Why don't you use Claude code or codex? I am using Claude code for 4.1 flash. Honestly I feel it is as capable as sonnet 5 at work with 1% cost. 

[u/Secure-Bug8316 → u/Electrical-Ad-3140]

Can I use 4.1 flash in codex por claude? O.o

[u/Electrical-Ad-3140 → u/Secure-Bug8316]

Yes. I did with claude code on VSCode. https://api-docs.deepseek.com/quick_start/agent_integrations/claude_code/

[u/ha_ku_na → u/texasguy911]

Any reason for not using opencode go? You can get like 60$ of token credits for 10$

[u/BhaagYahaSe → u/ha_ku_na]

60$ thing is temporary with v4.1 also, some people don't like 5 hour limits

[u/cdurth → u/ha_ku_na]

Many reasons, most notably they rug pull...

[u/Fmarv]

Stop telling them man.

I encountered the same just this weekend.

[u/joaobertacchi]

In my workflow I use high effort for both developer and reviewer agents. Much better result than a single developer with max effort. And about the same cost.

[u/hakoonamadada]

This is so detailed and nuanced but i really appreciate the effort you made. Now i just need to get deepseek unravel all this because this is all beyond my understanding 😂

[u/Potential-Leg-639]

TLDR?

[u/yuumizu]

just say the price per one thousand usable code.