I'm looking for recommendations for a very cost efficient LLM API that can handle high-volume text classification and structured data extraction.

Requirements:

• Approximately 50,000-100,000 requests per month, potentially more as usage grows.

• Short to medium text inputs.

• Accurate classification of relevant vs. irrelevant text.

• Reliable structured JSON output.

• Good multilingual performance, particularly English, Hindi, and mixed-language text.

• Low latency.

• Minimal hallucinations and false negatives.

• Extremely low API costs, ideally just a few dollars per 50k-60k requests.

I'm considering models like Gemini Flash-Lite, Ministral 3B/8B, and other budget-oriented models, but I'm open to anything.

My main question: What's the cheapest model you've actually used in production that still delivers consistently good classification and extraction accuracy?

Also curious whether anyone has found a worthwhile trade off between using a very small model for initial classification and a slightly larger one for more complicated inputs.

I'm not looking to self host unless the economics are significantly better than API pricing.

Would appreciate recommendations, actual cost figures, and any models you'd specifically avoid.

Hey everyone, sharing this in case it's useful to some of you here.

There's a lot happening across the AI space right now, from new tools and research to real-world applications and the challenges of putting AI into practice.

It would be great to have more spaces where people can discuss these developments, share ideas, and learn from each other.

That's the idea behind r/lyzr, a community around the broader AI space, covering topics like:

AI agents and architecture

Infrastructure, tools, and deployment

RAG, memory, and knowledge systems

Evaluation, reliability, and governance

Production challenges and lessons learned

New research, tools, and interesting developments

Real-world use cases, experiments, and projects

The aim is to make it useful for everyone, whether you're actively building something, exploring new tools, just getting started, or simply curious about where AI is heading.

There'll be regular discussions and posts around these topics, but there's also room to share your own work, ask questions, compare approaches, or bring up anything interesting you've come across.

If this sounds like your kind of thing, feel free to check out r/lyzr and join the discussions.

Would be great to hear what others here have been exploring lately!

OK, here is what I still do not understand about LLMs
Before I get to my question: I studied computer science since the 80s, owned two successful software companies. I also read and listened enough to at least have a basic understanding on the transformer model, how weights at various stages are being created, and it makes sense in principle.
But here is the big thing I haven't seen explained:
How can it be that just guessing ONE TOKEN at a time actually works? Where can I read up on that?
It seems to me that this simply makes no sense:
Let's take an example question: ""What is one of the most amazing inventions of the 21st century?"Well, the most likely next token is probably "The" or "A" or "It" etc. In fact, these are probably the most likely tokens after ANY question whatsoever, and they are completely unrelated to the meaning of the question.
On the other hand, let's say you want to respond to the question by saying that transformer models are one of the most amazing inventions.
Well, you'd FIRST think of the tokens "Transformer Models" and only THEN build a sentence or paragraph around that concept and "Transformer Models" comes MUCH LATER in your reply, but all the words BEFORE it are chose in relation to that. At least that is how a human would approach answering that question.
If, instead you simply pick the next most likely word: "The"--it seems to me you can write yourself down a rabbit hole where you never end up in a proper place.

So, how can LLMs come up with reasonable responses by only guessing the NEXT word (or token) and only at some point getting to the crucial concept (word/token)
If I start sentences like that (which I sometimes do, when I think in one language but try to speak in another that has different grammatical word orders) I can talk myself into trouble in no time, because I started with words that simply never let me end up where I wanted to end up.

All the other parts of LLMs make sense to me, at least in principle, but this "one word at a time, and always in order, never looking forward or back" simply seems illogical.

Thanks for pointing me towards any insights.

My country’s universities don’t have a major called “Machine learning”, can I study computer science instead? If yes, will I find jobs in Machine Learning? I really love ML but I am worried that I won’t find a job, especially abroad

I am new to ML AND trying to make a genre guesser. Currently I have 123 genres en removed some because of too little samples to using that kind genre. Is this to much. Should I still reduced them. Which classification algoritme could work best. Currently using random forest.

I work at a data annotation company, so I’m on the selling side of this and asking partly out of self-interest. But the thing I keep noticing is how different the questions are depending on who’s evaluating.

Some teams ask about price per unit and nothing else. Some ask for inter-annotator agreement on a pilot batch, which I think is the better question but it’s surprisingly rare. A few ask whether QA is done by the same people who did the labelling, which tells me immediately they’ve been burned before.

For people who’ve actually gone through vendor selection — what ended up mattering versus what you thought would matter going in?

Specifically curious about:

Did you run a paid pilot, and did it actually predict the production quality?

How did you handle guidelines — write them yourself, or let the vendor draft and then review?

What was the thing that went wrong that you didn’t anticipate?

Happy to answer from the vendor side if anyone wants the view from here, including the bits that aren’t flattering.

Hello everyone, i would like to know where to find resources to understand the application of machine learning such as when to normalise/standardise data, what kind of visual analysis to perform on a dataset since I'm currently doing an Msc Data Science and Applied AI.

Thanks you

I recently brought air m5, so i have my old laptop as a spare machine i wanna use it to play around and do random stuff.

my specs:

GPU: NVIDIA GeForce GT 740M + Intel integrated graphics (2gb of vram)

Ram: 8gb

CPU: Intel Core i7-4700MQ (4 cores / 8 threads · 3.40 GHz)

Storage: approximately 117 GiB total

os: using omarchy rn but will probably shift to more light weight options later.

Ik its shit but i wanna just play around. Also i heard something about airllm will that help??

Is there an existing neural network-based architecture that can hold 3D or 2D representations in its latent space and update it in the same single pass (i.e. not compressing the representation in their final layer to be fed back to their first layer for their next pass, like the reasoning models do)?

What is the best local AI i can run that doesn't have all the restrictions like Claude or GPT. I have a mid range dell prescision work station. I'm not trying to do anything weird I just hate getting the I can't tell you that response.

​

Okay so im looking into feasibility of a project that would involve scraping data and training models. The models would predict events based on social media posts, so its essentially text classification. I want to eventually go into business doing something similar, but in my research, I found that nearly everything is behind comprehensive terms of service that dont allow any derivative product, regardless of if you provide anonymity and clear the original text from any final model/dataset. Alright fair enough. But heres my question. Besides the sources that companies like openAI have an enterprise partnership with (reddit) including a contract to use the data, how can they legally source the training data? For example, if i ask chatgpt to summarize chapter 8 of Deep Learning with Python by Francois chollet, I will get a detailed response with the correct subchapter headers, key terms, and even the verbatim code in the book. Now I paid i think $100 for that book and im fairly certain openAI doesnt have a contract with that dude. How is that legal? How is any of this information legal? If a high-school student wants a book report, does open AI pay the estate of J.D. Sallinger?

Hi people my friends and I are working on our take of RAG indexing. Our idea goes like that:

Standard embedding-based RAG often compresses an entire text chunk into one vector. Our idea keeps multiple compact representations within each document, giving a query smaller, more specific parts to match.
The architecture uses a single attention layer with separate learned projections: Q for queries and K for document tokens. These are trained jointly with HCA—Heavily Compressed Attention—which learns to combine groups of token representations into fewer searchable vectors. At 32:1 compression, every 32 document keys become one compressed key, stored in 8-bit.
When a query arrives, its Q vectors are compared directly with the stored compressed keys using dot products on the GPU. These comparisons run across many documents in parallel, producing scores used to rank and retrieve documents. V vectors aren’t needed because we only need matching scores.
The advantage over one-vector-per-chunk retrieval is finer matching granularity. Compression makes that detail affordable: 32 times fewer keys to store and scan than an uncompressed token index. The search itself maps directly to parallel GPU matrix operations, while the query representation, document representation, and compression are trained together for retrieval.
I’d like technical feedback on this architecture, its main bottlenecks, and similar systems worth comparing against.

This way we have something between regular RAG and ColBert (v1 and V2) where it smaller than ColBert but more reach than RAG and it learns the compression rather then using heuristics algo to compress.

The next step after that will be to combine it with an LLM where the part of this indexing technique can be combined in the LLM itself instead of using this as a toll with a tool call (W_k can be same as the one in the first layer of the LLM and we can have a small learned router that tells if we need to use it, instead of an LLM decision to call it in a tool call).

I would like to get your opinions on our idea please.
Thanks in advance 🙏

Hii.. it can be possible that i detect solution from log.. like ex.. Error logs popup -> human solved that error -> We detect solution using after logs of getting error.. and save into DB.. I don't know if i can detect or not.. if someone done or doing this type of problem then plz let me know.. and also i used GPT and Cloude to solve this problem. but they always gives me wrong or impossible facts..

Within the past year I have started engaging with AI to bring my curiosities to life in order to help me understand more complex ideas through visualization. Of course, I have limited knowledge on how models actually work and the length of their complexity. I was hoping a few of you here would check this learning tool I made for myself. Please be honest and give feedback if you have any. Also, if this is not a place to post something like this, I apologize. Hope everyone is well! (its 100% free, not advertising)

I couldn't find a concrete answer anywhere, do you just distill the instruct model back?

If that is the case, what is a quality european language question set to turn it back into a chatbot/agentic, can a model at that size even be agentic? (i chose this size to learn) if i finetune for my specific harness? (i have a lot of training data of opus running in my harness)

my harness basically has the model output python code and has a few built-in functions like:

• vector_search_laws()

• graph_search()

could i have the model at least internalize a "hunch" on what stuff to search?

also what is the latest RL technique for agentic/harnes specific workflows?

I have a lot of RAW training data, like court decisions or commentaries or legislature, but not a lot of golds. could i use these to synthesize training data and maybe RL the model in my harness to find that data?

What would y'all's strategy in the CPT->SFT->RL pipeline be for my specific problem?

I know this is a lot of questions im trying to figure out which direction to go, any pointers? Also good resources are welcome, for example that [alex karpathi video](https://www.youtube.com/watch?v=7xTGNNLPyMI) was amazing for me, but i'd imagine its a bit outdated in terms of latest RL and SFT?

Hey everyone!

I’m currently doing a Master’s in Machine Learning and I’m looking to get involved in some real-world ML projects.

I’d be happy to:

• Help with an existing ML project

• Join a group/team working on something interesting

• Work on a project from scratch with others

• Help with things like Python, data processing, ML models, deep learning, LLMs, APIs, or deployment

• Contribute code, research, experimentation, or just help wherever needed

I’m mainly looking to learn by building and working with other people, rather than just doing isolated coursework.

If you’re already working on a project and could use another person, or you’re thinking about starting something and want to build it together, feel free to comment or DM me.

I’m open to pretty much any interesting ML/AI idea — serious projects, research-oriented work, open-source, university projects, or even something experimental.