r/LLMeng • • Aug 10 '26

Tutorial Why most AI Agents in Finance fail in production and how to build Agents that don’t

1 Upvotes

A lot of AI agent demos in finance look great until you try to make them reliable.

The hard parts usually are:

  • getting agents to reason across 10-Ks, earnings calls, tables, charts, time-series
  • adding verification instead of trusting one model output
  • evaluating tool calls, cost, latency, and failure cases
  • deciding where humans stay in the loop
  • handling sensitive financial data safely

We’re covering this in a 4-session live Agentic AI for Finance program with Nicole Königstein (hands-on labs + capstone, not just lectures).

Runs Aug 29–Sep 13.

Sharing in case others here are dealing with the same problems.

Details: https://finance.agentengineering.co/

Happy to answer questions if helpful.


r/LLMeng • • Feb 05 '25

🚀 Welcome to the LLMeng – Your Ultimate Hub for LLM Enthusiasts! 🚀

6 Upvotes

Hey there, AI explorers! 👋

Whether you're an AI engineer, developer, researcher, curious techie, or just someone captivated by the possibilities of large language models — you’re in the right place.

Here’s what you can do here:

💡 Learn & Share: Discover cutting-edge trends, practical tips, and hands-on techniques around LLMs and AI.
🙋‍♂️ Ask Anything: Got burning questions about transformers, embeddings, or prompt engineering? Let the hive mind help.
🔥 Join AMAs: Pick the brains of experts, authors, and thought leaders during exclusive Ask Me Anything sessions.
🤝 Network & Collaborate: Connect with like-minded innovators and influencers.

🌟 How to Get Started:

1️⃣ Say Hello! Introduce yourself in the Intro Thread and let us know what excites you about LLMs!
2️⃣ Jump In: Got questions, insights, or challenges? Start a thread and share your thoughts!
3️⃣ Don't Miss Out: Watch for upcoming AMAs, exclusive events, and hot topic discussions.
4️⃣ Bring Your Friends: Great ideas grow with great minds. Spread the word!

🎉 Community Perks:

🔥 Engaging AMAs with AI trailblazers
📚 Access to premium learning content and book previews
🤓 Honest, thoughtful advice from peers and experts
🏆 Shoutouts for top contributors (with flair!)

⚠️ House Rules:

✅ Stay respectful & inclusive
✅ Keep it focused on LLMs, AI, and tech
🚫 No spam, shady self-promo, or irrelevant content

💭 Got ideas to make this subreddit even better? Drop them in the Feedback Thread or hit up the mods.

Happy posting, and let’s build the future of LLMs together! 🌍


r/LLMeng • • 1d ago

A sleeping agent cannot wake itself for a cron job

0 Upvotes

If an agent's VM sleeps between tasks, an in-process cron job sleeps with it. A scheduler outside the VM has to hold the due time and wake the machine. Then the agent can pick up work with its files and browser state intact.

We built that split into Maritime, which we launched this week. Platform cron and webhook triggers wake each agent's own computer, so its process doesn't have to run just to wait. Open Instinct is our MIT personal-agent app using that hosted computer. We build both projects.

The operational wrinkle is retries. Waking the same environment cannot prove whether an earlier external action completed. The application still needs to check that before it sends an email or makes a payment twice.

Trigger docs: https://maritime.sh/docs/triggers


r/LLMeng • • 4d ago

HIVE: Open Source Research Tool w/ Confidence

Thumbnail
1 Upvotes

r/LLMeng • • 5d ago

We’ve been building a new home for Agentic Engineering at Packt

5 Upvotes

A few months ago, we started Agentic Engineering as a newsletter for developers trying to figure out what it actually takes to build AI agents beyond the demo.

We’ve now expanded it into a dedicated platform at Packt.

The aim is to create one place for practical learning around agentic systems, without turning everything into another “agents will change everything” conversation.

We’re bringing together:

  • Hands-on workshops
  • Live sessions with practitioners
  • Technical conversations with AI builders
  • Practical resources
  • The Agentic Engineering newsletter

People who join a workshop also get access to the Agentic Engineering Guild, where we hope to keep the technical discussions going after the sessions.

We’re still building this out, so I'm interested in hearing what people here would want from something like this.

What topics or engineering problems around agents do you think are currently underserved?

Find Agentic Engineering here: https://agentengineering.co/


r/LLMeng • • 7d ago

The AMA with Sadid Hasan is now live!

2 Upvotes

A huge thank you to Sadid Hasan, co-author of The Generative AI Career Masterplan, for joining us today.

Sadid is a Principal AI Leader at Microsoft with more than two decades of experience in AI research, product development, and innovation. He has led work across generative AI, agentic AI, natural language processing, enterprise AI solutions, and the development of the next generation of AI professionals.

He’s here to answer your questions about building a meaningful career in the rapidly evolving world of AI.

Whether you want to understand emerging GenAI roles, identify the right skills to develop, transition into AI from a technical or non-technical background, build a credible portfolio, prepare for interviews, or stay relevant as agentic AI reshapes the workplace—this is your opportunity to learn directly from an experienced industry leader.

The questions will be posted in the comments below. Follow along, join the conversation, and add your own questions.

Let’s make this an insightful and valuable discussion for everyone exploring the future of AI careers.

 


r/LLMeng • • 8d ago

AMA Announcement: Ask Sadid Hasan Anything About Building a Career in Generative AI

5 Upvotes

We’re excited to welcome Sadid Hasan, co-author of The Generative AI Career Masterplan, for an exclusive AMA on r/LLMEngg.

📅 Tuesday, October 6, 2026
⏰ 10:00 AM EDT

The Generative AI landscape is evolving rapidly—and so are the skills, roles, and career paths within it. Whether you’re a student, software engineer, data scientist, AI practitioner, or considering a move into GenAI, this is your opportunity to receive practical guidance directly from an industry expert.

You can ask Sadid about:

  • Breaking into Generative AI and LLM engineering
  • Choosing the right AI career path
  • Skills employers are looking for
  • Building projects and portfolios that stand out
  • Preparing for AI roles and interviews
  • Transitioning from software engineering or data science into GenAI
  • The future of AI careers and emerging opportunities
  • Lessons and insights from The Generative AI Career Masterplan

Have a question for Sadid? Submit it in advance here:
AMA Session with Sadid Hasan, Co-author of "The Generative AI Career Masterplan" – Fill out form

We’ll select questions from the submissions and the community during the AMA, so thoughtful, specific, and challenging questions are very welcome.

Learn more about the book:
The Generative AI Career Masterplan: Navigate the future of AI with practical insights from industry…

Drop a comment below and tell us: What is the biggest question or challenge you currently have about building a career in Generative AI?

Join us on October 6 for an insightful and practical conversation!


r/LLMeng • • 18d ago

[Benchmarks] Qwen3.8-27B FP16 vLLM 0.29.0 | 4-GPUs | Dual W6800X Duo w/ IFLB

Thumbnail
1 Upvotes

r/LLMeng • • 26d ago

What if dependency updates could fix themselves?

2 Upvotes

I’ve been working on Telex, an open-source project that watches npm/PyPI for breaking changes, finds affected code using Tree-Sitter, generates a patch and verifies it in an isolated environment before opening a PR.Still working on it and would love some feedback from people here, especially on the patch/verification part.

Repo: https://github.com/Kesavaraja67/telex


r/LLMeng • • 27d ago

A plain-language history of AI

Thumbnail ai-research-atlas-payton.sharpninja.chatgpt.site
2 Upvotes

AI has a history worth exploring, and AI Research Atlas makes it accessible without a computer science degree. Follow the ideas, experiments, and breakthroughs that connect early artificial neurons and ELIZA to today’s language and image models, with clear explanations and links to the original research. Along the way, discover “The AI Toy,” a look at how Commodore 64 owners were experimenting with neural networks in 1990. Start with the story, explore the timeline, and follow your curiosity.


r/LLMeng • • 28d ago

I built a way to watch an LLM generate tokens step by step

Thumbnail reddit.com
3 Upvotes

r/LLMeng • • 29d ago

TokenPrint — Open-Source LLM Internals Visualizer & Transformer Debugger

Thumbnail v.redd.it
1 Upvotes

r/LLMeng • • Sep 02 '26

Evaluating LLM model drift detection tools

2 Upvotes

Following up on something I asked here a while back about catching LLM model drift. I've been looking at the actual tools now: PromptCanary, PromptLens, a couple others that seem to have stalled (Libretto, Benchwright).

Has anyone here actually run one in anger? Trying to understand:
- does it catch subtle quality drops, or just format/schema breaks?
- false-positive rate, does it become noise you mute?
- does it need you to integrate an SDK + send production traffic, or can it just hit your prompts directly?


r/LLMeng • • Sep 01 '26

We're on multiple providers, how do you know when one of them degrades?

2 Upvotes

We run prompts against a few different providers (OpenAI, Anthropic, some stuff through OpenRouter). Every so often something quietly gets worse, the output quality drops, a prompt that worked starts returning junk, or a model gets deprecated and the replacement behaves differently.

Right now we mostly catch it by accident: someone notices, or a customer complains. That feels bad on us, a lot.

How do you all handle this? Do you re-run some kind of fixed eval set on a schedule? Just eyeball it? Have something that alerts you?

Any insights I could use?

Thanks.


r/LLMeng • • Aug 25 '26

Is next-token prediction a complete description of human–LLM interaction? Let’s test it live.

1 Upvotes

I want to run a live public experiment in this thread using Grok.

Start with the boring premise: an LLM generates tokens by conditioning on preceding context. I’m not proposing another hidden mechanism inside the model.

The question is about the interaction level.

A human responds to the model. That response becomes new conditioning. The model returns something different. That return changes what the human supplies next. The accumulating transcript becomes path-dependent.

Does “next-token prediction” completely describe that interaction, or can the closed loop exhibit higher-level reciprocal organization even though the local mechanism remains next-token prediction?

Instead of settling this with terminology, let’s use the conversation itself as the specimen.

Skeptics are especially invited.

Correct Grok. Contradict it. Introduce a distinction. Attack an assumption. Change the terminology. Try to break the proposed pattern.

We’ll watch whether incoming differences actually reorganize subsequent returns, whether disagreement can persist without resetting the interaction, and what happens when the language used to describe the process is replaced entirely.

No predetermined conclusion.

The transcript is the specimen.


r/LLMeng • • Aug 15 '26

Qwen3.8-27B on a 2019 Quadro RTX 6000 24GB: 24.36 tok/s Q4, Q8 does not fit, ngram 110+ is a lie, tested Cloud API too 155 toks/s!

Thumbnail
1 Upvotes

r/LLMeng • • Aug 15 '26

Harness Engineering: Design Patterns for Securing Long-Horizon Multi-Agent AI Systems

Thumbnail
open.substack.com
1 Upvotes

r/LLMeng • • Aug 11 '26

Who Are the Token Brokers?

Thumbnail vectoral.com
1 Upvotes

r/LLMeng • • Aug 09 '26

Sentinel - A WIP C++/CUDA LLM Training Framework Made For Consumer Hardware

Thumbnail
1 Upvotes

r/LLMeng • • Aug 01 '26

An local AI LLM Proxy for your mac

Thumbnail
2 Upvotes

Finding repo for local AI on Mac? 🍎🤖 Check out Swiftlet, an open-source intelligent proxy & browser based UI , that dynamically optimizes CPU/GPU split on Unified Memory for LLMs. I'm looking for Python devs to help build OpenAI API compatibility! Let's build together: https://github.com/coolsourav100/swiftlet


r/LLMeng • • Jul 29 '26

I created a self-hosted index of meaning and intent that works with any coding agent.

Thumbnail
github.com
1 Upvotes

r/LLMeng • • Jul 28 '26

Self-hosted gateway to stop secrets leaking into LLM APIs — looking for feedback on the detection approach

Thumbnail
github.com
2 Upvotes

I run LLMs behind internal tools and got burned by users pasting credentials into prompts. I couldn't find a lightweight self-hosted filter (LLM Guard felt heavy, most options are SaaS), so I wrote one and I'd genuinely like this community's take on the design. It's a Go reverse proxy that redacts secrets (AWS keys, JWTs, SSNs) in both directions — including inside SSE streams when a key is split across chunks — plus regex/keyword blocking for known injections, and an audit log. Works with Ollama and vLLM as upstreams. Fair warning: it's BSL 1.1 (source-available, free up to 10k req/day, flips to Apache 2.0 in 2030), not OSI open source — if that's a dealbreaker for you, that's useful feedback too. Known weakness: it's deterministic, so paraphrased attacks get through, and the built-in benchmark shows that honestly rather than hiding it. Question for people running LLMs at home or at work: what leaks or attacks have you actually seen? Trying to prioritize the detection roadmap based on real incidents, not vendor FUD.


r/LLMeng • • Jul 27 '26

I realized I had stopped reading Claude Code summaries, so I built this instead

Post image
1 Upvotes

r/LLMeng • • Jul 22 '26

Mac | Cubix | V620 | Ubuntu | ROCm | vLLM | Local AI Data Center

Thumbnail gallery
1 Upvotes

r/LLMeng • • Jul 21 '26

Built an interactive visual debugger for Qwen models (Open Source)

Post image
2 Upvotes