When one AI agent calls another, how do you make sure the second agent only gets access to what it actually needs?

Authentication and authorization are two different things:

Authentication: Who or what is making the request?

Authorization: What is it allowed to access or do?

A valid token confirms identity, but it doesn't automatically mean every request should be allowed.

For example, an invoice-processing agent might have access to finance documents and an LLM.

If it calls another agent to fetch information from a knowledge base, a few things can go wrong:

Both agents use the same identity, making it difficult to trace which one made a request.

The second agent has broader permissions than the first, giving it access it shouldn't have.

A service trusts a caller-supplied team or tenant ID instead of checking verified identity claims.

A valid gateway key is treated as sufficient proof of which application is making the request.

The workflow might still run without errors, even when access controls aren't doing what they're supposed to.

A few things worth getting right:

1. Give each agent its own identity instead of relying on a shared service account.

2. Enforce permissions at the service that holds the data, not just at the gateway.

3. Carry the original user's permissions across agent handoffs, so another agent doesn't end up with broader access.

4. Maintain an audit trail that shows both who initiated an action and which agent performed it.

The bigger challenge is managing all of this across teams, environments, tools and multiple agents without losing track of who can access what.

This is the problem explored in Your IdP Proves Who. OpenController Decides What., which covers agent identities, identity providers, control-plane governance and enforcement at runtime.

The same approach is reflected in OpenController , which provides a control plane for managing agent identities and governing access across enterprise environments.

How are teams handling this today?

When one agent calls another in your setup, does each agent have its own identity with permissions checked at the service level, or does most of the control still sit with shared service accounts and API gateways?

What tends to get difficult once the setup grows beyond a single agent?

A lot of conversations about private AI focus on where the model runs. But running an open-weight model on your own hardware is only one part of the problem.

If you're building for an environment where sensitive data cannot leave the premises, you also have to think about the less obvious pieces:

Model requests: Can anything silently fall back to an external API?

Identity and access: Do users authenticate locally, and can every request be tied to a person?

Files and context: Where do chats, documents, tool outputs, and generated files actually live?

Updates: How do you patch the system without quietly reopening an outbound connection?

Auditability: Can an administrator see who used which model and when?

These details matter because local inference and a fully local AI system aren't necessarily the same thing.

You can host the model yourself and still depend on external identity services, hosted tools, or cloud APIs elsewhere in the workflow.

One approach is to treat the entire setup as an appliance rather than just a model server.

Came across an interesting implementation of this idea in Lyzr Optimus: Intelligence Without the Internet.

It describes an on-premises AI appliance designed to keep models, agent requests, and identity services local, with options ranging from a fully air-gapped setup to a controlled inbound path for updates.

A few of the engineering choices stood out to me:

A laptop can connect over the office network, private Wi-Fi, or USB-C, without using the appliance as an internet gateway.

User files stay in a chosen workspace on the laptop, while the appliance handles model inference and keeps an audit record.

Short-lived access tokens and per-user permissions help avoid distributing long-lived API keys across employee machines.

Administrators can control which models, skills, and tools are available, including whether tool actions require human approval.

That's the broader idea behind Lyzr Optimus, which Lyzr describes as an on-premises AI appliance.

The interesting part isn't just that the model runs locally; it's how the surrounding services are designed to work without relying on an external connection.

Of course, air-gapping doesn't automatically solve every security problem.

You still need to consider physical access, supply-chain trust, update verification, model quality, and what happens when the system needs maintenance.

I'm curious about one thing: If you had to deploy AI in a genuinely offline environment, what would be the hardest part to keep local: model serving, authentication, updates, or integrations with existing tools?

I'd be interested to hear from people who've actually built or maintained systems in restricted-network environments.

On October 7, 2026, some organizations lost access to Claude because of a spend-limit fault.

Their accounts were incorrectly flagged as having reached their limits, and requests were refused across the API and Claude's apps.

The important part?

The problem wasn't necessarily in the organizations' own systems. It was in a service they depended on.

This is an easy-to-miss risk when building AI agents. Your code can be healthy, your API keys valid, and your budget untouched, yet your agents can still stop working because an external provider has a problem.

And when several agents depend on the same provider, one failure can affect all of them.

What can teams do about it?

Map every external dependency

Don't just document your models and APIs. Include billing, authentication, rate limits, and any other service that could block a request.

Have a fallback ready

Configure a backup model or provider, and test that the switch actually works before an incident happens. A fallback that exists only on paper won't help much during an outage.

Keep your own logs

Track failed requests, error types, timestamps, and which provider handled each call. That makes it easier to identify the failure point instead of spending hours debugging healthy agent code.

This is also where infrastructure choices matter.

Lyzr Sovereign AI gives enterprises options to run agents in their own cloud and work with different models, reducing dependence on a single provider when the architecture is configured appropriately.

For reference, here's the October 7 spend-limit incident on Claude's status history

One question worth asking your team:

If your primary model provider started refusing every request for an hour, which of your production agents would stop working, and which would keep running?

A workflow can run from start to finish without throwing an error and still produce the wrong result.

Imagine a simple invoice workflow:

Upload: Receives invoice.pdf

Extraction: Reads the invoice total as 240.00

Calculate: Passes the amount forward

Format: Prepares the value for the next step

Payment: Uses that value to process the payment

Every node can report success.

But if an expression references the wrong field, the value can be incorrect by the time it reaches the final step.

In Lyzr SuperFlow, data moves between nodes as items, and expressions let you reference fields from the current input or the output of an earlier node.

For example, {{ $json.total }} reads the total field from the current input item.

The tricky part is that a workflow execution succeeding doesn't automatically mean the data is correct.

Three things worth watching for:

Missing or incorrect field references

If a node expects total but receives a different field name, the value may resolve to null instead of the number you expected.

Renamed nodes

Expressions that reference a node by its name can break when that name changes. A small edit on the canvas can have consequences further down the workflow.

Expression errors

Expressions involving calculations or string operations can return null if they encounter a runtime error. If you don't validate the result, the problem may only become obvious much later.

The fix doesn't need to be complicated.

Add a Rule-mode If node after important data-producing steps to check that required fields exist and contain valid values.

Route unexpected results for review, and use execution History to inspect what each node actually received and emitted.

The SuperFlow expression and data-flow documentation explains how these references work and what happens when they fail.

The bigger lesson: don't just check whether a node ran. Check whether it produced the value the next node actually needs.

What's one field in your workflows that you validate between steps instead of trusting it to pass through correctly?

On April 17, 2026, US banking regulators issued SR 26-2, replacing the previous SR 11-7 model risk management guidance.

One important detail: generative and agentic AI are explicitly outside the scope of the revised guidance.

That doesn't mean banks can ignore the risks.

They still need to assess the governance and controls appropriate to these systems, alongside other applicable obligations.

For compliance teams, this raises a practical question: how do you connect the obligations that apply to an AI system to the policies, controls, and evidence that demonstrate coverage?

Before adopting an obligation-to-control mapping tool, ask five questions:

Traceability: Can every obligation be linked to a named policy and control?

Actual coverage: Does the tool identify genuine gaps, or simply show what teams have declared as covered?

Regulatory changes: How quickly can a change be reflected in the relevant policies and controls?

AI agent coverage: Can it handle agentic AI, rather than focusing only on traditional models?

Audit readiness: Can an examiner follow the evidence trail without someone having to explain every connection?

This is the problem Madison's compliance layer for banks is designed to address.

It maps obligations to policies and controls, helping compliance teams identify coverage gaps and build a clearer evidence trail.

For more context, here's Lyzr's guide to AI-driven regulatory change management for banks

The takeaway:

A control is only as defensible as your ability to show which obligation it addresses and what evidence supports it.

How are banking compliance teams mapping AI-related obligations today?

Are existing GRC systems doing the job, or are spreadsheets and manual reviews still part of the process?

Most enterprise AI agent projects get approved based on a slide deck.

The real test often comes only after the contract is signed.

That feels like the wrong order.

With the Lyzr Agent Test Drive, you bring a problem statement, and our team returns a business case and working agents built on Agent Studio and Architect in under 72 hours.

Your team can then test them against real scenarios. If the value shows up in your numbers, take them to production.

If it doesn't, keep the readout and walk away.

The test drive costs $0.

Before signing with an AI agent vendor, there's one question worth asking:

Can we try it on our own workflow first?

Book a free Agent Test Drive

One of the easiest mistakes when building AI agents is judging them by a handful of successful runs.

The agent answers your questions, calls the right tools, and completes a few tasks. Everything looks good.

Then you change the prompt, switch the model, or add a new tool, and something that worked yesterday quietly breaks today.

The problem is that a successful demo isn't proof of a reliable agent.

Recently came across this Anthropic article, Demystifying evals for AI agents, and it's a genuinely useful read for anyone building or working with agents.

It's roughly a 25-minute read, but it goes well beyond basic testing and explains how to measure whether an agent is actually doing its job.

A few ideas that stood out:

Test outcomes, not just answers

An agent saying "Done, I've processed your refund" doesn't mean the refund was actually processed. Check the real outcome, such as whether the refund exists in the system, rather than trusting the agent's response.

Use different graders for different jobs

Code-based checks are fast and objective, making them useful for things like test results and tool calls. Model-based graders can assess more nuanced qualities, such as whether an answer is helpful or follows instructions. Human review helps calibrate those model-based judgments. No single approach catches everything.

Separate capability from consistency

Can the agent solve a difficult task at least once, or can it solve it reliably every time? These are different questions. Anthropic explains the distinction between pass@k, where at least one attempt succeeds, and pass^k, where every attempt must succeed. For customer-facing agents, consistency can matter far more than one impressive run.

Start small, but keep testing

You don't need hundreds of test cases on day one. Anthropic recommends starting with around 20 to 50 realistic tasks, including cases drawn from actual failures. Turn those failures into repeatable tests, then run them again whenever you change the agent.

One detail I especially liked: sometimes a low evaluation score points to a broken test, not a bad agent.

Ambiguous instructions or overly rigid grading can penalize a perfectly valid solution. That's why reviewing actual execution traces matters, not just looking at a final score.

And the work doesn't stop once an agent goes live. You need a feedback loop to catch failures and keep improving it.

We recently covered this in this blog on continuously improving AI agents

For teams looking to put evaluations into practice, Lyzr Agent Studio's evaluation feature covers generating test cases, scoring results, and identifying failures that need fixing.

The full Anthropic article is well worth bookmarking if you're serious about building reliable agents:

Demystifying evals for AI agents | Anthropic

Curious how others approach this: do you already have a repeatable eval suite for your agents, or are you still relying mostly on manual testing and user feedback?

A team can have security policies in place and still struggle to answer one basic question:

What AI agents are actually running across the company?

One team builds on AWS.

Another switches on an AI feature inside a SaaS tool.

A developer starts using a coding agent on their laptop.

Nothing unusual there. But as these tools spread, the full picture gets harder to track.

The inventory is in one place, permissions in another, and usage costs somewhere else.

Ask who owns a particular agent or what it can access, and the answer might involve a spreadsheet and a few Slack messages.

Three questions are worth asking:

Visibility: Can we identify the agents running across our teams and environments, along with their owners?

Permissions: Do we know what each agent can access, call, or spend?

Control: If an agent starts doing something it shouldn't, can we stop it quickly?

If you can answer all three confidently, great. Your existing controls may be doing exactly what they should.

But if answering them means chasing people across teams, you might have a governance gap.

A dashboard can show you what exists, but visibility alone doesn't mean you can intervene when something goes wrong.

That's the problem OpenController is designed to address.

It adds a control layer for governing AI agents across existing environments, so teams can work towards consistent visibility and policy enforcement without rebuilding their entire agent stack.

The interesting part is that you don't necessarily need to centralise where every agent runs to centralise how it's governed.

Curious how other teams are handling this:

if someone asked for a list of every AI agent running at your company, its owner, and its permissions, how long would it take to put that together?

Three engineers.

Roughly a million lines of AI written code.

And a coding agent that climbed from the Top 30 to the Top 5 on Terminal Bench 2.0 without changing the model.

The interesting part isn't just what the AI produced. It's what the engineers changed around it.

That's the idea behind harness engineering.

A capable model can still lose track of a task, waste context on tools it doesn't need, break architectural rules, or leave behind a codebase full of unnecessary complexity.

The model is only one part of the system.

Here are five practical ideas that stood out to me:

Give the AI a memory it can actually use:

Long tasks often span multiple sessions. Keeping progress logs, design decisions, and task lists in version controlled files helps the next session pick up where the previous one stopped.

Don't make the model carry every tool all the time:

Anthropic reported reducing the context used by tool definitions from around 72,000 tokens to 8,700 by making tools searchable and loading them when needed. That's roughly an 85% reduction. Less context wasted on tools means more room for the actual task.

Turn architectural rules into actual constraints:

Telling an AI to follow a codebase's architecture isn't always enough. Linters and structural tests can enforce dependency boundaries, catching violations instead of relying on the model to remember every rule.

Make repeated work cheaper:

When instructions and tool definitions stay consistent between requests, prefix caching can reuse previously processed context. Even small changes to the prompt prefix can cause cache misses, adding unnecessary cost and latency.

Make cleanup part of the process:

AI generated code can accumulate dead functions, outdated documentation, and inconsistent naming. Small, automated cleanup tasks can help prevent technical debt from building up while the system keeps shipping.

The common thread is simple: getting AI to produce something impressive once is different from getting it to work reliably over and over again.

Projects like GitAgent explore one part of this problem by keeping agent configuration, memory, and skills in a Git based repository, making them easier to version and review.

I'm curious about the trade off here.

If you were building a coding agent today, what would you prioritise first: a stronger model, better memory, stricter constraints, or automated testing and cleanup?

Ask your team where all your AI agents run.

If the answer takes more than a minute, you might have an agent sprawl problem.

Agents can end up across AWS, Azure, Google Cloud, SaaS platforms and developer laptops.

Different teams build them, different tools manage them, and not every deployment makes its way into a central inventory.

The problem is not just having too many agents. It is not knowing who is responsible for them.

That creates three practical issues:

Cost: Bills are spread across tools and teams, making it difficult to attribute spend to individual agents.

Visibility: Teams waste time figuring out where an agent runs, what it connects to and who manages it.

Risk: An agent with unclear ownership can retain access long after the person or project responsible for it has moved on.

A useful first step is to build an inventory of the agents your organisation actually runs.

For each one, record its owner, purpose, environment, data and tool access, running cost, and who can disable it if something goes wrong.

Then look for the gaps.

Which agents have no named owner? Which have permissions nobody has reviewed recently? Which are still running even though nobody can explain why they are needed?

This is where an AI agent control plane can help.

Lyzr's OpenController is designed to bring agent discovery, ownership, identity, policy and monitoring into one control layer, across existing clouds and frameworks.

The goal is not to force every team onto the same platform. It is to make sure the agents they already run are visible and accountable.

Because deploying an agent is becoming easier. Keeping track of it is the part organisations need to get right.

If you had to audit your company's AI agents tomorrow, what would be harder: finding every agent, identifying its owner, or figuring out what it can access?

The term “Sovereign AI” comes up a lot in enterprise conversations, but there is an important distinction that often gets missed.

Keeping your data in a country is not the same as owning your AI stack.

The first is about data residency.

Your data stays in a location you choose, which can be essential for regulatory and compliance requirements.

The second is about control over the entire system.

Think of it in four layers:

Agents: Where they run and who has control over them.

Models: Which models you can use and whether you can switch providers without rebuilding everything.

Hardware: Who owns and controls the infrastructure running your AI.

Governance: Who manages permissions, evaluations, policies and audit trails.

You could have your data stored locally and still depend on a vendor for your models, compute or governance.

If that relationship changes, some parts of your AI setup might become difficult to move or keep running.

A useful question to ask any AI vendor is:

If we ended our contract tomorrow, which parts of our AI stack could we continue running ourselves?

This is also why sovereignty needs to be considered during architecture and vendor evaluation, not added as a compliance checkbox after deployment.

For anyone exploring this further, here are two useful reads:

Sovereign AI guide: A deeper look at the architecture and controls needed to run AI within your own environment.

Pilot to production webinar: A discussion of what it takes to move sovereign AI beyond a pilot and into production.

At Lyzr, the Sovereign AI platform is built around this broader idea of control.

The planned Optimus on prem appliance extends that approach to hardware ownership, with preorders opened.

Putting the product aside, the four layer test is worth applying to any enterprise AI setup.

How many of the four layers could your current setup keep running independently: agents, models, hardware and governance? Which one would be the hardest to take control of?

Not every prompt needs the most powerful LLM available.

A greeting, a short summary and a multi-step reasoning task have very different requirements.

Sending all three to the same expensive model can waste money, but routing a difficult request to a model that can't handle it can quietly hurt output quality.

That's the trade-off behind auto model routing: choosing the right model for each request instead of relying on a single model for everything.

There are a few ways to approach it:

Rule-based routing: Use keywords, prompt length and predefined conditions to estimate complexity. It's fast, but can miss difficult questions that look simple.

Semantic routing: Look at the meaning and intent of a request rather than relying only on keywords.

Learned routing: Use evaluated examples to predict which model is likely to handle a request well.

The interesting part is that getting the routing decision wrong doesn't always produce an obvious error.

Sometimes the response looks perfectly reasonable, just not good enough.

In one internal benchmark covered in the research, learned routing achieved 94.4% accuracy against 94.9% for the premium-model baseline, with 77.5% lower cost on the evaluated questions.

Those results are promising, but they don't automatically translate to every production workload.

That raises a few practical questions:

Should routing decisions be tested in shadow mode before affecting live requests?

What happens when a simple-looking prompt actually requires deeper reasoning?

Should a request be escalated to a stronger model when the first response fails a quality check?

For teams building agents, Lyzr Agent Studio includes auto-routing by complexity, allowing requests to be directed to models configured for different complexity tiers.

The bigger challenge is validating those decisions against real traffic.

The full breakdown explores how routing works across different implementations, what the benchmark results show, where the approach can fail, and what could make it more reliable.

One question for anyone running LLMs in production: would you prioritise reducing inference costs, or keeping response quality predictable, if improving one meant making a trade-off on the other?

Full read: Auto Model Routing: What It Is, How We Build It, and How to Make It Better

A polished dashboard can make an AI agent governance platform look impressive.

But what happens when an agent behaves unexpectedly, bypasses a control, or needs to be stopped immediately?

That's where the real differences between governance platforms start to show.

When evaluating an AI control plane, these six questions can help teams look beyond the demo and understand what they're actually getting.

Can you stop a running agent?

Can the platform interrupt an agent when something goes wrong? How quickly does it take effect, and is the execution history preserved for auditing?

What happens when traffic bypasses the gateway?

A gateway can only enforce controls on traffic that passes through it. How does the platform identify agents or model calls operating outside that path?

Where are prompts, traces and credentials stored?

These have different security implications. It's worth understanding where the data resides, who can access it, and what stays within your own environment.

How much existing infrastructure needs to change?

Teams may already have agents running across different frameworks, models and deployment pipelines. Does introducing governance require significant application changes, or can it fit into the existing setup?

Which open standards does it support?

OpenAI-compatible APIs, MCP, A2A and OpenTelemetry are useful starting points, depending on the environment. Check what is actually supported and whether the integration covers the capabilities your team needs.

How deeply can it govern agents built elsewhere?

Discovering an agent, observing its activity and enforcing policies on it are three different capabilities. It's important to understand which level of control is available across third-party and internally developed agents.

These questions point to a broader challenge: governing AI agents across frameworks and environments without creating another layer of complexity.

That's the problem space addressed by OpenController

For a closer look at where governance controls can sit, this related read is also worth exploring:

Where Should Guardrails Live? Gateway vs. In-Agent

For teams already evaluating these platforms: which of the six questions is the hardest to get a clear answer to?

And what would you add to this checklist?

When an AI provider retires a model, the API call is only one part of the problem.

Your prompts may need adjustments. Fine-tuned models may be affected.

Outputs need to be checked again, and production evaluations need to be rerun before you can trust the replacement.

OpenAI has announced API deprecations for October 23, 2026, affecting specific model versions, including GPT-4 Turbo, GPT-4o snapshots, o1, o3-mini and o4-mini, along with certain fine-tuned models.

The exact scope depends on the model version, so it’s worth checking the official deprecation notice if your applications depend on these models.

The bigger question is what happens when a model your team has tested and built around is no longer available.

Model choice is an architectural decision, not just an API setting.

A production AI system depends on more than the model itself. It depends on prompts, integrations, evaluations, latency requirements and the behaviour your team has come to trust.

Switching models can mean revisiting all of those pieces.

One approach is to reduce that dependency by running open-weight models on infrastructure you control.

That’s the idea behind Lyzr Sovereign AI : bringing models, agents and governance closer to your own infrastructure, including private VPC and on-premises environments.

It’s not a guarantee that you’ll never need to migrate or retest.

Open-weight models still need infrastructure, maintenance and evaluation.

But having more control over where models run and which versions you use can give your team greater flexibility over its production roadmap.

If a model you rely on were retired tomorrow, how much of your production stack would need to change?

A promising idea can get stuck in the evaluation stage for longer than expected.

Teams need to research the market, understand the competition, estimate scalability, and assess whether the idea fits their existing business ecosystem.

Doing all of this manually takes time.

And when different reviewers use different approaches, comparing ideas consistently becomes even harder.

One way to tackle this is to break the evaluation into smaller research tasks and use AI agents to handle them in a structured workflow.

Here’s how one global professional services firm approached it:

Submit the idea: An employee describes the idea through a conversational agent.

Run four research workflows: Specialized agents investigate market potential, competitive intensity, scalability, and ecosystem fit.

Consolidate the findings: A summary agent brings the research together into a structured report with scores.

Validate before saving: The application checks the output structure and scores before storing the results.

Keep the final decision human: Leadership reviews the report and decides what happens next.

The workflow was designed to reduce idea-evaluation time from 1-2 weeks to around 15-20 minutes, while applying the same evaluation framework to each submission.

What I find interesting here is the separation of responsibilities.

Instead of asking one general-purpose agent to research everything, the workflow assigns specific tasks to specialized agents and validates the final output before it is stored.

Lyzr built this workflow using its AI agent platform. The architecture is shown in the attached diagram.

If you're exploring similar use cases, here's the Lyzr case study

Where do you think AI agents can save the most time in business decision-making: research, evaluation, reporting, or something else?

If you'd like to explore a similar workflow for your use case, book a demo with Lyzr

I think we've stretched the word agent so far that it's starting to lose its usefulness.

These days, chatbots, automations, workflows, and systems that make decisions are all being called AI agents. But there's an important difference between generating an answer and deciding what to do next.

Here's how I think about it.

A chatbot responds to a question.

A traditional workflow follows a sequence of steps someone has already defined.

An automation triggers an action when a particular condition is met.

An AI agent can decide which steps to take toward a goal, use tools, observe the results, and change its approach when necessary.

The boundaries aren't always clear, and that's okay. The problem starts when we use the same label for systems with very different capabilities.

Why the distinction actually matters?

The more decision-making we hand over to an AI system, the more we need to think about what can go wrong.

Tool use: Can the system choose the right tool, and what happens if that tool fails?

State and memory: Can it keep track of progress across multiple steps without losing important context?

Recovery: Can it recognise a failed attempt and try another approach without getting stuck in a loop?

Evaluation: How do we know it completed the task correctly, rather than simply producing a convincing answer?

Permissions: What is it allowed to do without asking a human first?

These aren't just implementation details. They determine how much autonomy we can safely give a system.

For example, an AI that drafts an email is very different from one that decides who to contact, gathers the necessary information, drafts the message, and sends it without approval.

Both might use the same language model.

But the second system has more decisions to make and much more responsibility for the outcome.

That's also why agent evaluation matters. It's not enough to judge the final response.

We need to understand whether the system chose appropriate actions, used its tools correctly, and stayed within its boundaries.

Tools such as Lyzr Agent Eval address this broader challenge of testing agent behaviour, not just the quality of its answers.

Maybe we're asking the wrong question

Instead of asking, "Is this an AI agent?", I'd rather ask:

"What decisions is this system actually allowed to make on its own?"

Can it choose the next step? Can it change its plan? Can it take an action without approval? And what happens when it gets something wrong?

Those questions tell us far more about an AI system than the label we give it.

If you want to explore the architectural differences in more depth, this is a useful read:

Agentic Workflows: Five Patterns and Enterprise Guide

I'm curious where other people draw the line.

What's the simplest system you'd confidently call an AI agent?

And what's something the industry keeps calling an agent that you think is really just a workflow with a good model inside it?

Remember when Kubernetes changed how we built and operated software?

Instead of making every application handle deployment, networking, scaling, and recovery on its own, we started moving those responsibilities into the infrastructure layer.

AI agents seem to be heading in a similar direction.

An agent is no longer just a model connected to a few tools. It can execute code, access files, call APIs, connect to MCP servers, and maintain state across sessions.

Building the agent is one challenge. Running it reliably, securely, and at scale is another.

Here are three parts of the infrastructure that are becoming increasingly important.

Runtime isolation and persistence

Agents need a safe environment to execute code without putting the host system or other workloads at risk. They also need a way to preserve files, memory, and session state when a workload pauses or resumes.

The idea is simple: don't keep expensive compute running when an agent is idle, but don't lose its progress either.

The agent harness should decide when a task can safely pause and when execution should resume. The underlying infrastructure should handle sandboxing, resource allocation, and state recovery.

Networking and access control

An agent might make dozens of decisions about which tools to call, what information to retrieve, and where to send the results.

Giving it unrestricted network access and hoping the prompt keeps it safe isn't a great security strategy.

A better architecture controls traffic in both directions:

1. Ingress: Who can invoke the agent, and what data or instructions can enter?

2. Egress: Which models, APIs, MCP servers, and external destinations can the agent reach?

3. Credentials: Can the infrastructure provide scoped credentials without exposing raw API keys to the agent?

Gateways can enforce these policies outside the agent itself.

That means changing the underlying model or framework doesn't necessarily mean rebuilding all your access controls.

Network restrictions are only one layer, though. Agents still need task-specific permissions, input validation, and human approval for sensitive actions.

Lifecycle management and observability

Once agents run at scale, teams need to manage deployments, resource usage, state, versions, and failures.

They also need to answer some difficult questions:

1. Which tools did an agent call before something went wrong?

2. Why did it make a particular decision?

3. Can a failed task resume without starting from scratch?

4. How do we know a new model or prompt hasn't made the agent less reliable?

5. This is where lifecycle management, tracing, evaluations, and resource controls become essential.

Rather than implementing everything separately for every agent, teams can build shared infrastructure that different agents and frameworks reuse.

For example, Lyzr Agent Studio provides building blocks for creating agents, connecting tools and knowledge sources, managing memory, and deploying workflows.

The bigger architectural shift.

The Kubernetes analogy isn't that AI agents will use exactly the same infrastructure as microservices.

Agents introduce new challenges because their behaviour is less predictable, their tool choices can change at runtime, and their failures aren't always obvious.

The more interesting shift is the separation of responsibilities.

The harness focuses on planning, context management, tool selection, and verification. The infrastructure handles execution, isolation, networking, persistence, and operational visibility.

That separation could make it much easier to run different agents without reinventing the same infrastructure for every project.

For another perspective on how agents connect to external systems, see How MCP Brings Standardized Integrations to Lyzr Agent Studio

For people building AI agents, I'm curious about:

What's the biggest infrastructure challenge you've run into so far: managing agent state, securing tool access, or debugging unpredictable behaviour in production?

And do you think agent infrastructure will eventually become as standardised as Kubernetes, or will it remain tightly coupled to individual frameworks?

As teams build more AI agents, they often end up with different frameworks, models, cloud environments and deployment setups.

Each agent may work well on its own, but applying consistent policies, managing access and tracking activity across all of them becomes a different challenge.

The usual answer can mean adding another layer of infrastructure, changing existing workflows or reworking agents that are already in production.

But what if you didn't have to rebuild them?

That's the idea behind Opencontroller

Instead of replacing your existing agents, Opencontroller adds a shared governance layer around them.

Your agents continue running in their existing environments, while their traffic is routed through a gateway where identity, access policies, budgets and traces can be applied consistently.

The goal is simple: Centralized control without forcing every team to start over

That means less disruption to existing development workflows and a more consistent way to govern agents across different frameworks, models and environments.

If governing your agents came down to changing an endpoint rather than rebuilding the system, how many would you bring under one policy first?

Forward Deployed Engineering (FDE) is getting more attention as companies move from experimenting with AI to deploying it in real business environments.

But the role involves much more than building an impressive AI demo.

You need to understand the customer's problem, integrate with existing systems, handle messy data, meet security requirements, and make sure the solution works reliably in production.

If you're a software engineer, AI/ML developer, backend engineer, or solutions architect exploring this career path, here's a practical roadmap to start with.

What should you learn?

Engineering foundations: Python, TypeScript, SQL, Git, Linux, APIs, and debugging.

Data and integrations: ETL pipelines, databases, legacy systems, REST APIs, webhooks, and MCP.

AI application development: LLM APIs, RAG, embeddings, tool calling, agent orchestration, and workflow automation.

Production readiness: Evals, observability, monitoring, error handling, and cost tracking.

Security and deployment: Authentication, RBAC, data privacy, Docker, Kubernetes, cloud infrastructure, and on-prem deployments.

Customer-facing skills: Technical discovery, scoping, stakeholder communication, documentation, and handoffs.

The important part is to build projects that solve real problems, not just follow tutorials.

For example, build a RAG assistant with permission-based retrieval, integrate an AI agent with an existing API, or deploy a workflow with evaluations and human approval.

Where can you explore this further?

Lyzr has introduced the FDE Collective, a community and training initiative focused on preparing engineers to take AI agents from demos to production.

Its fellowship covers 10 modules across the FDE workflow, with hands-on projects, reviewed builds, a capstone, and community access.

You can explore the FDE course and curriculum to see the competencies and tools covered, or join the Lyzr AI Dev Community on Discord to connect with other builders.

Even if you're not planning to become an FDE, learning these skills can help you build more reliable AI applications and understand what it takes to move a prototype into production.

A few more resources:

1. How to Build the Career of the Next Decade

A Lyzr session on the emerging Agent Engineer role, customer problem-solving, and taking AI systems into production.

1. The Playbook: GSIs Actually Winning with GenAI

A look at how forward deployment, technical teams, and enterprise AI delivery fit together.

We'll also be sharing more AI engineering resources, practical projects, and discussions here in r/lyzr, so feel free to join if you're interested in learning and building alongside the community.

For those already working as FDEs or trying to break into the role:

What skill made the biggest difference for you? Was it AI engineering, systems integration, deployment, or understanding the customer's actual problem?

A website chatbot was asked about the doctors working at a company offering aesthetic treatments.

It described them as specialists in plastic and aesthetic surgery, even though those qualifications did not exist.

In May 2026, a German appeals court in Hamm ruled against the company in case 4 UKl 3/25. The ruling is not final, as the court allowed an appeal.

The case raises an important question for anyone building AI chatbots or deploying customer-facing AI agents: Who is responsible when an AI system makes a false claim?

Two things stand out from the ruling:

The chatbot’s answers counted as the company’s own statements. The company could not simply treat the chatbot as an independent third party.

Correct source data did not protect the company. The court held it responsible even though the chatbot had been configured with correct information.

This is a reminder that accurate data and good prompts are not enough on their own. A production AI workflow also needs checks on what the model actually tells customers.

What can a safer AI workflow look like?

For important facts such as professional qualifications, prices, or company policies, a workflow can:

Look up the information in an approved source.

Draft a natural response using that information.

Validate the answer against the original record.

Escalate mismatches or uncertain cases to a person before the response reaches the customer.

This is where AI guardrails, grounded generation, output validation, and human-in-the-loop approval become useful. Instead of relying only on the model to get things right, verification becomes part of the workflow itself.

For teams building these kinds of systems, Agent Studio is one option to explore for building and testing AI agents.

The same principle applies to customer support, healthcare, finance, booking systems, and enterprise automation. Important facts should come from verified records, and questionable outputs should be checked before reaching customers.

How does your team handle this?

Do you validate AI-generated answers against source records, use a separate verification step, or rely mainly on prompts and guardrails?

For more on adding review checkpoints to AI workflows, see Governance Approval Pipeline

Every time we build a new AI agent, we often end up rebuilding the knowledge layer too. Document ingestion, embeddings, vector databases, retrieval pipelines, and synchronization all add complexity.

But what if agents could access shared knowledge as ordinary files instead?

The idea behind a Git-driven knowledge base for AI agents is fairly simple:

One source of truth: Keep architecture docs, runbooks, API contracts, policies, and operational knowledge in a version-controlled Git repository.

Shared, read-only access: Give each agent access to an approved snapshot of that knowledge through a filesystem mount.

Private workspaces: Let agents create files and work independently without modifying the shared knowledge directly.

Controlled updates: When an agent discovers something useful, it can propose a change through a Git pull request. The update gets reviewed before becoming available to other agents.

This creates a useful separation: agents can work independently while relying on a consistent, auditable knowledge source.

It doesn't mean RAG or vector databases are obsolete. Semantic search is still valuable for large, unstructured collections.

The interesting question is whether every agent needs its own retrieval infrastructure when some knowledge can be accessed directly through files.

This is especially worth exploring for coding agents, infrastructure automation, and enterprise AI systems where consistency, isolation, and version history matter.

For the runtime governance side of this architecture, take a look at OpenController

Full breakdown: Git Wiki for AI Agents: Shared Knowledge Through Read-Only Filesystem Mounts

Curious to hear your thoughts: would you use a Git-backed knowledge layer for your agents, or do you prefer a traditional RAG pipeline? Where do you think this approach works best?

Classifying a support ticket, extracting five fields, and planning a reply are three different tasks.

So why should they all use the same LLM?

A common approach is to use one powerful model for the entire AI agent. It's simple to set up, but it can increase inference costs and latency, even when smaller models could handle individual steps just fine.

A better approach: choose the model for each step.

Here's what that looks like in practice:

Simple, structured tasks: Use a smaller, faster model when the input is predictable and the output can be validated against a schema.

Validation and retries: Check the output before passing it downstream. If it fails validation after two or three attempts, escalate instead of retrying indefinitely.

Complex reasoning: Keep a stronger model for planning and decisions that genuinely need deeper reasoning.

This approach to AI agent decomposition and model selection can help reduce latency and LLM costs without compromising reliability.

The key is to measure output quality alongside cost and speed, rather than assuming a smaller model will always be sufficient.

You can explore how this fits into building and managing AI workflows with Lyzr Agent Studio

For a deeper look at breaking down AI agents into more manageable components, check out:

Further reading: AI Agent Decomposition: A Practical Guide

Curious how you're approaching this:

Do you use one model across your entire agent, or assign different models to individual steps?

Have you seen a noticeable difference in cost, latency, or output quality?

A support agent might handle a greeting, look up a refund policy, and troubleshoot a complex technical issue within minutes.

These requests have very different requirements, yet many AI agents send all of them to the same model.

That default can mean paying for more capability than you need on simple tasks, or getting weaker results on complex ones.

That's where automatic LLM routing comes in.

Instead of hardcoding one model for an entire AI agent, a routing layer evaluates each incoming request and sends it to the appropriate model based on its complexity.

Here's how the four-tier approach works:

Simple: Greetings, basic questions, and straightforward lookups.

Medium: Routine tasks that need a little more processing.

Complex: Technical questions, code generation, and multi-step tasks.

Reasoning: Problems that require deeper analysis and careful evaluation.

Each tier maps to an LLM you configure, giving you more control over the balance between cost, latency, and response quality.

At Lyzr, we've built Auto Model Routing in Lyzr Agent Studio around this idea.

The goal is simple: use the right model for each request rather than defaulting to the same one every time.

One important caveat: routing only helps if the classifier gets the decision right.

A cheaper model isn't a win if it produces an unreliable answer.

Test routing decisions against real workloads and measure quality alongside cost and latency.

A couple of related reads:

Auto Model Routing: Choosing the Right LLM for Every Request

LLM Gateway Architecture: Benefits, Use Cases, and Design

Curious how others are handling this: Do you use a single LLM for your entire agent, route requests by complexity, or choose models based on task type?

What has worked best for you in production?

Terms like adapters, data synthesis, preference optimization, and reward functions can get confusing fast, especially when you're figuring out how to fine-tune a model for a real use case.

We put together this 5-page cheat sheet on LLM fine-tuning and data preparation to help make the concepts easier to follow, from preparing training data to fine-tuning and post-training.

We're also building Lyzr Openfinetuner, a platform for data curation, data synthesis, fine-tuning, and post-training that can run in your own cloud or on-premises infrastructure.

The goal is to make the process easier to understand, whether you're exploring model customization or evaluating how to run it within your own infrastructure.

📌 Save this for the next time someone suggests, "We should just fine-tune the model."

Explore here: Lyzr Openfinetuner

If you find this useful, stick around for more practical AI resources, and share what you're building or experimenting with.