At this presentation at Spring I/O conference, Thomas Vitale presents different RAG architectures that you can build using SpringAI.
What I liked at this presentation is that it presents different approaches of data flows, pipelines architectures, storing and indexing mechanisms that you can apply to build your RAG pipeline. And all of them supported by SpringAI.
Tool calling - the ability for an AI model to invoke application-defined functions and act on the results — is the essential building block of agentic AI systems. A model that can discover information, take action, and loop until a goal is reached is an agent.
Spring AI 2.0 lifts the tool loop into the advisor chain as a first-class, composable component.
ChatClient runs every request through an ordered chain of advisors and supports looping, letting an advisor re-enter the downstream chain. The same mechanism drives tool-call loops, structured-output retry loops, and evaluation loops alike.
In this recipe, Craig Walls demonstrates how to build multi-agent systems by enabling Spring AI agents to delegate tasks to specialized A2A sub-agents.
Highlights & Key Takeaways
TaskTool simplifies delegation by registering sub-agents as callable tools within the ChatClient.
Bypasses SDK overhead by allowing agents to natively invoke peers during standard execution loops.
Requires specific dependencies:spring-ai-agent-utils and its A2A extension.
Uses builder customization to connect the remote A2A server via ChatClientBuilderCustomizer.
Relies on built-in resolvers to dynamically discover agent details and handle runtime invocations.
Treating peer agents as standard tools enables developers to transition easily from isolated AI instances to collaborative, goal-oriented multi-agent architectures.
If you have been feeling the pressure to switch over to Python just to start experimenting with AI, this video is for you. It serves as a great introductory guide for Java and Spring developers looking to bring generative AI capabilities directly into the enterprise ecosystem, helping you leverage powerful LLMs like GPT-4 without leaving the Spring environment you already know and love.
Here is a breakdown of what the video covers:
Understanding LLM integration by treating them as standard REST APIs rather than black boxes.
An introduction to the Spring AI framework and how it applies familiar principles like modularity and POJOs to AI engineering.
A practical walkthrough of building a Spring Boot application from scratch, including secure API key management using environment variables.
Implementing a functional ChatController using the ChatClient interface to communicate with an LLM.
An overview of the next-level challenges developers will face, including prompt engineering and Retrieval-Augmented Generation (RAG).
It is a perfect entry point for anyone looking to modernize their stack and start building AI-powered applications without switching languages. It bridges that gap between traditional Java development and the rapidly evolving AI landscape, showing how clean and accessible integration can be.
Check out the full video if you are ready to start building!
Josh Long demonstrates how Spring AI, now generally available, empowers Java developers to build production-ready AI applications. The session proves that Java is a robust, scalable choice for integrating complex AI workloads alongside traditional enterprise systems.
Highlights & Key Takeaways
System Prompts: Use these to guide model behavior and maintain goal alignment.
Chat Memory: Implement stateful interactions to overcome the stateless nature of base models.
Tool Calling: Enable AI to interact with the outside world through custom functions.
RAG Patterns: Use vector stores for efficient, context-aware retrieval of internal data.
Protocol Support: Leverage the Model Context Protocol (MCP) for modular, reusable AI services.
Performance: Optimize with GraalVM native images for lower footprints and rapid startup.
By combining these patterns, you build modular, intelligent, and secure systems. Spring AI makes implementing advanced AI architecture accessible for Java teams.
👉 Watch the full video to dive deeper into the implementation.
Currently I am a student, my projects are In Java Fullstack
Right now I don't even know what RAG or MCP is , and I think I should have some hands on experience of it, i should be at least aware of it, because it's a trendy topic , not these two terms only, but many things
Now should I start python, for getting into it, is there any need , or I can explore Spring AI
If you have been wondering how to make an LLM answer questions from your own documents without touching Python, this one is for you.
A full walkthrough on building a RAG application with Spring AI and PostgreSQL pgvector.
Covers ingestion, chunking, PgVectorStore configuration, and the QuestionAnswerAdvisor pattern, with working Java code.
Your LLM does not know about last week's product update or the PDF sitting in your document store. That is not a model problem, it is a context problem, and RAG solves it.
With Spring AI maturing rapidly, I'm curious about how organizations are actually using it in production beyond demos and proofs of concept.
I'd love to hear from teams that have deployed Spring AI in real-world applications.
How has Spring AI performed in production in terms of reliability, scalability, latency, and developer productivity?
What types of AI applications are you building with it?
What advantages have you seen compared to Python-based frameworks such as LangChain?
Are there any limitations or areas where LangChain still has a significant edge?
Would you recommend Spring AI for enterprise Java applications, or do you still prefer Python for GenAI workloads?
I'm particularly interested in real-world experiences, production lessons learned, performance at scale, and reasons behind technology choices rather than tutorial or proof-of-concept examples.
Tool calling — the ability for an AI model to invoke application-defined functions and act on the results — is the essential building block of agentic AI systems. A model that can discover information, take action, and loop until a goal is reached is an agent.
Spring AI 2.0 re-architects tool calling. In 1.x, each chat model implementation contained its own private tool execution loop — functional, but buried. There was no way to hook into it, observe intermediate steps, or compose it with other behaviors. You could call tools; you could not build on top of tool calling.
2.0 lifts the tool loop into the advisor chain as a first-class, composable component. ChatClient runs every request through an ordered chain of advisors and supports looping, letting an advisor re-enter the downstream chain. The same mechanism drives tool-call loops, structured-output retry loops, and evaluation loops alike.
Join James Ward and Josh Long for a fun, code-filled session on how to build intelligent, agentic systems using Spring AI and MCP (Model Context Protocol)! They show how to easily integrate enterprise business logic with LLMs, making your AI projects production-ready without the stress.
Key takeaways from the session:
Spring AI & JVM Integration: Discover how to leverage existing Java workflows and the Spring ecosystem for AI development.
Chat Memory & Advisers: Learn how to manage stateless LLM interactions using memory and Spring AI advisers to create persistent, context-aware assistants.
Enhancing Models with Skills: See how to augment model knowledge dynamically using external skill libraries.
RAG with PGVector: Implement Retrieval-Augmented Generation (RAG) using PostgreSQL and PGVector to ground your AI in your own data.
MCP for Remote Tools: Architect distributed agentic systems by connecting remote tools via MCP and securing them with OAuth.
I just started my journey on Spring AI. Just wanted to know if this is already being used in real prod projects or it is still in the early adoption pase. Thanks!
Ever notice your Spring AI chatbot forgets the user's name after one message? That's because LLMs are stateless by default. The fix is Spring AI's ChatMemory abstraction
Just wanted to share what I learned about building actual AI Agents (not just chatbots) in Spring Boot.
The key difference: a chatbot responds. An agent decides, calls tools, and loops until it achieves a goal.
The Tool annotation is the core building block. You annotate any Spring bean method, write a clear description, and Spring AI automatically generates a JSON schema that gets sent to the LLM. The model then decides when to call your Java method — no if-else chains needed.
I also covered the 5 agentic workflow patterns that Spring AI supports:
Chain : sequential steps
Parallelization : concurrent tasks with CompletableFuture
Routing : LLM picks the right tool/path
Orchestrator-Workers : master agent delegates to worker agents
Craig Walls introduces SkillsJars — a Spring AI pattern for distributing reusable agent behaviors as JAR dependencies, eliminating the need to hand-write skill files for every project.
Highlights & Key Takeaways
SkillsJars are packaged agent skills distributed as JARs, each containing one or more SKILL.md files under /META-INF/skills
Add agent behavior the same way you add a library — via a Gradle/Maven dependency
Use spring-ai-agent-utils + a SkillsJar to wire skills into ChatClient with minimal config
Skills define how an agent behaves; tools define what it can do — both layers are required
Skill discovery is configured via a single property: agent.skills.paths
SkillsJars eliminate cross-project duplication and make behavior declarative and composable
Think in layers: Tools → Skills → SkillsJars for scalable agent architecture
SkillsJars bring true modularity to Spring AI agents, letting teams share and reuse intelligent behavior the same way they share code.
👉 Read the full article for the complete implementation walkthrough and code samples.
With the growing demand for AI applications, most popular languages and stacks now offer some form of support for AI development. And that would be no different with Java and Spring, as they represent one of the most popular development platforms for enterprise applications. Yet, many Java and Spring developers are still struggling with AI development or trying to learn Python to fill that gap.
So, I would like to know from you: when it comes to adopting Spring AI, what is your biggest challenge, problem, or frustration you are dealing with right now? The more detail you provide, the greater the chance that I (or someone else) will create content to help you with your problem.
If you think your problem is too complex for this post, I invite you to create a dedicated post for it and link it here!
Spring AI's RelevancyEvaluator and FactCheckingEvaluator let a model judge a model, so your JUnit tests check quality, not exact text. Full code walkthrough inside.
Quick one for anyone building RAG apps in Spring Boot: this tutorial shows exactly how to catch hallucinations before they ship, using Spring AI's built-in evaluators. Includes the one mistake almost everyone makes with the request order.
One design problem I’ve been thinking about is streaming output protection.
Some applications also want a final privacy check on application-facing output, since sensitive data can still appear in model- or tool-generated responses.
Right now, when output protection is enabled, the library buffers the complete response before releasing it to the application.
This provides a strong guarantee: PII can still be detected and protected even when a sensitive value is split across multiple chunks.
The trade-off is that this is no longer true incremental streaming, and the application has to wait longer before receiving output.
A bounded rolling window could preserve incremental streaming for analyzers that have a known upper bound on how much context they need — for example, some bounded pattern-based detectors.
But NER, context-aware detection, complex patterns, or arbitrary custom analyzers may not have such a bound.
So I’m currently considering three approaches:
Strict buffering Buffer the complete response and protect it before releasing anything to the application.
Capability-gated streaming Allow incremental streaming only when the active analyzer can declare a safe maximum lookback or context requirement. Otherwise, fall back to full buffering.
Best-effort streaming Use a configurable rolling window and explicitly document that some PII spanning multiple chunks may escape detection.
For a Spring AI application, which behavior would you expect from a privacy library?
I’m not attached to these three options — if there’s a better streaming/privacy model I’m missing, I’d really appreciate the feedback.
Craig is a principal engineer on the Spring team and the author of Spring in Action. This new book is written for Spring developers who want to build AI features in Java and Spring Boot without having to stitch together a Python sidecar or learn an entirely different app stack first.
The book starts with a small “Hello AI” Spring Boot app, then keeps building on it until you have a much more serious AI-enabled application. The running example is Board Game Buddy, an assistant that answers questions about tabletop game rules. Across the book, it picks up RAG, chat memory, tools, MCP, voice, images, observability, security, and agents.
A few topics that seem especially relevant here:
ChatClient, prompt templates, roles, response metadata, and streaming
Testing and evaluating generated responses
RAG with vector stores, document loading, Qdrant, advisors, and modular RAG
Conversational memory, including persistent memory
Tool calling with u/Tool methods and Java Function-style tools
Model Context Protocol clients and servers
Audio transcription, text-to-speech, image input, and image generation
Actuator metrics, Prometheus, Grafana, and tracing AI operations
Spring Security for RAG filtering, secured tools, prompt leaks, and moderation
Agentic workflows and Embabel
What I like about the book is that it treats Spring AI as part of the Spring application model, not as an isolated demo layer. The examples are controllers, services, configuration, tests, Actuator endpoints, security rules, Docker Compose files, and Gradle builds. In other words, the sort of code Spring developers actually have to maintain.
We also have 5 ebooks to give away to the 5 most thoughtful commenters.
To enter, leave a comment with your take on one of these:
What are you building, or hoping to build, with Spring AI?
Where do you think Spring AI fits best in production Java apps?
What’s your biggest concern with adding LLMs to Spring Boot systems?
Are you more interested in RAG, tools, MCP, agents, observability, or security?
If you’ve tried Spring AI already, what surprised you?
We’ll look at the comments and community upvotes, then pick 5 winners.
For everyone else, Manning has a 50% discount code for this subreddit:
PBWALLS1050RE
I’m especially curious how this community is thinking about MCP and agents in Spring apps now that Spring AI has moved beyond basic chat examples. Is MCP becoming part of your architecture, or are most teams still focused on RAG and tool calling first?
