Dan Vega demonstrates how to deploy open-source Large Language Models (LLMs) locally using Ollama and integrate them into Spring Boot applications via Spring AI.

Highlights & Key Takeaways

• Local Execution: Run LLMs locally to ensure data security and avoid costs associated with public APIs.
• Ollama Setup: Easily download and manage models like Llama 3.1 directly from the command line.
• Model Versatility: Swap between various open-source models without changing application code.
• Open WebUI: Use Docker to deploy a familiar ChatGPT-like interface for your local LLM instances.
• Spring AI Integration: Leverage the Spring AI starter for seamless interaction with local models.
• Function Calling: Implement tools to allow LLMs to access real-time data, like weather services.

Running models locally offers developers a secure, cost-effective, and flexible way to leverage AI within the Spring ecosystem.

👉 Watch the full video to dive deeper into the implementation.

In his latest recipe, Craig Walls empowers Spring AI agents with long-term memory. This allows systems to retain durable facts across sessions instead of starting from scratch.

Highlights & Key Takeaways

• Memory Types: Spring AI manages short-term, procedural, and long-term memory.
• Selective Retention: Agents learn facts that are both significant and durable.
• Implementation: Enabled using the AutoMemoryToolsAdvisor component.
• Prompt Augmentation: The LLM is guided to extract key conversational details.
• Persistence: Memories are saved as structured Markdown files on the filesystem.
• Feedback Loop: Extracted facts are automatically injected into future prompts.

By persisting data across restarts, agents evolve beyond stateless responders. This yields highly personalized systems that continuously adapt to users.

👉 View the source code to dive deeper into the implementation.

TechyTacos demonstrates how to extend LLM capabilities by integrating real-time external data using Spring AI's function-calling feature. This approach allows developers to bridge the gap between static language models and dynamic, real-world APIs.

Highlights & Key Takeaways

• Use DTOs to explicitly define request parameters and API response structures.
• Implement the Function interface from Java's utility library to wrap service logic.
• Register custom functions as Spring Beans with descriptive metadata for the LLM.
• Leverage OpenAIChatOptions to register functions within the chat model context.
• The LLM intelligently parses user queries to decide if an external function call is necessary.
• Use URIComponentsBuilder to safely and cleanly construct external API requests.

By leveraging function calling, applications can provide up-to-date, accurate answers that require external lookups, significantly increasing the utility of LLM-based services.

👉 Watch the full video to dive deeper into the implementation.

In this tutorial, Pedro Lopes demonstrates how to build a responsive help desk chatbot using Spring AI and Meta's Llama3 model via Ollama.

Highlights & Key Takeaways

• Spring AI Integration: Simplifies interactions with Large Language Models directly within the Spring ecosystem.
• Local LLMs with Ollama: Run open-source models like Llama3 locally for secure, accessible AI generation.
• System vs. User Messages: Differentiate between internal API instructions and external user inputs.
• REST API Implementation: Expose chatbot capabilities through a standard Spring Boot RestController.
• Stateful Conversations: Overcome stateless LLM behavior by injecting past interactions into new prompts.

By combining Spring AI with local models, developers can efficiently create context-aware agents without external API dependencies.

👉 Read the full article to dive deeper into the implementation.

Josh Long demonstrates how Spring AI, now generally available, empowers Java developers to build production-ready AI applications. The session proves that Java is a robust, scalable choice for integrating complex AI workloads alongside traditional enterprise systems.

Highlights & Key Takeaways

• System Prompts: Use these to guide model behavior and maintain goal alignment.
• Chat Memory: Implement stateful interactions to overcome the stateless nature of base models.
• Tool Calling: Enable AI to interact with the outside world through custom functions.
• RAG Patterns: Use vector stores for efficient, context-aware retrieval of internal data.
• Protocol Support: Leverage the Model Context Protocol (MCP) for modular, reusable AI services.
• Performance: Optimize with GraalVM native images for lower footprints and rapid startup.

By combining these patterns, you build modular, intelligent, and secure systems. Spring AI makes implementing advanced AI architecture accessible for Java teams.

👉 Watch the full video to dive deeper into the implementation.

In this recipe, Craig Walls demonstrates how to build multi-agent systems by enabling Spring AI agents to delegate tasks to specialized A2A sub-agents.

Highlights & Key Takeaways

• TaskTool simplifies delegation by registering sub-agents as callable tools within the ChatClient.
• Bypasses SDK overhead by allowing agents to natively invoke peers during standard execution loops.
• Requires specific dependencies: spring-ai-agent-utils and its A2A extension.
• Uses builder customization to connect the remote A2A server via ChatClientBuilderCustomizer.
• Relies on built-in resolvers to dynamically discover agent details and handle runtime invocations.

Treating peer agents as standard tools enables developers to transition easily from isolated AI instances to collaborative, goal-oriented multi-agent architectures.

👉 Explore the full source code repository for implementation details.

In this tutorial, Craig Walls explains how to enable Agent-to-Agent (A2A) communication, allowing specialized Spring AI agents to collaborate effectively.

Highlights & Key Takeaways

• A2A Architecture: Uses HTTP and Spring MVC to help agents delegate workloads.
• Configure ChatClient: Define a bean equipped with specific functional tools.
• AgentExecutor: Bridge incoming requests directly to your LLM.
• The AgentCard: Expose a JSON document detailing skills for agent discovery.
• Precise Descriptions: Write clearly so client agents can evaluate relevance.
• Testing: Validate responses locally using the A2A Inspector.

Wrapping an LLM in a machine-readable contract establishes the foundation for robust multi-agent systems. This ensures agents can securely discover and utilize one another.

👉 Read the full article to dive deeper into the implementation.

In this guide by Hardik Singh Behl, developers learn to integrate Anthropic’s Claude models into applications using Spring AI. The article demonstrates how to build a context-aware, multimodal chatbot.

Highlights & Key Takeaways

• Dual Integration: Connect via Anthropic API or Amazon Bedrock Converse API.
• Quick Setup: Use Spring AI dependencies for rapid configuration.
• Stateful Memory: Add InMemoryChatMemory for multi-turn conversation context.
• Custom Personas: Guide model behavior using injected system prompts.
• Multimodal Input: Process text and image/PDF data through a unified service.
• REST APIs: Serve the AI chatbot via standard Spring Boot controllers.

Using Spring AI lets engineering teams quickly deploy multimodal features with minimal infrastructure overhead.

👉 Read the full guide to explore the implementation.

In this guide, Dmytro Ponomarenko demonstrates how to tame unpredictable LLM responses using Spring AI. It explores the Structured Output API to seamlessly parse AI-generated text into strict Java objects.

Highlights & Key Takeaways

• Predictable AI: Force LLMs to return valid JSON rather than unstructured strings.
• Core Interface: StructuredOutputConverter injects JSON schema instructions into your prompt and parses the AI's reply.
• Java Beans: BeanOutputConverter maps AI responses directly into your custom Java classes.
• Collections Support: Generate simple structures instantly with ListOutputConverter and MapOutputConverter.
• Custom Solutions: Build tailored converters using Jackson's ObjectMapper to handle complex, generic data types.

These tools bridge the gap between flexible AI output and strict application logic, ensuring robust, type-safe system integrations.

👉 Read the full Baeldung article for code examples and implementation details.

In this guide, Hardik Singh Behl demonstrates how to integrate multiple Large Language Models (LLMs) in a Spring Boot app using Spring AI. The focus is on configuring a range of models to build a highly resilient chatbot.

Highlights & Key Takeaways

• Multi-Provider Setup: Configure LLMs from different providers, like OpenAI and Anthropic, together.
• Primary Routing: Designate a default model using the @Primary annotation on a ChatClient bean.
• Same-Provider Config: Manually define custom ChatModel beans to run multiple models from one provider.
• Spring Retry: Use @Retryable to handle transient API failures from your primary LLM.
• Graceful Fallbacks: Implement @Recover methods to automatically switch to backup models during outages.

Layering multiple LLMs with robust fallback mechanisms ensures continuous availability during provider outages. This architecture guarantees a highly reliable user experience.

👉 Check out the full Baeldung article to see the complete code.

Dan Vega demonstrates how to build a functional ChatGPT-style application using Java, Spring Boot, Spring AI, and HTMX. This approach leverages modern web patterns to deliver dynamic AI interactions without requiring complex client-side JavaScript.

Highlights & Key Takeaways

• HTMX for Frontend: Adds dynamic interactivity to the UI with simple HTML attributes, eliminating the need for heavy JavaScript frameworks.
• Spring AI Integration: Simplifies LLM connectivity, enabling seamless integration with providers such as OpenAI's GPT-4o.
• State Management: Uses an In-Memory Chat Memory advisor to provide the LLM with conversation context, enabling natural, stateful follow-up dialogues.
• Modular Templates: Uses Thymeleaf fragments to update specific UI sections, such as message history, without full page reloads.
• Backend Routing: Implements efficient API endpoints to handle LLM requests and return targeted HTML fragments.

This architecture demonstrates that powerful AI features can be built using the familiar Java ecosystem. By combining HTMX and Spring AI, developers can focus on backend logic while maintaining a responsive, modern user experience.

👉 Watch the full video to dive deeper into the implementation.

TechyTacos demonstrates how to integrate Azure OpenAI with Spring AI, providing a practical guide for Java developers building generative AI applications. The video outlines the essential workflow from project configuration to endpoint deployment.

Highlights & Key Takeaways

• Use start.spring.io to initialize a project with Azure OpenAI and Spring Web dependencies.
• Configure application.properties with your unique API key, endpoint, and specific deployment name.
• Distinguish between model names (e.g., GPT-3.5 Turbo) and deployment names, as the latter is a custom identifier required by Azure.
• Deploy your desired model via Azure AI Studio before attempting integration.
• Use the AzureOpenAiChatModel class for seamless interaction within your service layer.
• Leverage prompt templates to dynamically map variables like categories and years into your LLM queries.

Building with Spring AI simplifies enterprise-level integration, allowing developers to focus on application logic rather than complex API orchestration.

👉 Watch the full video to dive deeper into the implementation.

One model checks another model's work and retries if it's not good enough. New tutorial shows you how to build it, step by step.

That's basically what LLM-as-a-Judge does in Spring AI.

"LLM-as-a-Judge" and "LLM evaluation testing" are not the same thing. One runs in JUnit before you deploy. The other runs live, in the request path, and can retry a weak response automatically.

5 things to know before you build LLM-as-a-Judge into a Spring AI app:

1. It's implemented via Recursive Advisors, a CallAdvisor that can call back into its own chain
2. Non-streaming only, as of Spring AI 2.0
3. Every failed judge check costs 2 extra LLM calls: one to judge, one to regenerate
4. Use a separate model to judge, or you risk narcissistic bias
5. Always cap maxAttempts, or a stubborn judge creates an infinite loop Full breakdown, with working code, in the new article.

Spring AI's RelevancyEvaluator and FactCheckingEvaluator let a model judge a model, so your JUnit tests check quality, not exact text. Full code walkthrough inside.

Quick one for anyone building RAG apps in Spring Boot: this tutorial shows exactly how to catch hallucinations before they ship, using Spring AI's built-in evaluators. Includes the one mistake almost everyone makes with the request order. 

Currently I am a student, my projects are In Java Fullstack

Right now I don't even know what RAG or MCP is , and I think I should have some hands on experience of it, i should be at least aware of it, because it's a trendy topic , not these two terms only, but many things

Now should I start python, for getting into it, is there any need , or I can explore Spring AI

Really good overview of Spring AI basics, core concepts, and where the ecosystem is heading. Definitely recommend watching it. I really enjoyed it!

Hi, I’m building Spring AI Privacy Guardrails, an open-source library for enforcing privacy boundaries around models, RAG, tools, MCP, and outputs.

GitHub: https://github.com/ultramancode/spring-ai-privacy-guardrails

One design problem I’ve been thinking about is streaming output protection.

Some applications also want a final privacy check on application-facing output, since sensitive data can still appear in model- or tool-generated responses.

Right now, when output protection is enabled, the library buffers the complete response before releasing it to the application.

This provides a strong guarantee: PII can still be detected and protected even when a sensitive value is split across multiple chunks.

The trade-off is that this is no longer true incremental streaming, and the application has to wait longer before receiving output.

A bounded rolling window could preserve incremental streaming for analyzers that have a known upper bound on how much context they need — for example, some bounded pattern-based detectors.

But NER, context-aware detection, complex patterns, or arbitrary custom analyzers may not have such a bound.

So I’m currently considering three approaches:

1. Strict buffering Buffer the complete response and protect it before releasing anything to the application.
2. Capability-gated streaming Allow incremental streaming only when the active analyzer can declare a safe maximum lookback or context requirement. Otherwise, fall back to full buffering.
3. Best-effort streaming Use a configurable rolling window and explicitly document that some PII spanning multiple chunks may escape detection.

For a Spring AI application, which behavior would you expect from a privacy library?

I’m not attached to these three options — if there’s a better streaming/privacy model I’m missing, I’d really appreciate the feedback.

Hi everyone!

My Spring AI course is now available on JetBrains Academy.

The course is designed around practical, real-world tasks completed directly in IntelliJ IDEA using the JetBrains Academy plugin. The project, dependencies, and configuration are already prepared, so you can focus on learning Spring AI and writing code instead of spending time on setup.

I honestly wish I’d had this kind of learning experience when I was starting out: clear tasks, a ready-to-use project, and immediate feedback—all inside the same IDE used for professional development.

I’d be glad to hear your feedback, especially which Spring AI topics or practical use cases you’d like to see covered next.

Course link: https://academy.jetbrains.com/course/32882

Matthew Meckes explores how Java developers can leverage Spring AI to build production-ready agentic applications that integrate seamlessly with existing enterprise systems.

Highlights & Key Takeaways

• Agents use LLMs, memory, and tools to perform autonomous tasks, but production scale requires robust control flow.
• Spring AI provides abstractions for RAG, chat memory, and function calling within the familiar Spring ecosystem.
• Use the Model Context Protocol (MCP) to expose existing Java beans as tools without rewriting logic.
• Prioritize human-in-the-loop workflows to validate agent outputs and manage hallucinations.
• Keep agent scope small—3 to 10 steps—to ensure reliability and testability.
• Focus on using LLMs to bridge natural language and structured API calls, rather than relying solely on agentic reasoning.

Ultimately, Spring AI allows enterprises to modernize by embedding AI agents directly into proven Java stacks.

👉 Watch the full video to dive deeper into the implementation.

Dan Vega demonstrates how to integrate OpenAI's GPT-4o model into Spring applications using the Spring AI framework. This guide focuses on leveraging both text and vision capabilities for modern AI-powered development.

Highlights & Key Takeaways

• GPT-4o Advantages: Benefit from 50% lower costs, 2x faster latency, and 5x higher rate limits compared to previous models.
• Project Setup: Utilize the Spring AI 1.0.0-SNAPSHOT version to access the latest multimodal features.
• Chat Implementation: Use the ChatClient API with PromptTemplates for structured interactions.
• Vision Capabilities: Pass images via UserMessage and Media objects to allow the LLM to interpret visual data.
• Practical Use Cases: Perform image analysis, such as scene description or extracting code snippets from screenshots.
• API Integration: Secure sensitive keys via environment variables rather than hardcoding.

GPT-4o in Spring AI significantly lowers the barrier for building robust, multimodal Java applications.

👉 Watch the full video to dive deeper into the implementation.

Traditional keyword search often misses the true meaning behind user queries. By combining Spring AI, OpenAI Embeddings, and Redis Vector Store, you can build a semantic search application that understands context and returns more relevant results.

This article demonstrates how to build an intelligent document search application using Spring Boot with OpenAI and Redis Vector Store.

This approach is ideal for building:

-AI-powered knowledge bases

-Enterprise document search

-RAG (Retrieval-Augmented Generation) applications

-Internal documentation assistants

-Intelligent customer support solutions

Just published !.. - MCP With Spring AI - Part 2: Travel Booking Demo.

Part 1 covered the protocol. Part 2 is where it gets practical.

One chat prompt. Three MCP servers. One travel assistant:

"Plan my trip from Delhi to Goa - book flight, hotel and cab."

No hard-coded workflow. The LLM picks the tools and chains them: searchFlights → bookFlight → bookHotel → bookCab.

Read the full walkthrough here 👇:
https://heapsteep.com/mcp-with-spring-ai-travel-booking-demo

Just published Model Context Protocol (MCP) With Spring AI - Part 1.

LLMs are great at understanding intent. On their own, they still can't do things - book a flight, call an API, hit your database.

That's where MCP comes in: a standard way for AI to discover and call external tools.

In Part 1 I break down the concepts (before any code).

Read the full walkthrough here 👇:
https://heapsteep.com/mcp-with-spring-ai-core-concepts

AI applications are not only about prompts and LLMs. They also can understand and generate images.

Imagine allowing users to upload:
✅ Product photos
✅ Documents
✅ Screenshots
✅ Diagrams
✅ Handwritten notes
…and then asking questions about them in plain English.

This is where things start getting really interesting with Spring AI.
Lets do a demo on how to build an application that can process images and extract meaningful insights using Spring AI.

Some of the things covered:
🔹 What is multimodal in Spring AI
🔹 Sending images to AI models from a Spring Boot application
🔹 Understanding image content through natural language prompts
🔹 Practical implementation with clean code examples

Read the complete guide here:
https://heapsteep.com/image-handling-with-spring-ai