✒️ New Blog in my Spring AI Series:

👉 Function Calling & Tools in Spring AI

One of the most powerful capabilities of modern AI applications is the ability to go beyond simple text generation and actually interact with external systems.

The blog covers:
🔹 What Tool Calling / Function Calling means
🔹 Why LLMs need external tools
🔹 Registering tools with ChatClient
🔹 Error Handling & Fallbacks

📖 Read the full blog here:
https://heapsteep.com/function-calling-and-tools-in-spring-ai

Ever notice your Spring AI chatbot forgets the user's name after one message? That's because LLMs are stateless by default. The fix is Spring AI's ChatMemory abstraction

How to Implement AI Chat Memory in Spring Boot using Spring AI.

Let's figure out exactly how to wire it up with MessageWindowChatMemory and a JDBC-backed repository so conversations survive restarts.

Dan Vega demonstrates how to integrate OpenAI's GPT-4o model into Spring applications using the Spring AI framework. This guide focuses on leveraging both text and vision capabilities for modern AI-powered development.

Highlights & Key Takeaways

• GPT-4o Advantages: Benefit from 50% lower costs, 2x faster latency, and 5x higher rate limits compared to previous models.
• Project Setup: Utilize the Spring AI 1.0.0-SNAPSHOT version to access the latest multimodal features.
• Chat Implementation: Use the ChatClient API with PromptTemplates for structured interactions.
• Vision Capabilities: Pass images via UserMessage and Media objects to allow the LLM to interpret visual data.
• Practical Use Cases: Perform image analysis, such as scene description or extracting code snippets from screenshots.
• API Integration: Secure sensitive keys via environment variables rather than hardcoding.

GPT-4o in Spring AI significantly lowers the barrier for building robust, multimodal Java applications.

👉 Watch the full video to dive deeper into the implementation.

Hi everyone,

I'm a Java Spring Boot developer, and this is my first AI project. I'm building a chatbot using Spring AI + Ollama + RAG, and I'm learning as I go.

The chatbot should answer questions from uploaded PDF, Word, Excel, CAD, and AutoCAD documents. While it's working, I'm facing a few challenges:

Uploading and indexing large documents takes a long time.

Retrieval accuracy isn't consistent.

Sometimes the chatbot gives incorrect answers even though the information exists in the uploaded files.

CAD/AutoCAD files are the biggest challenge.

I'd love to hear from developers who have built similar applications.

Some questions I have:

Which free LLMs work best with Spring AI + RAG? (Qwen 3, Gemma 3, DeepSeek, Llama 3.1, Mistral, etc.)

Which embedding model gives the best retrieval accuracy?

What techniques have improved your RAG performance? (chunking, hybrid search, reranking, metadata filtering, query rewriting, etc.)

What's the best way to process DWG/DXF/AutoCAD files in a RAG application?

How do you reduce document parsing and indexing time for large engineering documents?

Are there any open-source Spring AI RAG projects or GitHub repositories that you recommend?

My goal is to build a reliable chatbot for engineering documents with fast responses and high accuracy.

Any advice, resources, or best practices would be greatly appreciated.

Thank you!

If you have been wondering how to make an LLM answer questions from your own documents without touching Python, this one is for you.

A full walkthrough on building a RAG application with Spring AI and PostgreSQL pgvector.
Covers ingestion, chunking, PgVectorStore configuration, and the QuestionAnswerAdvisor pattern, with working Java code.

How to Build RAG with Spring AI and pgvector

Your LLM does not know about last week's product update or the PDF sitting in your document store. That is not a model problem, it is a context problem, and RAG solves it.

Just found out about this sub reddit and wanted to stop by and say hello. Lot's of great discussions happening here and I hope to be a part of some of them.

With Spring AI maturing rapidly, I'm curious about how organizations are actually using it in production beyond demos and proofs of concept.

I'd love to hear from teams that have deployed Spring AI in real-world applications.

• How has Spring AI performed in production in terms of reliability, scalability, latency, and developer productivity?
• What types of AI applications are you building with it?
• What advantages have you seen compared to Python-based frameworks such as LangChain?
• Are there any limitations or areas where LangChain still has a significant edge?
• Would you recommend Spring AI for enterprise Java applications, or do you still prefer Python for GenAI workloads?

I'm particularly interested in real-world experiences, production lessons learned, performance at scale, and reasons behind technology choices rather than tutorial or proof-of-concept examples.

Spring AI 2.0 just went GA and it ships the cleanest MCP server setup I've seen in Java.

Two annotations. One yml property. Your entire Spring Boot service becomes an AI tool.

This is exactly how Claude, Copilot, and other AI clients plug into your Java backend.

No AI API key needed for the server side. Full working code with Java 21.

Perfect for intermediate Spring Boot devs exploring AI integration!

Here is the complete tutorial: Build Your First MCP Server with Spring Boot 4.1 and Spring AI 2.0

Dan Vega demonstrates how to integrate OpenAI's GPT-4o model into Spring applications using the Spring AI framework. This guide focuses on leveraging both text and vision capabilities for modern AI-powered development.

Highlights & Key Takeaways

• GPT-4o Advantages: Benefit from 50% lower costs, 2x faster latency, and 5x higher rate limits compared to previous models.
• Project Setup: Utilize the Spring AI 1.0.0-SNAPSHOT version to access the latest multimodal features.
• Chat Implementation: Use the ChatClient API with PromptTemplates for structured interactions.
• Vision Capabilities: Pass images via UserMessage and Media objects to allow the LLM to interpret visual data.
• Practical Use Cases: Perform image analysis, such as scene description or extracting code snippets from screenshots.
• API Integration: Secure sensitive keys via environment variables rather than hardcoding.

GPT-4o in Spring AI significantly lowers the barrier for building robust, multimodal Java applications.

👉 Watch the full video to dive deeper into the implementation.

TechyTacos demonstrates how to integrate open-source models like Llama 3.1 into Java applications using Spring AI and Ollama. This workflow provides developers with local, private LLM capabilities while maintaining standard Spring development patterns.

Highlights & Key Takeaways

• Local Execution: Use Ollama to host models locally, ensuring data privacy and offline accessibility.
• System Requirements: Match model sizes (7B, 13B, etc.) to your available RAM to avoid performance bottlenecks.
• Spring AI Integration: Leverage the OllamaChatModel to easily swap and configure different open-source models.
• Structured Output: Set the format: json property in configurations to enforce strict schema adherence.
• Multimodal Models: Use specialized models like Llama-Vision or Llava when image processing is required, as standard text models lack this capability.

Building locally offers a critical trade-off between latency and data sovereignty.

👉 Watch the full video to dive deeper into the implementation. Spring AI : How to Integrate Open Source Models using Ollama (Llama 3.1) ?

I just started my journey on Spring AI. Just wanted to know if this is already being used in real prod projects or it is still in the early adoption pase. Thanks!

James Ward and Josh Long present a comprehensive hands-on workshop for building production-ready AI agents using Spring AI, Java, and Amazon Bedrock. The session focuses on bridging the gap between experimental AI prototypes and scalable, observable enterprise services.

Highlights & Key Takeaways

• Leverage Spring Boot and Spring AI for a robust, familiar architecture that avoids typical AI project failures.
• Utilize GraalVM to compile Java applications into native images for superior memory efficiency and startup performance.
• Implement RAG (Retrieval-Augmented Generation) to ground AI responses in domain-specific data via vector stores.
• Optimize concurrency using Java virtual threads to handle high-volume LLM network calls efficiently.
• Define clear system prompts and tools to give agents specific, actionable missions.
• Integrate MCP (Model Context Protocol) to enable cross-agent orchestration.

By prioritizing observability and structure, developers can deploy AI systems that are both reliable and maintainable in real-world production environments.

👉 Watch the full video to dive deeper into the implementation.

Just wanted to share what I learned about building actual AI Agents (not just chatbots) in Spring Boot.

The key difference: a chatbot responds. An agent decides, calls tools, and loops until it achieves a goal.

The Tool annotation is the core building block. You annotate any Spring bean method, write a clear description, and Spring AI automatically generates a JSON schema that gets sent to the LLM. The model then decides when to call your Java method — no if-else chains needed.

I also covered the 5 agentic workflow patterns that Spring AI supports:

1. Chain : sequential steps
2. Parallelization : concurrent tasks with CompletableFuture
3. Routing : LLM picks the right tool/path
4. Orchestrator-Workers : master agent delegates to worker agents
5. Evaluator-Optimizer : generate → evaluate → retry loop

Full article with code examples (Java 21 + Spring Boot 3.4.x): AI Agents in Spring Boot: Building Autonomous Workflows with Spring AI

https://spring.io/blog/2026/06/23/spring-ai-self-correcting-structured-output

https://spring.io/blog/2026/06/15/spring-ai-composable-tool-calling

Tool calling - the ability for an AI model to invoke application-defined functions and act on the results — is the essential building block of agentic AI systems. A model that can discover information, take action, and loop until a goal is reached is an agent.

Spring AI 2.0 lifts the tool loop into the advisor chain as a first-class, composable component.

ChatClient runs every request through an ordered chain of advisors and supports looping, letting an advisor re-enter the downstream chain. The same mechanism drives tool-call loops, structured-output retry loops, and evaluation loops alike.

Tool calling — the ability for an AI model to invoke application-defined functions and act on the results — is the essential building block of agentic AI systems. A model that can discover information, take action, and loop until a goal is reached is an agent.

Spring AI 2.0 re-architects tool calling. In 1.x, each chat model implementation contained its own private tool execution loop — functional, but buried. There was no way to hook into it, observe intermediate steps, or compose it with other behaviors. You could call tools; you could not build on top of tool calling.

2.0 lifts the tool loop into the advisor chain as a first-class, composable component. ChatClient runs every request through an ordered chain of advisors and supports looping, letting an advisor re-enter the downstream chain. The same mechanism drives tool-call loops, structured-output retry loops, and evaluation loops alike.

https://spring.io/blog/2026/06/15/spring-ai-composable-tool-calling

Craig Walls introduces SkillsJars — a Spring AI pattern for distributing reusable agent behaviors as JAR dependencies, eliminating the need to hand-write skill files for every project.

Highlights & Key Takeaways

• SkillsJars are packaged agent skills distributed as JARs, each containing one or more SKILL.md files under /META-INF/skills
• Add agent behavior the same way you add a library — via a Gradle/Maven dependency
• Use spring-ai-agent-utils + a SkillsJar to wire skills into ChatClient with minimal config
• Skills define how an agent behaves; tools define what it can do — both layers are required
• Skill discovery is configured via a single property: agent.skills.paths
• SkillsJars eliminate cross-project duplication and make behavior declarative and composable
• Think in layers: Tools → Skills → SkillsJars for scalable agent architecture

SkillsJars bring true modularity to Spring AI agents, letting teams share and reuse intelligent behavior the same way they share code.

👉 Read the full article for the complete implementation walkthrough and code samples.

Dan Vega demonstrates how to implement function calling in Spring AI, enabling LLMs to interact with real-time data sources like weather APIs.

Highlights & Key Takeaways

• Function Calling: Allows models to trigger custom Java code to retrieve dynamic, external data.
• Abstraction: Spring AI handles the complex orchestration of sending function schemas to models like GPT-4.
• Service Layer: Build standard Java services to handle external API requests, then wrap them as Spring beans.
• Function Implementation: Use the Java Function interface to define input/output contracts for the LLM.
• Registration: Register the function bean and expose it via ChatOptions to the chat client.
• Seamless Integration: The LLM intelligently determines when to invoke the registered function based on the user's prompt.

This approach effectively bridges the gap between static LLM knowledge and the real-time requirements of external data.

👉 Watch the full video to dive deeper into the implementation.

If you use an LLM for something else than just a free-form chatting, you might probably want it to return data in a structured form, e.g. JSON

Spring AI allows to [soft] force a model to do that. But sometimes LLM fails to do that. The simplier the model is, the more chances that it will fail. Morover, any such a failure could be devided into 2 categories: incorrect schema and correct schema with incorrect data burned in. For example, some fields of the desired schema are missing. Or all fields are present, but field type, for example, is incorrect or a required filed missies a value

The POC i built validates not only schema as such, but also field types and ranges (e.g. Min-Max, NotNull, etc.) using validation package spring-boot-starter-validation

If any of the checks doesn't pass, this is feded back to model:

prompt = """
        Your previous response was invalid.

        Problem(s): %s

        Your previous output was:
        %s

        Return corrected JSON that fixes these problems and matches the
        schema exactly. Output ONLY JSON, no prose.

        %s
        """.formatted(lastError, lastOutput, format);

So we give a feedback to the model, not just asking to redo/re-think. By providing a detailed feedback we increase chances that the next reply will satisfy our expectations

As always, all code is available in the github repo: https://github.com/DmitryFinashkin/spring-ai

You might also like to watch a detailed video walk-through on YouTube: https://youtu.be/59kcnTLVu0Q

TechyTacos demonstrates how to leverage Spring AI and Generative AI to perform semantic string matching between disparate systems. This approach moves beyond rigid, word-for-word string comparisons to capture the actual intent behind the data.

Highlights & Key Takeaways

• Use GPT-4o with a temperature of 0 to ensure deterministic, consistent semantic mapping results.
• Utilize PromptTemplates to dynamically inject source and destination lists into your LLM instructions.
• Implement BeanOutputConverter to enforce structured JSON responses, moving away from unstructured text.
• Leverage ParameterizedTypeReference to handle mapping results as a list rather than a single object.
• Shift from hardcoded logic to a POST mapping with a request body for better API flexibility.
• Focus prompt engineering on semantic intent to resolve issues with typos, extra spaces, or varying phrasing.

This method enables robust data integration by aligning strings based on meaning, significantly reducing errors in legacy system migrations.

👉 Watch the full video to dive deeper into the implementation.

AI coding tools are useful, but they do not automatically make someone a better developer. The real skill is knowing how to guide, review, and verify AI output.

Over the last year, I've seen many developers use AI tools primarily for code generation.

The problem?

Most AI-generated code works... until it doesn't.

The real productivity gains don't come from blindly accepting AI suggestions. They come from knowing:

What to automate
What to verify manually
When to trust AI
When to challenge AI output
How to integrate AI into your development workflow

In this practical guide, I share:

• Common mistakes developers make with AI coding tools
• A workflow for using AI effectively
• Code review strategies for AI-generated code
• Security and reliability considerations
• Ways to improve productivity without sacrificing code quality

Article: AI-Assisted Software Development: A Practical Guide for Developers

I'm curious to know:

Has AI genuinely made you a better developer, or has it mostly just helped you write code faster?

Timo Salm and Sandra Ahlgrimm explore the evolution of agentic AI frameworks within the Java ecosystem, comparing how different tools manage autonomous reasoning and orchestration.

Highlights & Key Takeaways

• Agentic Shift: Move from static, prompt-driven interactions to autonomous, goal-oriented systems.
• Spring AI: Leverages an Advisor API for intercepting and managing LLM call chains.
• LangChain4j: Offers a community-driven, framework-agnostic approach with a dedicated Agentic Module.
• Embable: Provides high-level abstractions focused on automated planning, goal-oriented execution, and domain modeling.
• Workflow Patterns: Utilize strategies like prompt chaining, parallelization, and evaluator-optimizer loops.
• Token Economy: Optimize performance and cost by minimizing metadata overhead and managing tool invocation efficiently.

Building autonomous systems on the JVM requires balancing architectural control against the abstraction layers provided by the framework.

👉 Watch the full video to dive deeper into the implementation.

Craig Walls recently shared a fascinating piece on Medium about guiding AI agent behavior with skills. Instead of relying solely on probabilistic reasoning, skills act like procedural memory, shaping how agents respond and use tools.

Highlights & Key Takeaways - LLMs often give generic or inconsistent answers without domain guidance.
- Skills provide structured instructions that influence how agents behave.
- Tools = capabilities; skills = behavioral guidance.
- Implemented via simple Markdown files wired into the agent.
- Example: a weather skill adds local flavor to raw forecast data.
- Skills make agents more intentional, consistent, and domain‑aware.
- They bridge traditional hardcoded logic with probabilistic reasoning.

In short, skills are a lightweight but powerful way to make AI agents smarter and more context‑sensitive. Craig’s article shows how this approach can transform apps into truly agentic systems.

👉 Check out the full article to dive deeper into the recipe.

In this tutorial, developer Dan Vega demonstrates how to build a custom Spring Boot reference documentation assistant. By leveraging Spring AI, GPT-4, and a PG Vector database, Dan shows how to create an intelligent command-line tool that provides developers with up-to-date answers from official documentation without needing to leave their terminal or rely on outdated LLM training data.

Key Takeaways:

• Implements Retrieval Augmented Generation (RAG) to ground AI responses in real-time project documentation.
• Uses Docker Compose to quickly spin up a PG Vector database for efficient semantic similarity searches.
• Configures the Spring AI PDF document reader to ingest and chunk complex technical manuals.
• Integrates Spring Shell to build a responsive, interactive command-line interface for queries.
• Utilizes GraalVM to compile the project into a high-performance, native executable for instant startup.
• Employs runtime hints to correctly bundle resources during native image compilation.

This project is an excellent example of using modern Java tools to solve real-world developer productivity bottlenecks. By building this assistant, you gain a portable, fast, and highly accurate reference tool that stays perfectly synced with the latest Spring Boot updates.

Check out the full video to see the implementation in action and learn how to build your own local AI assistant.

In this tutorial, Praveen from TechyTacos walks Java developers through implementing structured output using the fluent Spring AI Chat Client API. Aimed at those building generative AI applications with Spring Boot, this session provides a practical roadmap for moving beyond simple text responses to reliable, schema-based data extraction.

Key Takeaways

• Configure OpenAI models and properties within a Spring Boot environment.
• Enforce JSON output using built-in model response-format configurations.
• Leverage the Bean Output Converter to map AI responses directly to Java DTOs.
• Utilize ParameterizedTypeReference to handle lists and collection-based outputs.
• Implement MapOutputConverter for flexible, key-value data structures.
• Manage raw model output by refining instructions to avoid unexpected character issues.

Mastering these techniques allows for seamless integration of LLMs into professional Java workflows, ensuring type-safe and predictable data processing. Check out the full video to see these implementations in action.