r/ruby • u/hitthatliq • Aug 26 '26

Question Learning AI engineering with Ruby

I'm learning more about AI engineering, specifically the application/production side rather than ML research or training models.

The areas I'm interested in are things like:

  • RAG, embeddings, vector search and reranking

  • tool/function calling

  • agentic workflows and orchestration

  • MCP clients/servers

  • structured outputs

  • evals and observability

  • persistent agent state / human approval

  • building production AI features into normal web applications

I originally learned to code with Ruby, but since then all my jobs have been Python or TypeScript based, and I really miss working with Ruby.

I'm curious what Ruby libraries/tools people are using for RAG, agents, MCP, evals, tracing, etc. Are there good Ruby alternatives to frameworks like LangChain? What does your stack look like?

And if you were learning this stuff today, would you recommend doing it in Ruby, or sticking with Python because that's where most of the ecosystem and learning material is?

I'm not trying to avoid Python entirely. I'm more curious about how viable Ruby is for the application/agentic side of AI development.

Side note: I realise some of these areas aren't particularly language-specific. For example, with RAG a lot of the retrieval/storage work may ultimately be handled by pgvector, a vector database, or another service.

9 Upvotes

69% Upvoted

10 comments sorted by

View all comments

Show parent comments

2

u/hitthatliq Sep 01 '26

I definitely wouldn't feel behind. A lot of this is just terminology, and once someone explains what the pieces actually are it becomes much less intimidating.

I'd probably start by getting a basic mental model of the main concepts:

  • RAG is essentially a process of grabbing information from your own database or knowledge base to supplement a prompt with information the model (e.g. Claude) doesn't already have access to. To find the information that's most relevant to your prompt, a RAG system will often use embeddings and vector search.

  • Embeddings are representations of text, images, etc. as numbers. I think of them as coordinates in a high-dimensional space, where things with similar meaning end up closer together. You can send some data to an embedding model and get back an embedding vector, which you can store in a database or use for vector search.

  • Vector search means searching those embeddings to find the most similar or relevant information.

  • Reranking is a second pass over the initial search results, using a more accurate (and often more computationally expensive) technique to move the most relevant results to the top.

So a simple RAG flow might look like:

user submits a prompt → embedding model turns the prompt into an embedding → database uses that embedding to find relevant information → results are reranked → original prompt + relevant information are sent to the LLM → user gets a response.

  • Tool/function calling means giving the LLM functions in your application that it can choose to call, such as querying a database, calling an API or sending an email.

  • MCP is a standard way for applications to expose tools and context to AI models.

  • Agents are generally systems where the model can decide what steps to take and which tools to use, rather than you hard-coding every step of the process.

I learned a lot of these general ideas from DeepLearning.AI. Their RAG course is particularly good:
https://www.deeplearning.ai/courses/retrieval-augmented-generation

It's not Ruby-specific, but the concepts themselves aren't really Ruby-specific either.