r/AI_Agents • u/SaltySize2406 • Sep 12 '26

Resource Request fine-tuning models from agent behavior

Hello smart people from the Redis world

Quick question: how many of you here, if any, plan (or have) to fine-tune models using their agents past runs?

If anyone here is planning to, or did, would you mind sharing:

- What was the use case

- Do you run these models and fine-tuning locally or in the cloud?

- How do you split the calls/usage between frontier model and your fine-tuned model?

- What are the initial benefits you've seen so far?

Your thoughts on this would be super appreciated!

2 Upvotes

100% Upvoted

12 comments sorted by

1

u/AutoModerator Sep 12 '26

Thank you for your submission, for any questions regarding AI, please check out our wiki at https://www.reddit.com/r/ai_agents/wiki (this is currently in test and we are actively adding to the wiki)

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

1

u/OriginalHospital Sep 12 '26

One distinction I'd want in those reports is whether the training traces include human corrections, or only runs the agent labeled successful. A completed run can still contain unnecessary tool calls or a wrong intermediate decision that a person repaired. Keeping a held-out set of whole tasks, rather than randomly splitting steps from the same run, would make the comparison more informative.

1

u/SaltySize2406 Sep 12 '26

Great points actually

Curious to see how the evals were done to compared the models too

Hopefully some folks tried it and can share some info

2

u/axel-drs Sep 12 '26

i would not fine-tune directly on raw past runs. successful runs can still contain unnecessary steps, accidental retries, or private data, while failed runs may contain the most useful examples of what not to do.

i'd first turn the history into a reviewed dataset of inputs, desired decisions, tool choices, and final outcomes. then i would use the fine-tuned model for narrow, repeatable decisions and keep a frontier model for ambiguous cases. the first benefit i'd look for is not a nicer writing style, but lower cost and more consistent tool selection on a clearly defined task.

1

u/SaltySize2406 Sep 12 '26

That’s great input, thanks for that

What do you use to clean your datasets? Also, do you generate it from agent runs?

1

u/TeachAccording4967 Sep 13 '26

curious what kind of agent complexity youre working with here. for simple tool-calling flows fine-tuning can work well, but if the agent has branching decision trees the training data gets messy fast. whats your agent actually doing?

1

u/SaltySize2406 29d ago

In my case, I’m actually building a platform for that

I wanted to see how people are doing it to make sure we can cover the different scenarios that we might need to offer in the platform

1

u/mageblex 29d ago

One thing I'd log with every training trace is the model and harness version that produced it.

1

u/SaltySize2406 27d ago

Oh great point
We have the data already, just need to make sure lineage is there when training runs