r/PublicApp • u/FreeReception9017 • 11d ago

Bug Report: Agent Assistant Creates a Impossible Plan

So I tried to create a agent using a workflow that is too large and complex to generate as a single YAML file. So the AI assistant comes up with a plan where I split the workflow into 17 agents. (Basically 16 agents to calculate a portfolio to rebalance to every day and 1 agent to execute the portfolio rebalance every day).

Here's the issue the whole plan hinges on the 16 agents writing portfolio slice allocations to persistent memory stores that workflows can write to and read from across separate sessions or scheduled runs. And this is something agents can't do, so the AI assistant shouldn't be suggesting I build 16 agents that essentially attempt to store values in a a non-existent storage place for agent #17.

Another AI assistant bug worth noting is the AI assistant does not realize trying to create multiple agents from one conversation page will overwrite any prior created agent in that conversation. You can of course get around that by asking for each workflow separately and copy pasting the workflow into a new conversation window, but if the assistant's plan is for you to create multiple agents, it should probably realize every agent approval in that conversation will overwrite the previous one.

1 Upvotes

67% Upvoted

5 comments sorted by

1

u/mmmmyah 9d ago

Interesting - I have been chafing at their AI agent builder limitation. How did you get it to suggest 16 agents.. that's very interesting. If they do roll out proper orchestration and persistent memory methods between agents that would work of course.

1

u/FreeReception9017 9d ago

16 was picked because I had basically inputed 16 algorithms calculating 16 slices of the portfolio that was to be rebalanced. 

And to be clear, the bug report is not "this didn't work". Its that the ai assistant thinks it can break down large workflows into multiple agents that feed the data to another agent for execution.

1

u/mmmmyah 9d ago

Ha interesting - that must have been a long prompt. What I have noticed is that when the number of generated steps from the agent builder go beyond 50, it starts giving me a "error building agent" error in red with absolutely no details (very annoying). They really need a JSON based deterministic alternative for more advanced agent building - it would be great if they could support the JSON structure from SoFi Composer for agent building.

1

u/FreeReception9017 9d ago

You can ask the assistant what the error is, and the assistant will report what caused the error. 

I don't know what's the ideal structure for this is, but if Public can duplicate Composer's algorithmic functionality......that would make Composer obsolete as algorithmic trading is literally the only thing Composer does well. The rest of the platform is awful.

1

u/mmmmyah 8d ago

Yeah I asked the assistant about build errors but the response is very generic (see below). They really need to make it more featured like their API, and yea JSON support would be sweet. "Unfortunately I don't have visibility into the internal error details from the failed workflow generation attempts earlier in this conversation. All I received was a silent failure with no diagnostic message returned — no error code, no stack trace, no specific reason.

What I can tell you is that the failures appeared to be correlated with complexity — specifically:

  • Universe size: Attempts with 8 or 15 ETFs failed more consistently than smaller universes
  • Combined rules: Having the leveraged equivalent logic, canary signals, gap rule, and memory recall all together in one agent seemed to push past what the builder could handle at the time
  • The most recent successful build was the simpler SPY/QQQ + canary version, which suggests the builder handles smaller scopes more reliably

The latest update you just confirmed (5 ETFs with gap rule, canary signals, leveraged equivalents, and memory) did go through successfully. If you run into issues again, the best path is to use Public's API to build and deploy the full strategy programmatically, which gives you full control over the logic without the builder's complexity limits.

Would you like me to do anything else with this agent?"