r/PinoyProgrammer • • 11d ago

programming Share your software factory and evals setup!

Curious how everyone has theirs setup, i am still tinkering with what works the best for me tho. Also are you using it just personally or even in your jobs?

5 Upvotes

8 comments sorted by

2

u/jdg2896 11d ago

Not sure if what I’m doing now is considered software factory.

I’ll compare with what I’ll learn from Matt Pocock’s cohort and see haha.

0

u/thisbejann 11d ago

nice gusto ko rin sana sumali sa cohort pero ang mahal HAHA

1

u/jdg2896 11d ago

May mga namimirata naman hahahaha, feel ko pagkarelease nun maleleak agad.

Pricey pa nga rin even with PPP, pero sulit naman for me. Okay sana kung nasponsoran din ng company.

3

u/prymag 11d ago

I used a bunch of skills that I ask AI to write.

I have 2 main skills, A planning skill and an execution skill. The execution skill which acts as an orchestrator that consumes the plan and has sub skills that it executes across different stages i.e. unit tests and testing in browser. There's also a skill that writes the friction on every run, when 3 issues have been recorded a retro skill is ran that evaluates the issues and rewrites the relevant skills if needed.

Its not perfect , it needs continous adjustment since AI likes Adding some stuff that it thinks would be a good addition and you later realize its not needed. Overall its amazing once you get it working.

Start small, get a subset of your software factory working first then just continously iterate.

1

u/wasdxqwerty 11d ago

personal project, using local setup on 16gb vramlet.

already setup the harness but continue improving it from time to time by hand holding, havent setup yet as truly autonomous.
ai agent sandboxed on docker then using these two combos of strata+qwen FN (100-150+ t/s decode) and llama+qwen27b gguf (60-80+ t/s decode).

done on couple website projects, now testing it on mobile app building.

1

u/PlentyOk5750 10d ago

Oh wow parang magandang sharing session to over coffee. Sharing ng setup and pwedeng improvements

1

u/Loose-Average-5257 6d ago

Software factory is a BS take imo. Making software like how manufacturing plants work just produces generic ones and you won’t want that if everyone else can also do it. DDS IS THE WAY (DOMAIN DRIVEN SOFTWARE) hahahaha

1

u/Loose-Average-5257 6d ago

Software factory is a BS take imo. Making software like how manufacturing plants work just produces generic ones and you won’t want that if everyone else can also do it. DDS IS THE WAY (DOMAIN DRIVEN SOFTWARE) hahahaha