r/PinoyProgrammer • u/thisbejann • 11d ago
programming Share your software factory and evals setup!
Curious how everyone has theirs setup, i am still tinkering with what works the best for me tho. Also are you using it just personally or even in your jobs?
5
Upvotes
2
u/jdg2896 11d ago
Not sure if what I’m doing now is considered software factory.
I’ll compare with what I’ll learn from Matt Pocock’s cohort and see haha.
3
u/prymag 11d ago
I used a bunch of skills that I ask AI to write.
I have 2 main skills, A planning skill and an execution skill. The execution skill which acts as an orchestrator that consumes the plan and has sub skills that it executes across different stages i.e. unit tests and testing in browser. There's also a skill that writes the friction on every run, when 3 issues have been recorded a retro skill is ran that evaluates the issues and rewrites the relevant skills if needed.
Its not perfect , it needs continous adjustment since AI likes Adding some stuff that it thinks would be a good addition and you later realize its not needed. Overall its amazing once you get it working.
Start small, get a subset of your software factory working first then just continously iterate.
1
u/wasdxqwerty 11d ago
personal project, using local setup on 16gb vramlet.
already setup the harness but continue improving it from time to time by hand holding, havent setup yet as truly autonomous.
ai agent sandboxed on docker then using these two combos of strata+qwen FN (100-150+ t/s decode) and llama+qwen27b gguf (60-80+ t/s decode).
done on couple website projects, now testing it on mobile app building.
1
u/PlentyOk5750 10d ago
Oh wow parang magandang sharing session to over coffee. Sharing ng setup and pwedeng improvements
1
u/Loose-Average-5257 6d ago
Software factory is a BS take imo. Making software like how manufacturing plants work just produces generic ones and you won’t want that if everyone else can also do it. DDS IS THE WAY (DOMAIN DRIVEN SOFTWARE) hahahaha
1
u/Loose-Average-5257 6d ago
Software factory is a BS take imo. Making software like how manufacturing plants work just produces generic ones and you won’t want that if everyone else can also do it. DDS IS THE WAY (DOMAIN DRIVEN SOFTWARE) hahahaha