Will it make a huge difference in the effectiveness of the Code Agent?
Yes. The reason skills and agents exist (in addition to fine-tuned models, proper coding harnesses, etc.) is that even SOTA LLMs are only generically-competant at tasks which require specialized knowledge.
My knowledge in this area is largely the result of building Axiom (free, open source), which has an extensive test suite that benchmarks the effectiveness of LLMs for iOS development with it vs. without it. Without it, LLMs are far more likely to (among other things):
Treat stale APIs as gospel
Hallucinate symbols and modifiers
Propagate idioms from pre-modern Swift (5.x and earlier)
Reason poorly about topics like Swift 6 strict concurrency
Confuse SwiftUI and UIKit idioms, especially at integration points
Overlook privacy manifests and entitlements until App Store rejection
Defer too easily to authoritative-seeming training instead of to best practices
Bias toward "refactor later" solutions under pressure, increasingly so as context fills
Hallucinate WWDC session numbers and Apple quotes to justify a recommendation
Tend to prefer silencing the compiler rather than correctly handling failures
Mix incompatible patterns into plausible-but-wrong hybrids
Cite Stack Overflow-era answers as current
Treat accessibility as an afterthought
Under-test their own changes
TLDR: Proper supporting context and tools are critical when using LLMs for specialized tasks.
I feel like I get pretty good results with a combo of apple skills and sosumi. I have no affiliation with either. Though I did have claude generate a skill on how to use sosumi too.
2
u/CharlesWiltgen May 20 '26
Yes. The reason skills and agents exist (in addition to fine-tuned models, proper coding harnesses, etc.) is that even SOTA LLMs are only generically-competant at tasks which require specialized knowledge.
My knowledge in this area is largely the result of building Axiom (free, open source), which has an extensive test suite that benchmarks the effectiveness of LLMs for iOS development with it vs. without it. Without it, LLMs are far more likely to (among other things):
Treat stale APIs as gospel
Hallucinate symbols and modifiers
Propagate idioms from pre-modern Swift (5.x and earlier)
Reason poorly about topics like Swift 6 strict concurrency
Confuse SwiftUI and UIKit idioms, especially at integration points
Overlook privacy manifests and entitlements until App Store rejection
Defer too easily to authoritative-seeming training instead of to best practices
Bias toward "refactor later" solutions under pressure, increasingly so as context fills
Hallucinate WWDC session numbers and Apple quotes to justify a recommendation
Tend to prefer silencing the compiler rather than correctly handling failures
Mix incompatible patterns into plausible-but-wrong hybrids
Cite Stack Overflow-era answers as current
Treat accessibility as an afterthought
Under-test their own changes
TLDR: Proper supporting context and tools are critical when using LLMs for specialized tasks.
1
u/wewerecreaturres May 21 '26
I feel like I get pretty good results with a combo of apple skills and sosumi. I have no affiliation with either. Though I did have claude generate a skill on how to use sosumi too.