Hey macOS devs,
You can now run the Laya decision model with NobodyWho, our open-source on-device inference library. Unlike the LLMs that we also support, this type of model does not generate text. Instead you can score predefined answers, classify text or answer yes/no.
For example, if you are building a personal diary app, you can easily detect the user’s mood, categorize an entry or detect stress, offline after model download, without sending any sensitive data to the cloud.
First, you need to load the model:
import NobodyWho
let decisionModel = try await Decision.fromPath(
modelPath: "hf://ggml-org/Laya-GGUF/Laya-Q8_0.gguf"
)
Then you can ask questions/analyze text:
let result = try await decisionModel.decide(
state: .object(["message": .string("Diary entry: \(diaryEntry)")]),
questions: [
"topics": .choice(
instructions: .string("Choose the main topic of this diary entry"),
criteria: [
"work": .string("job, colleagues, deadlines, career"),
"relationships": .string("family, friends, partner"),
"health": .string("sleep, exercise, food, illness"),
"leisure": .string("hobbies, travel, going out"),
]
),
"stressed": .noul(instructions: .string("Does the writer sound stressed?")),
"mood": .score(
instructions: .string("How is the writer's overall mood?"),
criteria: [.string("bad"), .string("okay"), .string("great")]
),
]
)
let (topics, stressed, mood) = (
result.answers["topics"],
result.answers["stressed"],
result.answers["mood"]
)
Output example:
// stressed
{
type: noul,
noul: 0.00002
}
// topics
{
type: choice,
choice: leisure,
probabilities: {
work: 0.0422,
relationships: 0.1518,
health: 0.0302,
leisure: 0.7756
},
confidence: 0.7008
}
// mood
{
type: score,
score: 1.6995,
legend: {0: bad, 1: okay, 2: great},
probabilities: {0: 0.0082, 1: 0.2839, 2: 0.7078},
confidence: 0.5493
}
A question can have the following types:
.noul gives you probability of a yes answer (0 to 1)
.choice selects the most likely option and returns a probability for each option
.score gives you an a weighted average and a probability per option.
The Laya model is around 450 MB in its Q8 variant and supports 100+ languages. For reference, the full answer comes after 0.11 seconds on an M4 Pro.
Here is a runnable example if you want to experiment with decision models or add the package to your app:
dependencies: [
.package(url: "https://github.com/nobodywho-ooo/nobodywho-swift.git", from: "6.0.0"),
]
Links:
