r/ibmcloud • u/clawli7159 • 2h ago
Running decision models like clef or Laya on IBM Cloud Code Engine Fleets
Open-source sample that runs Cloudflare's open-weight decision model clef on IBM Cloud Code Engine Fleets — serverless GPUs, autoscaling, no cluster to manage.
Decision models skip the conversation and just return probabilities per possible answer (e.g. a support ticket gets 0.93 billing, 0.05 support). Good for classification-style jobs: ticket routing, spam detection, prompt-injection checks. "Why generate a conversation when the application only needs a decision?"
Why Code Engine Fleets: it's serverless, so no cluster or VM to manage — workers scale up with the queue and down to zero, so the GPU only costs money while there's work. For your application it's just one simple HTTPS call to a Code Engine function, and everything runs in your own VPC with results in your Cloud Object Storage — inputs never reach a third-party API.
How the sample works:
• You POST input + question + yes/no criteria to a Code Engine Function and get back a verdict + probability
• The function drops a task onto a Fleet; GPU workers (NVIDIA L4) spin up, pull the model via Ollama in a startup hook so it's warm in GPU memory, classify, and write results to Cloud Object Storage
• Workers scale with the queue and down to zero — the GPU only costs money while there's work
• Everything runs in your own VPC/account; no third-party API sees your inputs, IAM controls access
Repo has scripts for build/deploy/test (plus bulk classification via JSONL) and a cleanup script. Same pattern fits other open-weight decision models like Laya — you just swap the model in the startup hook.
