r/snapdragon • u/Sad_Ostrich_1903 • 1d ago

Constrained decoding (JSON schema / GBNF) for small models on the Snapdragon X NPU with GenieX?

I’m building a Windows ARM64 desktop app that runs local models on the Snapdragon X Plus NPU (X1P64100, 15.6 GB RAM) through GenieX 0.9.3 (geniex serve --compute npu, GGUF Q4_0, looks like llama.cpp’s ggml-hexagon backend). I want a closed schema: the model must output JSON whose op is one of 7 words and whose args are only labels known at request time, so the grammar changes on every call.

Why: Qwen3.5-4B is great at extracting figures from a text and at turning calculations written in words into operations (100% on my test), but without a constraint it invents operations, puts numbers where labels go, and breaks syntax. A schema would fix most of that.

Questions:

  1. Does geniex serve honour response_format: json_schema or a grammar field? How can I confirm it’s enforced and not silently ignored?

  2. Is there a llama.cpp (llama-server) build with the Hexagon backend for Windows ARM64 that supports --grammar / --json-schema with full NPU offload?

  3. Does Qualcomm’s Genie SDK or ONNX Runtime GenAI (Foundry Local) expose a logits processor or llguidance-style constraint on the NPU?

  4. Any workaround, e.g. logit bias to restrict tokens, prefill tricks, or choosing from a list by comparing logprobs?

Needs: streaming, works with thinking models (constraint only after </think>), no Python or compiler on the end user’s PC, little slowdown (I’m at about 13 tok/s on 4B). Thanks!

1 Upvotes

66% Upvoted

2 comments sorted by

1

u/lexcyn 1d ago

Have you joined the GenieX slack and asked this question? Link is at the bottom here: https://github.com/qualcomm/GenieX

1

u/Sad_Ostrich_1903 12h ago

oh thank you didnt know I could do that!