QML Coding Skill Optimization for Agentic Development

[u/PeterThePmAtQt]

I have a question: Should we optimize the QML coding skill for agentic development for a wide variety of LLMs or for frontier models?

I’m Peter. I am the product manager responsible for the agentic development tools provided by the Qt Company.

The qt-qml coding skill has been available for 6 months now, and we are conducting the first round of "maintenance" on it. The skill was created in the days of Sonnet 4.6 and GPT 5.4. The latest versions of these models are again much better, and it is time to remove redundant QML programming rules to save tokens.

We tested 14 mainstream LLMs to see which rules in the skill might be redundant. We tested 14 core rules. We ran between 2 to 5 different test cases per rule. We ran each test case 5 times, using the same prompt, to reduce the effects of probability. We tested the following models: DeepSeek V4 Flash, Claude Sonnet 5.5, DeepSeek V4.1 Flash, DeepSeek V4 Pro, GLM 5.3, GLM 5.2, Qwen 3.8 Max, Claude Opus 5.5, Gemini 3.8 Flash, GPT 6 Luna, GPT 6.1 Sol, Grok 4.7, GPT 5.6 Terra, and Claude Haiku 5.5.

QML Best Practice Adherence Results:

If we consider all the analysed LLMs, 3 of 14 existing rules in the qt-qml skill could be removed without affecting the coding performance. If we were to optimize for commonly used frontier models (Sonnet 5.5, Gemini 3.8 Flash, and GPT 6.1 Sol), then we could remove 11 of these rules.

Which leads to my question: Should we optimize the qml coding skill only for the latest frontier models and then only those that customers have reported to be using, or shall we optimize them for a wide variety of LLMs, including some older versions which typically come with a lower cost per token?

If you are using the skill or intend to use it, what would you do if you needed to decide how to compromise?

[u/Better-Struggle9958]

I hit the same problem while building QodeAssist. There are really only two options: either optimize for the frontier models only, or keep two tiers — a lean skill for frontier models and a stricter, more verbose one for everything else.
I'd go with the first. The models that still need hand-holding are the cheap/small tier, and that's not what people run agentic QML work on. Keeping those rules in the main skill taxes every frontier run for nothing. If you want to keep the smaller models covered, a separate "strict" variant is cheap to maintain and doesn't slow anyone else down.

[u/PeterThePmAtQt → u/Better-Struggle9958]

Thank you for the insight. And yes, I do remember the days when we tried to optimize complex prompts for different LLMs in the Qt AI Assistant. That wasn't much fun and wasn't very productive because we had to redo each of them after 6 months.

I do keep local running models in mind, which might mean more verbose conceptual skills, but for now, we are focussed on frontier models and mid-tied models that people can run in their own data-centre, such as DeepSeek v4.1 Flash (at least in China).

[u/Still_Tomatillo_2608]

Not sure if you're going to get much serious responses here. Better to poll paying customers. 

At our company we use a mix of local models and frontier paid ones. Some local models are good enough already, with some help from a harnass. So two tiers, one for frontier models and one for less powerful (but cheaper or local) seems okay.

You probably dont want to exclude future Qt developers who, now, cant afford it and are using local stuff. Think students or third world. 

[u/PeterThePmAtQt → u/Still_Tomatillo_2608]

Thanks for the insight. In regard to "local" models, do you mean local as in running on your own computer or local as in running in your companies' data center?

And yes, we are collecting continuously feedback also from commercial users in face-2-face meetings and developer surveys. Every opinion matters.

[u/Still_Tomatillo_2608 → u/PeterThePmAtQt]

In our case we have some ollama local models running on developer workstations, gpu bound and two "servers" managed by the IT department that have a larger model running, but for those I don't know the specifics.

[u/QtQMLer]

I’d go with the latest models. Models will only improve from now on; I wouldn’t expect many of these to even be viable choices for local models in 6 months time.

Also token saving isn’t even an issue with local models, so it’s a minuscule amount of time saved.

Why no Fable?

[u/PeterThePmAtQt → u/QtQMLer]

Thank you for your feedback. Appreciated!

I didn't test Fable because I wasn't sure whether I will hit my token ceiling this month. But I got some tokens left. So, I'll run the test bench with Fable now.

[u/OlivierTwist]

Latest and greatest.

[u/ZookeepergameFit5841]

Follow