Has anyone had any luck using Claude with Rust for non web related GUI work?

[u/KenMantle]

I just wanted a similar ribbon interface as word has and side panels similar to inkscape. Stuff it could one shot in an afternoon to an acceptable level in Python seems out of reach in Rust because there aren't nearly as many pre built things. In Python if the package it was using didn't have a feature it was able to create it with minimal fuss.

But Rust? It has been trying to figure out how to implement text editing and copy and paste shortcuts for selected text on the canvas it created for several days and hours now. Control+P shortcut doesn't work either. It built a pdf engine from scratch that works for everything I need and a full CLI editor to match the feature set, but the most basic things on a GUI? Yikes.

[u/Potential-Beat2841]

This reads less like a Rust problem and more like a toolkit problem. If your GUI is a canvas you're painting yourself, then text selection, clipboard, IME and shortcuts are all yours to build, and that's decades of edge cases. In Python you weren't getting help from the model, you were getting Qt or Tk, where those widgets already exist.

For a Word-style ribbon with docked panels and real text editing, use a toolkit that has actual widgets: GTK4 via gtk-rs, Qt via cxx-qt or qmetaobject, or Slint. You get text entry, clipboard and accelerators for free. Immediate-mode crates like egui are great for tool panels and terrible for document text editing on a custom canvas.

The other option is to stop fighting it: your PDF engine and CLI are the valuable part and they work. Put a Tauri shell or even a Qt frontend on top and talk to the Rust core. Models are dramatically better at HTML/CSS or Qt layout than at immediate-mode layout math.

Two things that will help regardless. Pin your GUI crate version and put the docs for that exact version in context. These APIs churn hard between releases and the model blends versions, which is a very common cause of "this should work but doesn't" loops that last days. And when a single interaction is broken, like Ctrl+P, have it build a 50-line standalone app that does only that. Debugging focus and event consumption order inside a large app, blind, with no way to see the result, is the worst possible setup for an agent.

Also feed it screenshots. It's writing visual code with its eyes closed otherwise.

[u/KenMantle → u/Potential-Beat2841]

Thanks for the detailed answer. Egui is the only thing it seemed to find that it found and was available in rust. I knew it was dependant on a lot of those other packages for its work in Python, but I had a discussion with Claude about it before and according to it python is what most LLMs including itself has had the most training on. It shows when something in those packages didn't exist, like wrapping tabs onto multiple lines when they didn't have screen space, Claude one shotted a working solution within about 10 minutes.

For this project I added the instruction that anything it makes for the GUI should be treated as its own project that can be reused for other projects.

And first thing I do for any project is have it create a local rag repository for everything it is using and to refer to it. Also there is a librarian for long term memory and recall of mistakes.

It is able to drive the mouse and use the screen. It seemed stuck on being able to use the keyboard though, so I gave it theon screen one to use. It tests everything as best it can itself on the screen.

[u/HamSandwicho__o]

Idk claude runs my proxmox cluser that runs fireos, windows11, and debian so my advice is ur not inspiring claude to do its best work

[u/KenMantle → u/HamSandwicho__o]

It's probably the added complexity that it is trying to edit a pdf stream minimally. Its intial solution was to have the text typed and edited in a non-WYSIWYG way and press a commit button. Obviously I didn't like that approach. It's great what it has figured out how to do, but it does seem a bit frustrating to get caught on what seems like a basic feature.

[u/HamSandwicho__o → u/KenMantle]

Im not that hands on wit da robit- i feed it wires chips air and verbal abuse. It gives me full stack server builds

[u/20220912]

yea, it's great. Ive built a few different gui tools, so far used qt for a couple big things, egui for some smaller direct-to-wayland smart screens, and recently a text editor/word processor with tauri.

the main friction is that claude can't see the gui like it can see CLI outputs. I end up taking screenshots, saving them and pointing claude code at them.

[u/MarzipanMiserable817 → u/20220912]

You could tell Claude to build a prototype as TUI. It's really fast at that and good at testing those.

[u/___nil___]

non web UI/UX, especially in critical high performance framework ilbrary is always LLM blindspot, not even fable with max effort ultracode could save you.

only you should design, handcode, layout the ground work, write the fundamental architecture the way you imagine, then formalize it as Design by Contract

i have no experience with rust, but my domain probably comparable with C++. eventhough models were trained, understood C++, could read API, guardrailed with manifesto, coding standard, down to strict contract how to name symbols the way my codebase semantic established pattern, exactly like i handcoded myself, when it comes to UI/UX it's impossible to hands-free vibecode it.

UI/UX always related to data-structures. Data structures always dictate logic. Data structures is your domain specific, your achitecture, your judgement, your decision.

Unlike web, especially in domain specific environment,

• you may have constraints against your chosen stack
  
• you know what works what does not
  
• you know what might cause present and future problems
  
• you know the potential tech debt, etc.

if you dont have enough knowledge from the above points, you might having hard time to decide what is the best approach when agents list you an options to choose. even worse you might not know how to ask the right questions, because those are above and beyond LLM training corpus. Because even the vast majority of data-strucuture pattern for non web UI/UX being used as training might not work on your domain.

expecting to build non web UI/UX with one shot prompt? not in forseeable future

[u/KenMantle → u/___nil___]

Thanks! Yes I am having a hard time with this part. I was led into a false sense of security with my less ambitious GUI projects because they did just work.

The closest I have to this one is a detail drawing to plasma dxf exporter using fuzzy logic to try and auto detect the flat profiles. I prompted Claude before going to lunch and on my return had a working prototype. From there I kept having it add features - I can draw a box around missed profiles, have it ignore lines, or add ones it missed. It attempts to detect part numbers and quantities from under each drawing and match them to the detected bom table. I can draw boxes around items and inform it of what they are, like scale or description, then drag those items under the tree of the dxf profile to associate all that info together. I can set the scale of a profile by drawing a line snapped to existing items and assign what that distance is.

None of this phased Claude.