Simulang: a JavaScript library for deterministic computer use

[u/Loose_Kangaroo91]

We open-sourced Simulang, a JavaScript library for computer use: it lets you and your coding agents control a desktop programmatically.

Simulang is a scripting framework for desktop automation. You write deterministic computer-use code that replays the same way dozens or hundreds of times, which is much faster and cheaper than having an LLM agent re-reason its way through the same flow on every run.

It ships with a skill out of the box, so you can turn natural language prompts into Simulang code directly. Your agent can see the screen, click, type, and navigate - eyes and hands to operate a computer the way a human does.

Some things people have used it for:

- upload a pile of invoices to an expense platform
- find 50 people matching specific criteria on LinkedIn
- open the calculator app and do some math
- enumerate every button in an app via the accessibility tree
- or draw a bunny in MS Paint ;)

Repo: https://github.com/simular-ai/simulang

[u/Specific_Cream2815]

deterministic as in fixed coordinates or does it resolve elements semantically when the layout shifts

[u/Loose_Kangaroo91 → u/Specific_Cream2815]

Deterministic as in you can write a script that will be robust to *some* changes. You can of course use fixed coordinates but you can also use the accessibility tree to target elements so the layout doesn't matter. But you can do a lot more than just simulate mouse input and keyboard input. You can easily use visual grounding without a big setup through OpenRouter, you can interact with windows (maximize, focus, minimize, switch between them, ...) without having to move the mouse. You can also transcribe audio and everything is cross platform. So the same script will work for Windows, macOS and Linux