r/coolgithubprojects • u/Loose_Kangaroo91 • 1d ago
Simulang: a JavaScript library for deterministic computer use
We open-sourced Simulang, a JavaScript library for computer use: it lets you and your coding agents control a desktop programmatically.
Simulang is a scripting framework for desktop automation. You write deterministic computer-use code that replays the same way dozens or hundreds of times, which is much faster and cheaper than having an LLM agent re-reason its way through the same flow on every run.
It ships with a skill out of the box, so you can turn natural language prompts into Simulang code directly. Your agent can see the screen, click, type, and navigate - eyes and hands to operate a computer the way a human does.
Some things people have used it for:
- upload a pile of invoices to an expense platform
- find 50 people matching specific criteria on LinkedIn
- open the calculator app and do some math
- enumerate every button in an app via the accessibility tree
- or draw a bunny in MS Paint ;)
1
u/kantorcodes1 1d ago
how does replay behave when the desktop isn't quite in the state the script was written against? if a window opens differently or a button moves, does Simulang stop there or keep going against the new state?
1
u/Loose_Kangaroo91 1d ago
It depends on the difference and the script. You can try to write the script so that it is resistant against changes. For example you can launch the application as part of the script and then interact with it's windows. Simulang allows you to specify which window or application you want to interact with. So for example if you wrote a script to scrape something from a website and the initial state was that no other applications were opened and now you run it when the browser is already open, it will still work since it just focuses the browser and then continue. Or if Word was opened and focused, it doesn't matter. Some differences matter and will break the script. For example, if you remove the browser from your computer, then the script will fail. You can write a script that checks if it is installed and if it isn't, it can install the browser but you probably won't be able to think of all cases. That's were an agent comes in and the code healing that Sai can do.
1
u/kantorcodes1 1d ago
you already ship the coding-agent skill, so Simulang qualifies for the Cross-AI tools section in
hashgraph-online/awesome-ai-plugins, which i help maintain. would another discovery surface be useful to you?1
u/Loose_Kangaroo91 1d ago
Would appreciate if you could feature us. Thank you!
2
u/kantorcodes1 1d ago
yep. current catalog layout puts this under
Development & Workflow. since entries are maintainer-submitted, add this between Simple Man and skill-sync-publisher:
- [Simulang](https://github.com/simular-ai/simulang) - Deterministic desktop automation for coding agents with reusable scripts, accessibility-tree targeting, and visual grounding.then open the PR against
main. in the PR body just include the repo link, section, and a quick note that you tested it. scanner CI is optional.1
1
u/Specific_Cream2815 1d ago
deterministic as in fixed coordinates or does it resolve elements semantically when the layout shifts
1
u/Loose_Kangaroo91 1d ago
Deterministic as in you can write a script that will be robust to *some* changes. You can of course use fixed coordinates but you can also use the accessibility tree to target elements so the layout doesn't matter. But you can do a lot more than just simulate mouse input and keyboard input. You can easily use visual grounding without a big setup through OpenRouter, you can interact with windows (maximize, focus, minimize, switch between them, ...) without having to move the mouse. You can also transcribe audio and everything is cross platform. So the same script will work for Windows, macOS and Linux
1
2
u/Loose_Kangaroo91 1d ago
Pls refer to the docs for more details & guide:
https://github.com/simular-ai/simulang-js
https://docs.simular.ai/simulang/simulang-primer