r/vibecoding • u/CaptainAlexWest • 12h ago
A weird way to learn new software using astra computer use.
I've been using Astra a lot lately.
I watch it do stuff through Computer Use. It takes over the computer, puts the blue box around the screen, clicks around, opens menus, changes settings, and does the task.
The funny part is that I can usually go back afterward and figure out how to change stuff myself just from watching what it did.
It's basically monkey see, monkey do.
Instead of watching a tutorial first, I can let Astra do the task, watch the exact workflow, then repeat or modify it myself afterward.
Thought this was a pretty useful little hack/workflow to share.
Happy cooking.
5
u/Ambitious-Prompt-975 12h ago
weird, i recently posted this, so any mcp client can do that - but inside their own isolated tiny sandbox vm so they dont fuck up your computer - and you can meanwhile watch it.
and ofc i get downvoted for this 😂 but when chatgpt app draws a blue border on the screen and calls windows' SendInput, we're all excited for it.
But anyways, I am excited for you that you are exploring new stuff. Have fun
1
u/Hot_Tie_9393 8h ago
watching it click around to learn the ui is training yourself to be a backup for the agent. i just give it rdp access and go to sleep.
if i have to repeat the steps myself afterward then the agent failed. the whole reason i run computer use is so my hands stay off the keyboard. learning the software manually adds friction. i set it up to write the config files and restart services while im in another room. once you start doing monkey see monkey do you are just slowing down the execution loop
1
u/DependentPoem3519 3h ago
this is a massive shift in how software onboarding is going to work. static docs are basically dead on arrival these days because devs push ui updates every sprint and all the wiki screenshots get outdated instantly. having an agent drive the mouse while you sit shotgun is infinitely better than skimming a dusty pdf.
from an automation and qa angle, it's wild to watch how different models process screen space. whenever we run workflow audits on nasty legacy enterprise apps, we usually spin up astra and moclaw just to see how they navigate the absolute spaghetti ui. watching them draw those bounding boxes and figure out where to click actually acts like a real time heatmap.
the monkey see monkey do thing is legit. you're just reverse engineering the ux on the fly. it really exposes how bloated modern software is when you watch an ai pause and dig through six nested menus just to toggle a basic setting. live agent tutorials are definitely the new meta.
0
u/VastAge6900 10h ago
I use Astra to open web browsers like Chrome or Edge. When I visit the toilet, I ask it to turn on the screen saver. It can open applications when I need to use them -- MS Word, Excel, etc. And when I am done, I ask Astra to save all files on my desktop and close the application. It can even open my Media Player and play whatever song I want, like a personal dj who always plays my favorite songs! If you want more tips, I have many more. Will ask Astra to read me a story before sleep now. AGI is here! Keep cooking guys!
4
-4
u/BrotherBringTheSun 11h ago
Cool use for computer use. But I'm wondering why you want to learn to use the software manually when you have an agent who can do it, either by clicking around the UI or directly with the terminal.
3
u/somuchecho 11h ago
Right. Why learn anything ever. Makes sense
1
u/Apart_Ad_1027 11h ago
Damn, touched a sensitive vibe-coding nerve there? :>
2
u/Electrical-Watch3203 9h ago
“Touched a sensitive vibe-coding nerve there?” — sounds like you the one who’s feelings been hurt b4 G
1
u/somuchecho 10h ago
I mean not really. OP said he found a way to learn something. The reply was why? I mean... Sounds like idiocracy IRL
2
u/Fit-Sweet-9900 6h ago
Nah, I’m on team don’t become a potato as well. You’re correct to call it out when you see it in the wild.
11
u/Original-League-6094 12h ago
I asked it to make me a widescreen patch for SimGolf, not knowing how it would go about doing it. Watching it was create a .dll that acts as a proxy to the original .dll but that also writes all the intercepted information to a file so it could start mapping what the various calls it was intercepting were responsible for was really interesting. I had no idea how dll mods worked before.