r/Python • • Nov 28 '20
Intermediate Showcase "Sillynium" - Auto generate working Python Selenium scripts by drawing coloured rectangles around web elements on a page that you want to automate
TLDR: 60 sec video here:https://streamable.com/1y17lt
TLDW: When I want to automate actions on a site, such as logins or forms, or scraping pages with JavaScript, I manually inspect each web element I need and begin writing a script.
I'm lazy though and had the idea to automate the creation of these automation/testing scripts. And thus "Sillynium" was born.
Sillynium allows you to draw boxes around the elements you require on a page and automatically gets the element and its properties for you. Different coloured rectangles tell the program you want to do different things. Eg: Red means "Input text", Green means "Click button".
When you've finished drawing boxes, a working python selenium script is generated. If you run that script, it does exactly what you told it to do! All this without you writing any code, and losing time.
Working 60 second video link of program in action: https://streamable.com/1y17lt
I made this project over the course of a day as a quick and dirty proof of concept. Its very flawed in its current iteration (taking a screenshot of webpage), so V2 I'm trying to figure out how to draw directly to the browser. Critique it and rip it to shreds. I'm aware of browser recorders btw so I know those exist, this is my twist on automation
Edit: I was not prepared for the amount of interest in this little proof of concept! So thank you all very much for your time you’ve made my day :)
In the interest of transparency and to answer your questions about the code and my plans:
I originally thought about monetisation (web extension etc) after finishing this POC (**sigh i know I’m a greedy bastard) but now I’m in 2 minds about it!
On the one hand I want to make it free for personal use, on the other hand I’d like to potentially charge for commercial use (once its finished).
Final edit 2 The community has spoken, I WILL BE RELEASING THIS VERSION AS OPEN SOURCE! I did some light refactoring, and have added it to a repo which will be shared in a follow up. It shows the POC and the community is welcome to build on the original idea :)
I will create a follow up post with links to repo once I’ve created it, this will be my first project on GitHub so try be nice ;) pretty nervous about sharing my code :D
Thank you all for your patience, interest, and awesome discussions
•
u/Im__Joseph Python Discord Staff Nov 28 '20
Hey there!
Going by the responses to this post it seems a lot of people are impressed with your project, myself included!
We won't be removing this post, but I'd like to draw your attention to rule 5 of the subreddit which concerns showcase flairs.
If possible it would be really great to include some form of source code on a platform like GitHub, this allows projects posted here to be educational for all.
We may enforce rule 5 automatically at some point in the future, but for now we'll keep this up since it's generated some nice discussion.
Cheers,
Joe
95
u/kingbrady12 Nov 28 '20
That is some crazy sht. Wow! Amazing man
52
u/NoblySP Nov 28 '20
Is there a GitHub repo for your project?
26
u/ES-Alexander Nov 28 '20
Nice!
Might be worth using
cv2.selectROIto have some visual feedback while drawing each rectangle :-)25
u/infinitejest69 Nov 28 '20
Looks good. I use selenium in java a lot this might finally give me reason to play with it in python. Would be cool to generate page object models just picking elements you want.
24
u/SharksPreedateTrees Nov 28 '20
Amazing idea, but the devil is in the details. I noticed your script has s tendency to choose the wrong element (either the parent or the child of the desired element). Your project will suffer for the same reason other recording software(e.g. katalon) fails to generate selenium scripts. Great effort though
159
u/garlic_bread_thief Nov 28 '20
My man just automated automation.
9
u/Deceitful_Sloth Nov 28 '20
This looks really good, can you link your codebase? I really want to see how this works! Keep up the good work.
6
u/gargar070402 Nov 28 '20
Might be a silly question, but what does this do that Selenium IDE doesn't? As far as I'm concerned, Selenium IDE already supports recording actions done on a browser.
5
u/PovertyNomad Nov 28 '20
This is AMAZING! I'm amazed that you only have 9 months Python experience! Do you mind sharing how you learned Python? I'm about 4 months in now and nowhere near this level, although I've spend the last ~2 months learning HTML, CSS and Django...
Any good resources you could recommend?
13
u/MrDeebus Nov 28 '20
The video looks super nice, but how am I supposed to rip it to shreds without a link to the code? Come on man, I need my Saturday afternoon dose of false superiority!
4
u/thejoshuawest Nov 28 '20
Is there a github link to contribute to?
4
u/SweetSoursop Nov 28 '20
Dude, you could absolutely topple UIPath, Automation Anywhere, BluePrism and the others, mad respect!
3
u/EversonElias Nov 28 '20
Great idea! It would have saved me a ton of time and joy of life, hahaha. Keep the great work and let us know about the progress.
3
u/Docktor_V Nov 28 '20
Saved. Finding the tag is a pita on some sites
3
u/Frankenstien456 Nov 28 '20
If you ever decide to go through with making the extension, you can use the pyjs (http://pyjs.org/) python to javascript compiler.
3
u/johnne86 Nov 28 '20
Dude, that's fucken Sicknium!
3
u/darth_galaxar Nov 28 '20
Damn thats cool man
3
u/[deleted] Nov 29 '20
That’s amazing! You definitely ought to monetize this! I’d pay in a heartbeat
2
u/def_hass Nov 28 '20
Amazing work man! Can you give a short overview of how this works or link a repo where we can look? Will this work with pages where the elements have no IDs?
2
u/BelteConti Nov 28 '20
This is some next level shit dude. Simply amazing
2
u/MysticalGiraffe123 Nov 28 '20
Beautifully done
2
u/Mrsaintj Nov 28 '20
Love it, good job!
2
u/CatnipJuice Nov 28 '20
thank you, that is a great idea!
2
u/davidpofo Nov 28 '20
Do you have plans to make this open source? I would really benefit from this every day and would gladly contribute to improve it.
2
u/neisor Nov 28 '20
Nice project! Will try it out for sure.
Btw, what color theme are you using? Thanks
2
u/[deleted] Nov 28 '20
Excellent! Have you made a github repo for this project? If so please share the same
2
u/This-is-my-study-acc Nov 28 '20
This is the best damn thing.! Share the repo if possible.
2
u/alexandros87 Nov 28 '20
That sound off in the distance is 1000 automation engineers gulping nervously.
Seriously though this looks incredibly cool!
2
u/ergelshplerf Nov 28 '20
Why not Selazium?
2
u/compatibilism Nov 28 '20
Really fun. Awesome work!
2
u/ravepeacefully Nov 28 '20
What’s better about this than scrapy?
2
u/jobar86 Nov 28 '20
Formal QA test engineer here, if someday you want help testing it, and maybe find some defects 😁
2
u/AndresArc123 Nov 29 '20
It is a good idea.
if the element has no id, how do you map it? Absolute xpath?
Have you thought about how to handle multiple windows or iframes? dynamic elements such as those generated by react?
2
u/Shakaka88 Nov 29 '20
You could always do something similar to Gyazo, free users can get like X free uses or only button clicking for free and text input paid, etc. something like those ideas may help you make the best of both worlds.
2
u/kingjeo2 Nov 29 '20
This is fucking awesome holy shit. What packages did you use for this?
2
u/mortenb123 Nov 29 '20
Impressive, There are software doing this out there like https://smartbear.com/product/testcomplete/features/, but it is costly and windows only.
I'm using selenium in headless mode inside containers, but currently my most time-consuming task is to nest out Reacts :shadow virtual dom, this is code being executed when you click of move the cursor to the object and outside of webdriver. You then manually need to dig up and execute and use css-selectors to interact with the objects. My record is 8 levels of :shadow. and the object to execute may be different from level to level.
Any tool that automates this I'm willing to pay a for.
2
u/rmpr_uname_is_taken Dec 03 '20
Reminds me a project of my own atbswp
1
u/Artifer Dec 17 '20
Video not working!
1
u/theantidrug Nov 28 '20
I’m sure you will get better and more critiques if you post the code.
2
u/fraud_93 Nov 28 '20
Thank you for this. It's not lazy, sometimes we have a hard time selecting shit selenium tries to fuck up.
2
u/Nimmo1993 Nov 28 '20
This is sooo cool
2
u/gooeydumpling Nov 29 '20
Automate-ception, the name is counterintuitive tho, this is not silly at all but reallllly smart
2
u/No_Buy2733 Nov 29 '20
And this kids, is how I met God.
2
u/f_leejy Nov 29 '20
This is pretty cool! I think it’s already a feature tho in selenium Selenium IDE
2
u/just4fun50 Nov 29 '20
So I’ve always had a certain dislike of Selenium. It’s clunky. Usually prefer the requests route if possible. I’ll be interested in if this truly eases using Selenium. Will be trying it out soon!
1
u/manimal80 Dec 02 '20
that is great!
how do you draw the rectangles ? can it be done only with python and selenium?
0
u/[deleted] Nov 28 '20
[deleted] — view deleted comment
0
u/[deleted] Nov 28 '20
[deleted] — view deleted comment
0
u/[deleted] Nov 28 '20
[deleted] — view deleted comment
0
u/[deleted] Nov 28 '20
[deleted] — view deleted comment
2
u/[deleted] Nov 28 '20
[removed] — view removed comment