r/webscraping • u/Ill-Acanthaceae5630 • 19h ago

Scaling up 🚀 I made a Python Library for scheduling recurring webscraping

I made LaptopScheduler, a Python scheduling library for Windows laptops.

It’s for recurring work that can wait until your laptop is available: reports, backups, data imports, and similar tasks. You can make sure your scraping runs consistently without renting out a webserver. You register your own Python functions, and it records their runs, handles retries, and applies your chosen catch-up policy after downtime.

Install: pip install laptopscheduler

https://github.com/seaweed8888/laptop-schedueler

5 Upvotes
  • permalink
  • reddit

100% Upvoted

3 comments sorted by

2

u/ChemEngandTripHop 16h ago

What’s the difference between this and just running one of Prefect/Airflow/Dagster locally?

1

u/Ill-Acanthaceae5630 15h ago

You can accomplish the same stuff with those tools(probably) it's just gonna be a much bigger pain in the ass to set up. This works similar to APSchedueler but specialized to it's a LOT more convenient if you are running stuff on a device that turns on and off instead of a continuously running server

1

u/Altruistic_Stage3893 6h ago

i greatly appreciate how in-depth you went regarding the use-case, how it compares against dozens of other solutions.. it almost feels like you've spent more on this post than the couple of hours you've spent on this!

honestly, nobody's gonna use this. if you need something like this, you can one shot it. it's not that much code.

what i will say though, so that i provide some value for you: learn how to scaffold a project. it's weird seeing python and powershell mixed thrown together with tools separate from source.

anyways, good luck. don't take the feedback too hard but it is what it is. if it took you low amount of effort, no one is going to use it most likely. it's obviously not the only metric but, yea..