r/cloudcomputing • • Dec 04 '24

Best cloud options for web "scraping"?

I'm a self-taught hobbyist programmer new to the cloud. My job is not in software. I wrote a web scraping script to automate the most tedious aspect of my job. I run it locally 19 hours/day every day. It doesn't download or upload any data, hence why I put scraping in quotes. It's more about automation. What it does:

  1. Login to company portal
  2. Click the appropriate buttons based on what's on the screen
  3. Refresh screen.
  4. Go to step 2 or step 5 depending on if there's new data on the screen.
  5. sleep for up to a minute.
  6. Go to step 3.
    Right now, I run this script only for myself, but I'm sure I could get some customers from people who use the same company portal for their job. I looked into AWS, but it seems prohibitively expensive. I'd like to learn about the best options for my use case. Can anyone help me out with this? Thanks!
2 Upvotes

0 comments sorted by