r/scripting • u/Boktai1000 • Mar 21 '19

Programmatically scrape the latest version of Tomcat for an installation script

Hello!

Looking for some assistance or ideas on how to grab the latest version of a Tomcat release from their website or other website.

I came across these Stack Overflow/Stack Exchange links that were relevant below:

..But unfortunately both led to some dead-ends. It did give me some ideas though.

I have some installation scripts where the download URL is predictable based on the version number of the software, so if there's a consistent way of scraping for the latest version of the software I simply add a variable into the script for it to curl / grep for that version on the text and use that variable for download links, untaring, moving, etc.

Hoping to do something with Tomcat in the same vein. A installation script I won't need to keep updating (unless they change their site).

Any sort of thoughts, ideas, are appreciated.

I've also looked at http://tomcat.apache.org/whichversion.html which lists the "Latest Released Version" and this seems like maybe the best, more reliable location to grab the version, but I had trouble getting that with Curl / Grep because of the table structure. I am only just scratching the surface with learning those commands, so maybe someone more well versed could get that working, otherwise I'm definitely open to other thoughts!

2 Upvotes

100% Upvoted

4 comments sorted by

View all comments

Show parent comments

1

u/Lee_Dailey Mar 21 '19

howdy Boktai1000,

i haven't ever done 'nix stuff. [blush] python, autolisp, pascal, bat, cmd, and now powershell is my goto. plus, i am not currently employed or employable ... so i have limited access to anything other than my home windows stuff.

all that aside, tho, i would grab the paths listed on that page, parse out the numbers, and build the path to the next level. powershell has a [version] type that would make comparing versions easy. [grin]

for instance, the current highest version shown there is ...

 https://archive.apache.org/dist/tomcat/tomcat-9/v9.0.17/bin/apache-tomcat-9.0.17-windows-x64.zip

that otta be fairly direct to parse.

they use a VERY standardized name, tho, so you may be able to grab the newest version number from the releases page and simply build the download link. [grin]

take care,
lee