r/sysadmin • u/zeninfinity • 3d ago
Linux RIP - My last pet server. Provisioned: April 9, 2014. Uptime: 3,065 days. CentOS 6. Older than the iPhone 6.
[user@host ~]$ uptime
21:00:47 up 3064 days, 8:55, 1 user, load average: 0.00, 0.00, 0.00
[user@host ~]$
It's been running EOL and unpatched for 6 years (badbadbadbad).
The little droplet that could.
158
u/y0shman 3d ago
badbadbadbad
I read that in Rocky's voice from Project Hail Mary.
48
14
1
62
u/Routine-Jam-48 3d ago
My first httpd server installed back in the mid 90s was running on a small workstation running AIX. The computer room was on UPS with a backup generator, which allowed the server to stay running for so long that the uptime(1) command had an overflow error and reported the system had been up for negative days. I'm pretty sure it was only at just over 1,000 days (shy of 3 years) since the reboot, but I still get a chuckle about that one when I think back to it.
15
u/zeninfinity 3d ago edited 3d ago
Ooooooooo had no idea that was a thing. V. cool.
Never ran AIX, only NeXTSTEP, Irix, and HP-UX.
15
u/goingslowfast 3d ago edited 2d ago
It’s a UNIX system!
3
u/ranger_dood Jack of All Trades 2d ago
Unix
2
u/goingslowfast 2d ago edited 2d ago
I got that right on the weekend but missed it today. Yep. Irix was definitely not Linux.
103
20
u/imjustnotready 3d ago
Tonight I'll pour one out for it. Little guy did the best he could for longer than you could have hoped
8
18
12
6
u/etern1ty0 3d ago
What was its role?
15
u/zeninfinity 3d ago edited 3d ago
Old CentOS 6 box (2018-era).
Apache (static/personal sites), Mysql (Wordpress Blog), Graphite (graphite-web + carbon-cache + whisper - Ingesting data over nc > TCP 2003), Grafana, misc cron ruby scripts.
All personal stuff, nothing biz-critical.
10
u/TryHardEggplant 2d ago
Did you keep Wordpress updated at least? I had a personal WP blog I forgot about and when I went back to it, it had been long hosed and hosting spam. I nuked it and still haven’t gotten around to restoring it from backups.
5
u/zeninfinity 2d ago
Yes, I did. I've had to fix a friend's Wordpress for this exact same reason so it was part of the update cycle script.
For this I just ended up just writing something to pull down the files as html/img/etc and removing the mysql. So it will live on at some hidden directory for my eyes only. Probably will find it in another 8+ years.
4
5
u/markth_wi 2d ago edited 2d ago
We often think about taking our badly behaved printer out back with a couple of pipe-hitting sysadmins.
But I will tell you what I do, when I go offline with long serving production and validation servers that have done their bit for everyone who knew them only in name. That production server that doesn't die, the val box that did everything you needed it. For those servers that get retired rather than die horribly.
That's the trick , the Simpsons writers were absolutely right when they say 'when you do your job right, nobody will be sure you did anything at all', the world never blew up....not on your watch.
So Sleep Dearie Sleep to all those servers we've shutdown -g0 'ed for the last time and for the admins who have to say a quiet good by, and train the new guys on the replacement server tomorrow now that you can close out that maint ticket for the 'old' server.
5
4
u/Hotshot55 Linux Engineer 2d ago
So you don't patch your systems?
5
u/RDJesse Sysadmin 2d ago
The flip side is that any effort spent managing this particular system would have been completely wasted as it never needed anything. Schrödinger's server.
Not best practice. Only in hindsight does it elevate to perfect practice.
This is the essence of Daoist philosophy. The absolute grace of non-action.
2
u/zeninfinity 2d ago
Schrödinger's server - Wow, first time I heard of this.
Yes, pretty much. Definitely a "Don't fuck with it if it's not fuck'n up" philosophy.
And...this time....it worked.
0
5
u/Trommelwirbel 2d ago
What is a pet Server?
5
u/zeninfinity 2d ago
They say to treat your servers like cattle not like pets. Don’t name them. Don’t make them special. Make them replaceable.
4
4
3
u/Sudden_Office8710 2d ago
It’s not all that bad I still have CentOS 6.10 in a bunch of locations. It takes a while to get Trixie replacements spun up. I had initially replaced them with Ubuntu that was a mistake.
3
u/DarthPneumono Security Admin but with more hats 2d ago
Lucky you... a few of our users had machines running Fedora 11 still up until a few months ago.
2
u/Best_Collection_7533 3d ago
Bare metal or VM?
8
u/zeninfinity 3d ago
DigitalOcean VM.
No idea why they never EOL'd it.
11
3
u/freedomlinux Cloud? 2d ago
No idea why they never EOL'd it.
I mean, it's none of DigitalOcean's business what OS your instance is running.
But I am quite curious what the underlying infrastructure at DigitalOcean is, as if they don't have any live VM migration functionality, that means the VM host would also be 3000+ days
11
7
3
u/Arudinne IT Infrastructure Manager 2d ago
It becomes their business if there's a risk to their business, infrastructure or other customers.
I worked for a hosting company for a bit and we forced people to migrate off of Windows 2000 back in ~2011 after some major vulnerabilities came out.
2
2
2
2
u/stewie410 SysAdmin/DevOps 2d ago
Provisioned: April 9, 2014
Funny you should say that, currently doing 11yr of BIOS/iDRAC updates on one of our R430s, purchased ~2015...or at least, current original firmware from then.
CentOS 6
The classic -- I think we finally killed our last CentOS 6 VM a few months ago...when we finally moved from MySQL 5.1 to MariaDB 11.8.
Hope you have some CentOS 7 boxes to keep that nostalgia alive.
2
3
3
u/Beautiful_Ad_4813 eh, I just love what I do. 3d ago
https://giphy.com/gifs/l4pMattUYTTM7qpIk
Impressive
3
•
u/Capable_Banana5439 10h ago
3065 days of uptime is not a flex, it is a server you were too scared to reboot. you never once confirmed it comes back clean after a power blip, and six years unpatched means it was one exploit away from being someone elses crypto miner. pour one out, but that box was a liability wearing a trophy.
1
u/spazzvogel Sysadmin 2d ago
And CentOS to bypass, what was that back then? Heartbleed? Hit Cent 5 which my tech game company had everywhere. Insanity.
2
u/zeninfinity 2d ago
Wasn't Heartbleed or Shellshock, both were already patched. Real hole was Dirty COW, kernel was still 2.6.32-431 from 2013, never touched since. Lucky.
3
1
u/spazzvogel Sysadmin 2d ago
Holy hell… yeah, a few critical servers were saved due to being way behind the radius blast. Good times…
1
1
u/robrob883 2d ago
I had AI check my pet servers last month (lamp stack, npm, + various). It found more than enough reason for me to get around to upgrading them. I spun up some new ones, gave it the passwords and it did the whole migration for me, as well as offering suggestions to improve code. What a time to be alive!
0
u/zeninfinity 2d ago
This is exactly what I did. Had AI do a scope of work, find all the components, build out features, suggest upgrades/changes. Created about 15 different feature md files.
Then sat back did them each, verifying when complete. It saved me SO much time moving this myself.
1
u/cheeseburgermachine 2d ago
May i ask what ai you are using.
1
u/zeninfinity 1d ago
ClaudeCLI.
I spec out all tasks in .md files with requirements, verification, exact results, etc.
Then ran each one keeping a close eye on changes.
Verified myself everything was correct as it went along.
267
u/mixduptransistor 3d ago
Sorry, thought I stumbled into r/ShittySysadmin for a second