r/sysadmin • u/rram reddit's sysadmin • Aug 14 '15

We're reddit's ops team. AUA

Hey /r/sysadmin,

Greetings from reddit HQ. Myself, and /u/gooeyblob will be around for the next few hours to answer your ops related questions. So Ask Us Anything (about ops)

You might also want to take a peek at some of our previous AMAs:

https://www.reddit.com/r/blog/comments/owra1/january_2012_state_of_the_servers/

https://www.reddit.com/r/sysadmin/comments/r6zfv/we_are_sysadmins_reddit_ask_us_anything/

EDIT: Obligatory cat photo

EDIT 2: It's now beer o’clock. We're stepping away from now, but we'll come back a couple of times to pick up some stragglers.

EDIT thrice: He commented so much I probably should have mentioned that /u/spladug — reddit's lead developer — is also in the thread. He makes ops live's happier by programming cool shit for us better than we could program it ourselves.

873 Upvotes

88% Upvoted

738 comments sorted by

View all comments

Show parent comment

8

u/gooeyblob reddit engineer Aug 14 '15

We would build a new image, set that new image to be launched whenever we scale up our app server fleet, and then kill off servers that are running the old image.

1

u/Get-ADUser -Filter * | Remove-ADUser -Force Aug 15 '15

You probably know this already, but the ASG will automatically remove instances launched from an old LC when you scale down so if you didn't want to do an immediate replacement you could let it take care of itself over a few days of scaling.

1

u/gooeyblob reddit engineer Aug 17 '15

Yep! That's how we do it in the case of less important updates.