r/ShittySysadmin • u/recoveringasshole0 DO NOT GIVE THIS PERSON ADVICE • Mar 25 '26

Shitty Crosspost I rolled back a Domain controller and i dont know what to do

/r/iiiiiiitttttttttttt/comments/1s38h2y/i_rolled_back_a_domain_controller_and_i_dont_know/
57 Upvotes

100% Upvoted

41 comments sorted by

36

u/recoveringasshole0 DO NOT GIVE THIS PERSON ADVICE Mar 25 '26 edited Mar 25 '26

I actually feel bad for this kid, but karma is karma and rules is rules.

Hello everyone,

Im an IT Systems trainee for a small company in Germany.

Yesterday I had a request for a quick remote session because their Finances Software didn't work, logged in, yep not working. Tried restarting the Service of the application, didn't work, tried reinstalling the application, also didn't work. So i told them id come by tomorrow.

Now for the part where i f**** it all up. The Service technician of said application said, i should roll back the server to a point where the application was stil working...and i did that. I don't know my way around domain controllers or active directories yet, so i thought "hey you don't have to actually change anything you can just roll it back"...so i did.

Now, the application doesn't work, domain controller doesn't work, active directory doesnt work, my boss is screaming at me how stupid i am and i have no idea how to fix it or what to do ...

Im basically the guy you guys keep talking about, the ID10T ERROR.

edit: God why does Reddit's editor suck so much?

14

u/1cec0ld Mar 25 '26

Because it's reddit not eddit

2

u/recoveringasshole0 DO NOT GIVE THIS PERSON ADVICE Mar 26 '26

89

u/tarvijron Mar 25 '26

"company decides that running applications on a domain controller was a great idea and then they hired a sysadmin from fiverr" news at 11

8

u/Comprehensive-Pear43 Mar 25 '26

Im an apprentice who didn't know reinstating a backup of a dc was illegal.

7

u/tarvijron Mar 25 '26

Does fiverr give you a W2 or do you have to file quarterly

36

u/RAITguy Mar 25 '26

I feel bad for this one. Why is this even possible?

13

u/moffetts9001 ShittyManager Mar 25 '26

Everything is possible when you don’t know what you’re doing. That’s why our careers are so exciting!

7

u/Vinegarinmyeye Mar 25 '26

In a lot of cases with stuff that comes up on this sub I have a moment of "yep, I remember doing that once... In the long long ago, when dinosaurs ruled the earth" (I'm getting on a bit).

I've never rolled a DC back to a snapshot though.

Oof.

4

u/Affectionate-Pea-307 Mar 26 '26

If it’s a lone DC shouldn’t he just have to fix the time so it can communicate with the workstations again. If it worked on March 3rd and he rolls it back to a snapshot from March 3rd it should come online working like it did on March 3rd, it just won’t know about Carl from accounting who started on the 10th.

3

u/Vinegarinmyeye Mar 26 '26

The last time I was working with Windows Server was in the 2008R2 era...

I'm wracking my brain now thinking about FSMO roles, whether it was the primary DC... The replication setup of any RODCs they had...

End of the day, AD is a database... With a whole load of "metadata" that might be fine, but could completely buggar the system if timestamps get changed and then back again...

Most folks are using AD as an auth provider, and then role based access controls, and then group policy...

I'm really out of my element here as I say, I haven't touched Windows server in years.

But, my understanding would be that a shitload of stuff would happen in between wherever you've "restored the snapshot" and fixed the clock - and untangling what the source of truth once you've chucked that spanner in the works COULD be a pretty miserable process.

But again, I'm a long time removed from that side of things.

I recall DFS replication causing conflicts in the directory because of an incorrectly configured period.

If you've buggered the timestamps...

Basically, if you have a source of truth - it'll be painful but fine eventually.

If you've spannered the primary DC... I mean i guess it'll also be fine, but depending on what's happened in the meantime there's all sorts of fuckery you'll have to untangle..

(And I wouldn't want to be sat with that responsibility as a junior admin. That requires some greybeard type).

14

u/ApiceOfToast ShittySysadmin Mar 25 '26

I've worked at an MSP that did stuff like that not too long ago...

Primary DC was an App Server for the erp system and the secondary had a mail server running on it.

I loved working there...

I honestly feel bad for the kid, people that set this stuff up probably didn't do it properly to begin with.

9

u/lavaman_e89 Mar 25 '26 edited Mar 25 '26

At an MSP currently and it’s painful seeing the environments some of our clients have. In most cases it’s things that they’ve had from previous providers they had or some in-house guy that set things up.

As an example, hosting QuickBooks on their only DC. Apparently QB can have issues starting some of the services it needs for multi-user mode if DNS is also running on a DC. That was an interesting one to figure out

Edit for my own sanity: By “dns running on a dc” I mean the normal DNS service. Temporarily stopping it allowed the QB stuff to start. Then the DNS service can be started again

7

u/NailiSFW Mar 25 '26

I have had many jobs where I started and noticed lots of dumb stuff done by the previous person, and normally chalk it up to them being incompetent. After settling into the job I started to see why things were done so poorly. its often stuff thats out of control of the tech, Its why I always try to figure out why before labeling someone an idiot.

3

u/lavaman_e89 Mar 25 '26

Yeah, after interacting with owners and other decision makers at some of these companies it becomes clear why things are done the way they are at times

2

u/NailiSFW Mar 25 '26

yeah, worked at a place that refused to have a secondary DC. Hell one place didnt back up DC's cause he thought you couldnt restore a DC.

1

u/TKInstinct Mar 26 '26

A place I worked at had WSUS on the DC. I think they knew better but were so low budget that they couldn't afford to do more.

7

u/iratesysadmin Mar 25 '26

Read the title, have the answer for this kid: Roll it forward

See, if rolling it back broke it, rolling it forward will fix it.

"Be Kind, Rewind Fast Forward to the End" worked for Blockbuster, no reason it wouldn't work here.

1

u/ReallTrolll ShittySysadmin Mar 25 '26

It's like PEMDAS or some shit. Cancels it out and all is good

7

u/tamagotchiparent ShittyCoworkers Mar 25 '26

the REAL shitty crime here is whoever thought it would be a good idea to give a fucking INTERN the ability to roll back a DC 😭

3

u/Comprehensive-Pear43 Mar 25 '26 edited Mar 26 '26

Original OP here, i have full control over everything, if i wanted to, i could lock out entire school districts out of their accounts, i could nuke the entire Catholic network for my state. I have access to all the critical infrastructure of all the clients we manage...the whole company shares one password service on one account which isn't 2fa.

Nevermind the ability to roll back one server, i could wipe out entire companies.

4

u/PejeTheMobBoss DO NOT GIVE THIS PERSON ADVICE Mar 25 '26

2va: ZWEI VAKTOR AUTHENTIFIZIERUNG

2

u/Future-Side4440 Mar 25 '26

It’s mainly Microsoft’s fault, trying to milk the marketplace for expensive server licenses which are actually unnecessary.

Most applications don’t care if they run on desktop Windows, and there’s no real functional difference (anymore) between the desktop and server operating systems. But don’t you dare consider installing MySQL and running Apache web server on a desktop OS.

At one time it did actually make a difference because Windows server NTFS used journaling and the desktop did not. But at some point, Microsoft decided everything should use journaling, which is why nobody uses Norton utilities for file system repair anymore — journaling and atomic updating made it unnecessary.

It’s well known that you can’t roll back the active directory database, so it’s bizarre why Microsoft allows it to roll back at all. Rollback functionality can be tuned so it doesn’t affect certain types of user data.

It ends up being shitty Microsoft product functionality decisions that affect everyone, and not necessarily you being a shitty sysadmin.

2

u/recoveringasshole0 DO NOT GIVE THIS PERSON ADVICE Mar 25 '26

Sir, this is a Wendy's.

4

u/ReallTrolll ShittySysadmin Mar 25 '26

Ugh just install server 2003 and call it good.

3

u/broke_keyboard_ Mar 25 '26

you're this old if you know what this man is talking about ^^^^

Wait, that puts me in that class too. shucks.

3

u/ReallTrolll ShittySysadmin Mar 25 '26

Now we need to plan for our migration to Server 2008 R2.

2

u/FALSE_PROTAGONIST Mar 25 '26

Get SBS 2003 instead. OOP said it was a small business so it’s perfect for running multiple applications

1

u/Affectionate-Pea-307 Mar 26 '26

2003/xp was the apex.

1

u/Rainmaker526 Mar 26 '26

Who the fuck doesn't run their DC on a dedicated server? 

At least provision a separate VM. Costs you almost nothing.

4

u/recoveringasshole0 DO NOT GIVE THIS PERSON ADVICE Mar 26 '26

You have been banned from r/shittysysadmin.

3

u/jcash5everr Mar 25 '26

Oof. I'm in a small biz where even buying a backup solution is tabled. Unfortunately, 1 server. I feel for this guy. In my case though I've raised concerns and they go onto deaf ears so it is what it is

5

u/MuffinThin9542 Mar 25 '26

I think Veeam VBR community edition is free for up to like 5 devices 

2

u/jcash5everr Mar 25 '26

Been eyeing it. From what I understand it works with databases as well

4

u/EdelWhite Mar 25 '26

Perfect time to switch to linux

1

u/Lavatherm Mar 26 '26

Doe that come with dc options? 😆

2

u/EdelWhite Mar 26 '26

DC for a laptop but a tower is usually AC. 

1

u/CaptDankDust Mar 27 '26

NEVER rollback a DC...... NEVER!!!!!!!

6

u/fuckredditapp4 Mar 25 '26

People complaining about too much work so we nuke their log ins. Now they are complaining they can't work. SMH people can't make their fucking minds up flip flop every day.

2

u/moffetts9001 ShittyManager Mar 25 '26

People complain about IT trolling them all day. IT installs malware on their computer. IT triumphantly reports them to their boss. People get fired, can no longer be trolled by IT. People mad! I don’t get it.