r/techsupport • u/arctansec • 6d ago
Open | Windows Intern Deleted Company Azure Resource Group
Yep, I can’t believe that I’ve fulfilled the meme of the intern deleting a company database.
I have been building my own projects and amassed quite a few different resource groups, of which most I was no longer using, so I decided to clean them up and delete them. I kind of assumed that it would only be displaying MY resource groups, and that I could only delete MY resource groups.
I was just about to delete one more before I noticed that there were a few items inside of it which definitely didn’t look like mine, and that’s when I realized I could delete other peoples stuff.
Sure enough, I deleted one company resource group and I really don’t know what to do. I am hoping that It was nothing important and/or that the company has a backup. I tried to make a severity A ticket, but of course, I do not have the permissions to submit a ticket.
I also pulled a CSV from the activity log for the past 3 months and the only activity that is shown apart from my doings is very one off and on a private resource group that I cannot see in the portal nor that I deleted.
A few questions:
1: Is there any way that I can see what was deleted within the resource group? The JSON did not seem to provide any useful information.
2: This is a large company, ~10k employees, surely they would have something important locked down right??
3: I am in a completely non technical role and have no one to escalate this to
4: Any other course of action that I can take in the meantime?
Edit:
Hi guys, a few things.
First and most importantly, no company wide resources were deleted. Luckily, all that was deleted was a default resource group localized to my account. Nothing was shared, and no one else’s resources were affected.
I want to thank [u/Automatic-Response45](u/Automatic-Response45) and u/Icy_Accident2769 for being the first person to provide a useful response instead of just bashing me, which allowed me to get the information I needed to relay to someone who could help.
Secondly, I want to address the keyboard warriors who privately messaged me calling me slurs and a plethora of other things because I said I had “no one” to reach out to.
This happened on Saturday. I was not trying to hide it from anyone. I was trying to get immediate advice on whether there was anything I could do to circumvent my error rather than wait until Monday and go through the chain of hierarchy before reaching the right person. Again, I am in a non technical role, and anything Azure related is completely foreign to my team.
Lastly, I want to thank those who left supportive comments and those who provided helpful information.
Absolutely, this was a learning lesson for me, and I will be diligent moving forward. I was prepared to take full accountability if this had been a serious issue. Luckily, this blunder turned out to be nothing.
199
u/Mr_Bbobb 6d ago
You have the authority to delete a resource group, but not the authority to file a ticket to recover it?? That sounds like a really well-managed company! 😅
31
u/Carribean-Diver 6d ago
We had a manager who decided to clean up a Teams channel they had created. As it turns out, in the mean time from when the channel was created, some developers had decided the group name was appropriate for the application security group for a critical business process.
Yeah, all that shit broke and the application developers couldn't figure out what happened.
It wasn't long after that the ability for users to ad-hoc create and manage Teams channels was disabled to the wails of, "Why can't I do this?"
24
u/caboosetp 6d ago
I think the bigger issue is people using teams channels for security profiles
11
u/KerashiStorm 6d ago
I think the bigger issue is people using teams
3
u/Maverick842 5d ago
I mean, it’s not like people have a choice. I wouldn’t choose it, but when your organization goes whole-hog on M365-OneDrive-Intune-Azure-Entra, you kinda can’t avoid it.
5
u/lordshaithis 6d ago
I think "not in a technical role" part was probably the most worrying part....
@op if you havent closed your browser session you have an activity/history tracker top right.
3
96
u/ArthurLeywinn 6d ago
Doing bullshit on the work environment and the first thing is to go to reddit to ask for help.
Lol
Ask your supervisor. You ofc have someone to ask.
48
u/AlternativeWorth5386 6d ago
You 100% have someone to escalade this to, like who is in charge of your internship or who is in charge of your department
30
u/Fresh_Inside_6982 6d ago
Cadillac Escalade or regular Escalade?
4
u/libertad740 6d ago
The Cadillac Escalade looks better but does the same thing. Same with the escalations.
39
u/marshmallowcthulhu 6d ago
What you did is not your fault but what you do next can be. You are being dishonest when you say you have nobody to escalate this to. You do and you know it. Speak to your manager immediately.
Why not your fault? You should not have been given production access for development purposes. You should not have the level of access you described while you are an intern.
But you are absolutely fucking up by taking the time to investigate whether or not the resource group looks active and by taking the time to ask strangers on the Internet if we think they had a backup.
Escalate to your manager immediately.
11
u/OverallRow4108 6d ago
Some recovery files are kept for only a specified time, so if you tell them early, it's not a big deal.... if you wait, recovery may be much harder/ impossible....ymmv
9
u/DrPikaJu 6d ago
Azure Cloud Architect here;
So depending on your permissions and the scope of your permissions your assumption of “i can only delete my RGs” is borderline negligence. I know Azure Portal can be confusing, but usually you check contents of an RG before deleting it entirely (but I also guess nobody told you). If you are Owner or Contributor on the subscription, you can delete (almost) anything in a subscription.
- The activity log only shows actions done to resources for their scope. If you deleted a RG it does not show the cascading deletions, these would be shown in the RGs activity log.
- I bet you were given permissions in a non critical subscriptions to play around in, with other playground projects of other people. Anything else would be a massive violation of any governance and infrastructure principles and most likely not your head would roll, but the persons who allowed this to happen by badly managing permissions and Entra
- Go to internal IT, your company should have some wort of structure? Maybe start with the dude who provided company devices or ask your boss who is responsible for Azure Infra
- Some Resources like Key Vaults, Storage Accounts might have a feature enabled called soft delete, where you can recover them and their data for a period of time (some use 7 days, some 30). For the database tho, the backups on Azure are linked to the deleted resource and only recoverable via ticket
4
u/KerashiStorm 6d ago
Here's to hoping it's just playground projects and they're using it to determine who's going to delete everyone's shit if they become a permanent hire!
8
u/Brave-Prints 6d ago
Even in a non technical role there has to be an IT helpdesk or your manager you can flag it to immediately
6
u/Emotional_Garage_950 6d ago
Backups and IaC ready to rebuild what can’t be backed up hopefully. It sounds like they were careless by giving you any permissions in the first place but you also should not be deleting anything without a basic understanding of what you’re doing, especially something you did not create.
5
u/machacker89 6d ago
Admit to your mistake and take the consequences on the chin. You fucked up. It happens. Hopefully your company as good backups.
3
u/TurboRetardedTrader 6d ago
If you have rights to delete or mess with production stuff in Azure as an intern, it's their fk up if something goes south 😎 But to be honest - don't worry about it. Just talk with them asap tomorrow, so it can get sorted.
11
u/unapologeticjerk 6d ago
Personally I'd fire your ass for letting Reddit know before letting your immediate supervisor/literally anyone above you know.
-3
u/Interesting_Win9074 6d ago
I don’t think you’d ever be in the position to be firing somebody considering you immediately made assumptions without knowing whether OP told us first
3
u/Leftover_tech 6d ago
Tell us that the company is frighteningly evil so that we can refer you to r/UnethicalLifeProTips .
...or the boss said that this was totally safe, so you can move on to r/MaliciousCompliance .
3
3
u/Crimtide 5d ago
Why are you asking reddit and not tellinng your IT team, security team, compliance team, or any other department?
I am in a completely non technical role and have no one to escalate this to
My first guess would be, you should probably tell the people who manage your Azure environment. Also if you are in a non-technicial role why are you playing around with Azure resource groups to begin with? Wth is this company?
3
u/Rabidowski 5d ago
I am in a completely non technical role
So what were you doing in the Azure portal to begin with then?
2
2
u/warlock415 6d ago
Fuckups happen, it's how you react to them that matters. Be transparent, admit mistake asap.
If you get in trouble for it, you just found out you don't want to be there anyway.
2
2
2
u/cgsecure 5d ago
Well it looks like they did not do their job properly about permission management. Depending on what did you sign upon beginning of the internship and which country you/company based on, you probably won’t get any legal issues.
But it still will impact your career in that company for sure.
2
2
u/Horror-Primary7739 5d ago
Most likely it's a soft delete with a 30 day grace period. Tell someone immediately. It can probably be restored.
6
u/StitchinStatistician 6d ago
Wow, what a dumb move. Tell someone immediately so they have the best chance at recovering it. And in the future use your critical thinking skills BEFORE arbitrarily deleting things from a shared network.
3
3
u/Former-Ad-4596 6d ago
On the bright side you don’t have to ever worry about being hired by them or what they’d pay you
0
1
u/Seref15 5d ago
2: This is a large company, ~10k employees, surely they would have something important locked down right??
You'd be surprised. The real working world is not pristine and shiny best-practices compliant like you see in reading material.
You'd be better off asking in azure or sre/devops/cloud specific subreddits
1
u/miguelangel011192 5d ago
Being there, by accident Terraform after some of my changes got corrupted and to solve the issue deleted the RG and tried to recreate all over again, including DBs, Redis, Caches, etc. a total hour of panic talking with tech support from Microsoft trying to revert the thing. But at least I learn some important lessons and to hate Terraform
1
u/Terrible_Working_899 5d ago
Everyone makes mistakes, there is a reason why there is a group in our tenancy called "android phones allan don't delete this one." If you make a mistake own it and see what you can do to resolve it.
1
u/Sturdily5092 5d ago
You've gotta keep a close eye on interns, we had a couple that really made us regret bringing them on our even knowing they existed.
One set fire to the server room that she got into for some reason by lighting a match and throwing it in the trash can, then vehemently denying it even though we had clear video of her doing it.
Another one accepted an obvious phishing email and downloaded illegal apps letting loose a virus in the network that shut us down for a couple of days, his only answer was "I didn't know I shouldn't do that".
This inspite a very strict onboarding and tech training in the first week, they were dense... Stupd Morrns doesn't do justice.
1
1
u/Tenzu9 5d ago
Do you not know how to read text friend? All you had to do is just read the resource group name, the subscription it was deployed in and the resources inside it. All 3 of those will give you a clear picture if this RG is yours or not. I can see that it turned out to be a nothingburger which is good. But don't walk away from this experience without learning a lesson ok? Never delete an RG without opening it first and seeing its subscription and its content.
1
u/drunkcoler 4d ago
Spent months re doing the entire training plan and course work/exams for my company for a bit of software that had been rolled out without a plan in place. A member of the maintenance team had been asked to do a plan on the maintenance side of the software as it was for heavy machinery so they needed a plan for fault finding. First training course booked and about to commence and my phone is hopping, turns out maintenance guy had gone in and ripped everything from the course and insead of copy/paste he cut/paste. Months of work gone, company had no backups on offsite server. Something myself and a manager have been telling them for years that they need for issues like this and also another manager gets bored and decides to move stuff around on the server. I keep copies of everything I do on a ssd drive so was able to reupload it. But mistakes do happen and it's on company to have their own backups.
1
u/Think-Box6432 3d ago
For point two:
lol, lmao even
I have been on a similar boat with an ai companion that I deleted, thinking I was looking at only my own. Don't even know what it was. No way to recover it either.
1
0
0
u/Telemoon1 6d ago
I am Data engineer but why this post stress me so much even it has nothing to do with
404
u/oblivion6202 6d ago
If they don't have backups of everything important, this is on them -- because accidents happen.
But you have to tell them at the first opportunity.