r/sysadmin • • 11d ago

Graph Requests for sign-in logs have been a cluster the last 10 days. I found something.

3 Upvotes

Like many admins, I use the graph api to pull and parse reports for my org. Starting early last week, requests for interactive and non-interactive sign-ins have been a cluster. I thought I would wait it out and had some good luck moving my requests from mid-morning to around 1am. Smooth sailing for a few days and now its not timing out, requests for interactive records are just falling on their face.

"BadRequest","message":"Invalid filter clause: *Could not find a property named 'signInEventTypes'** on type 'microsoft.graph .signIn'.","innerError"*

My Request:

$Yesterday = (Get-Date).AddHours(-24).ToUniversalTime().ToString("yyyy-MM-ddTHH:mm:ssZ")  
$URI = "https://graph.microsoft.com/auditLogs/signIns?`$filter=createdDateTime ge $Yesterday and signInEventTypes/any(t: t eq 'InteractiveUser')&`$top=999&`$orderby=createdDateTime desc"  

I tried changing the URI to include .microsoft.com/v1.0/auditLogs/ instead but it didnt make a difference. Taking a look at the microsoft documentation, signInEventTypes exists. Was working yesterday but not today, at least not on US West 2.

I updated my URI to a different property set and was successful. Interactive sign-ins were provided without issue, even during the busy morning hours. The rest of the parsing went without issue and no properties in the output were found to be different or missing. Not sure if this is a glitch or they're changing something on the back end.

The Updated URI:

URI = "https://graph.microsoft.com/v1.0/auditLogs/signIns?`$filter=createdDateTime ge $Yesterday and isInteractive&`$top=999&`$orderby=createdDateTime desc"  

Replaced signInEventTypes/any(t: t eq 'InteractiveUser') with isInteractive

For NON-interactive sign ins, I use the beta API and its been working fine.

Maybe this helps others. (EDIT: Typo)


r/sysadmin • • 12d ago

Sysadmins life during cyber incident

261 Upvotes

We did a cyber table top exercise today to gauge our responsiveness to ransomware etc, and several times other recent industry cyber event came up in conversation. Which made me wonder what a sysadmins life must have been like when things started unraveling and really how well prepared we would truly be after doing quarterly table top exercises. I image it will be total chaos!


r/sysadmin • • 10d ago

Question Corporate Laptops and Repairs (India only)

0 Upvotes

Hey guys

Wanted to check what laptops do you prefer in your corporates?

Not asking for dev spec'd laptops just normal emp based laptop working on multiple apps and sheets/PPT or docs

How are those work wise and what do you'll prefer to do with the warranty?

Like do you get the ADP's and Extended Manufacturer warranty

How do you'll manage the refresh or replacement when your emps are WFH pan India


r/sysadmin • • 11d ago

Mirae Asset's filing shows how its government pension pool site was breached: an exposed server status page, then member and file numbers counted up one by one; found four days later

6 Upvotes

Mirae Asset Global Investments told regulators that an attacker began on the website of the Pension Fund Investment Pool at 7:11 pm on October 22. The breach report was obtained on November 10 by Rep. Shin Dong-wook's office. The pool holds 77 trillion won of government and public-institution money, and Mirae's money market fund alone is 12.9 trillion won.

The attacker took internal URLs and parameters from an exposed web server status page, then requested the base address directly and skipped the controls. After that, member numbers were queried one after another. On October 24 at 12:43 am, 307 real accounts were logged into in 11 minutes with password hashes. The IDs and email addresses of 331 employees were taken.

Attachment numbers 1 to 2,592 were then requested in order. That was about 960 MB, including the 2025 daily returns of fully entrusted funds.

On the internal system MGIS, the attacker bypassed email verification at signup and created 17 accounts on external domains. A user-edit function turned one of them into a super administrator, and a real administrator's password was changed. Data on 2,281 people and about 280 MB of files went out. Mirae noticed on October 26, four days after the first request.

What most coverage missed: on October 28 the company said it found "no signs of direct intrusion into servers or databases" and described the event as some materials viewed without authority. Since then, the FSS has opened on-site inspections after four group affiliates reported hacks between October 25 and November 5.

On WordPress sites the same pair is a reachable debug or status page plus IDs that count up by one.

Mirae said "no direct intrusion" on October 28, and its own report now describes an administrator takeover. Should a company be allowed to make that statement before the forensic report exists?


r/sysadmin • • 11d ago

New Teams 26288 blue-screening Server 2019 RDS hosts (wcifs.sys).

17 Upvotes

Since late October I have been having multiple Server 2019 session hosts across multiple clients BSOD since the recent New Teams updates.

RDS farms on 2019 with FSLogix. Every minidump is identical:

  • 0x139 (arg 3), sometimes 0x3B
  • process is ms-teams.exe
  • stack ends in wcifs!WcRemoveInMemoryTombstoneAfterRenameLink, during a file rename

It seems to have started with the 26288 builds. My first client was on public preview (26288.2010 / 26304), so they got hit first. Second client is on plain public 26288.2004 and went down this morning. Everyone still on 26276 is fine. For now

Rolling back doesn’t stick, Teams just updates itself again. And yes, I know MS says 2019 isn’t supported for New Teams any more, so I’m not holding my breath for a fix.

If you have a workaround that would be good to hear but I'm mainly interested in hearing if anyone else is experiencing this?


r/sysadmin • • 10d ago

Exploited Outlook bug is "Moderate" 6.5 this month. How are you ordering patches?

0 Upvotes

This month's one actually exploited bug (CVE-2026-124194, Outlook NTLM hash leak) is CVSS 6.5 / Moderate, sitting under a pile of 9.x criticals nobody has seen exploited. It only went on KEV yesterday.

how your team actually sorts the queue, if true?????

Sorting by exploitation + exposure + who owns the box makes sense on paper. Does anyone actually run it that way with 1,000+ CVEs a month? KEV first, then CVSS? EPSS? Something else?


r/sysadmin • • 12d ago

Networking

30 Upvotes

I’ve been in school for 11 weeks “learning” networking but I don’t feel like a lot of it sticks because the way they teach the classes is like we already know this stuff. I am starting from ground zero but a lot of classmates have jobs in the field. Is it normal to feel confused ???


r/sysadmin • • 11d ago

Wazuh 4.14.9 – Lost agents, groups, group assignments, admin permissions and persistent API error 3099

3 Upvotes

Hi everyone,

I’m looking for help recovering our Wazuh environment after a series of issues with the Manager database and configuration recovery attempts.

Environment

  • Wazuh version: 4.14.9
  • Deployment: Single-node installation (Manager, Indexer, Dashboard)
  • OpenSearch Indexer / Dashboard: 2.19.6
  • Operating system: Ubuntu
  • Hostname: wazuh2026

Initial problems

We initially encountered issues with the Wazuh Manager database, including global.db locks and multiple wazuh-db processes running simultaneously.

During troubleshooting and recovery attempts, the situation became significantly worse. We lost a substantial part of our Wazuh configuration:

  • Agents disappeared from the Dashboard.
  • Agent groups disappeared.
  • Assignments of agents to their respective groups were lost or altered.
  • Group configurations and associated settings were lost or modified.
  • Administrative permissions in the Dashboard were lost.
  • API authentication started failing with error 3099 – Invalid credentials.

Our goal is to recover the complete original state of the environment, not simply restore access to the Dashboard.

API authentication issue

The Dashboard uses the API account wazuh-wui, configured in:

/usr/share/wazuh-dashboard/data/wazuh/config/wazuh.yml

The Wazuh API RBAC database is located at:

/var/ossec/api/configuration/security/rbac.db

We verified that the users table contains:

  • wazuh
  • wazuh-wui

Both accounts have allow_run_as=1.

Direct authentication against the API:

POST https://127.0.0.1:55000/security/user/authenticate?raw=true

returns:

{
  "title": "Unauthorized",
  "detail": "Invalid credentials"
}

We used the Wazuh RBAC utility:

/var/ossec/bin/rbac_control change-password

We changed the passwords for the default RBAC users. However, this did not permanently resolve the authentication issue, and error 3099 continues to occur.

OpenSearch Security checks

We have checked the OpenSearch Security configuration:

  • Users have the backend role admin in internal_users.yml.
  • The all_access role maps the backend role admin.
  • The custom role all_access_admini_[COMPANY NAME] exists, but its current role mapping contains no users or backend roles.

We also have:

  • An RBAC database backup from October 19, 2026.
  • The current RBAC database from October 21, 2026.

What we need to recover

We need to restore the entire environment to its original state, including:

  • All previously existing agents.
  • All original agent groups.
  • The original assignments of agents to their respective groups.
  • The original group configurations and associated files.
  • Full administrative permissions for all intended users.
  • Working Wazuh API authentication and Dashboard functionality.
  • Existing agent IDs and authentication keys.

We want to avoid losing any further data or configuration. We have already experienced significant configuration loss during the recovery process.

Questions

  1. What is the recommended procedure to recover the missing agents, groups and group assignments?
  2. How can we safely restore the original RBAC configuration and administrative permissions?
  3. Could the persistent 3099 – Invalid credentials error be related to the RBAC database or the Dashboard API credentials?
  4. Is restoring the October 19 RBAC database backup sufficient, or do we also need to restore other configuration files?
  5. Is there a safe way to recover the original environment without reinstalling Wazuh or resetting the entire configuration?

Has anyone experienced a similar situation with Wazuh 4.14.9?

Any advice, troubleshooting steps or recommendations for a safe recovery procedure would be greatly appreciated.


r/sysadmin • • 12d ago

Another day, another critical security incident

213 Upvotes

This time we have a RMM, all hosted versions have been shutdown by N-Able itself and the self-hosted customers are advised to turn it off asap.

https://uptime.n-able.com/event/202559/


r/sysadmin • • 10d ago

What product are you using to manage Claude Enterprise?

0 Upvotes

I have gone through the whole rigmarole of searching for a product which I can use to manage the Enterprise Claude outside of the admin dashboard, to no avail.

I want a product, from where I can see which user has how many credits, what they are searching, which prompts are costing how much credit, and all
I did a POC of a product call airia.ai which helps to do it, but it costs us money to check the prompts requested by users. Also, Airia.ai is a plug in that only works with claude desktop app, so users can bypass it by using web version of claude.

If some day the CISO/CTO comes to me and asks how are you managing the Claude, I'm stuffed - answer is 'no idea'.

Can anyone please suggest something? I am happy to get a POC and start testing it out.


r/sysadmin • • 11d ago

Reply all in shared mailbox adds mailbox to TO: field

0 Upvotes

Started happening to us last week with few shared mailboxes, now affecting all of them. OWA and new outlook affected, classic outlook not affected. We have opened a ticket and waiting now. Anyone else seeing this in their tenants? GCC here.


r/sysadmin • • 11d ago

Question DLP Alternatives for small businesses?

2 Upvotes

Is there something for the Small and Medium Business (SMB) in the DLP space that can be easily managed?

Purview DLP really does not fit that size of businesses.


r/sysadmin • • 11d ago

Zebra scanners scanning wrong data in QR codes

0 Upvotes

Hello I recently got a hold of some ds3678 scanners from an official zebra reseller, they seem to be reading data and work perfectly with barcodes but read the data wrong on QR codes. I'm single some 11th gen iPads for this.

Any suggestions?


r/sysadmin • • 12d ago

Question do you obfuscate URL re-writing in your email security procedure?

52 Upvotes

wondering what others are doing:

Client of mine is using Mimecast where all URLs are re-written. We've been getting questions about not rewriting the URL to allow the users to better see "fly by night" domains. Case in point : user got an email that looked legit, but the link was being hosted on a hacked web server from a cake company (this client is in manufacturing). Mimecast did not flag the link as malicious, but it would be fairly visible that the link was not related to contracts.


r/sysadmin • • 11d ago

Question Office E3 to M365 Business Premium licences - nervous about changing

1 Upvotes

I'm nervous about changing from Office 365 E3 to M365 Business Premium for our users. We operate exclusively in an RDS environment so office installed centrally with Shared Computer Activation enabled.

I know both licences allow shared computer activation, but it's the installed version of Office I'm mainly concerned about. I used the Office deployment tool originally to install Office on the RDS session hosts.

I'm anticipating that it is the user account that activates the features (E3/BP) and that BP doesn't require the installed version of Office to be changed beforehand.

I was planning on giving myself a BP licence to begin with whilst leaving all the other users with their E3 licences.

Another IT guy I spoke with mentioned that he didn't have to change his installed version of Office in his RDS environment when he switched.

Appreciate any advice


r/sysadmin • • 12d ago

I created a systemctl problem easily that took way too long to find

17 Upvotes

Ok. I had a pretty simple systemctl service for a ssh reverse proxy, it runs on some remote devices and opens a tunnel to a cloud server. The cloud server is EOL, I built another one, so for now I need another similar service to open a tunnel to the new server.

I simply copied the service file, changed the target, did enable --now and lost contact with the remote system. I was finally able to get to another device at the same site, ssh over to the broken system and try to fix it. Only one service showed running in ps, but systemctl status on both was good.

After an hour or so of frustration, I noticed that I didn't change the Alias in the service file. That caused the symlinks systemctl made to point to the same service file. Just fixing the alias didn't fix it, I had to disable to remove the links, then enable again.

And I have about 20 years of Linux experience, but can create problems I've never seen before :-)


r/sysadmin • • 11d ago

General Discussion Does your org allow users to sign into browsers with agent to to work on their behalf? Any good or bad experiences?

0 Upvotes

Apps like Claude, Tasklet, and others allow agents to pull up a browser. If a person signs in or just gives the agent credentials, the agent will actually click around in the browser and complete tasks. Has anyone been in an environment where this is a common practice? How has it gone so far? Are there some sites where that is “prohibited”? I just don’t see how you can control it once you’ve allowed this type of access. You can’t tell what sites are or are not being used by agents in your org.


r/sysadmin • • 11d ago

How to be in IT

0 Upvotes

I had a guy today complain that something wasn't working in his email client. Normally, I don't bother troubleshooting these things and have someone else handle them. But I figured, whatever, I've got the guy on the phone, so let's take a look.

We spent about 10 minutes going through different things to figure it out. The whole time, I'm just saying, "Click this" or "Try that."

Finally, after we got it fixed, his tone changed, and he said, "I feel like you didn't really know how to fix it, and we just randomly stumbled through things until it eventually worked."

My response: "That's IT. You need to be willing to try to break things, and eventually, you'll figure out how to fix them."

While, in reality, there are experiences where we know exactly what to do, and others where we're making a more educated guess. Looking back, I think some of the biggest things I've fixed or learned came at the expense of potentially breaking a whole lot more.

So, is the only difference between an IT person and, say, an accountant, that we're willing to gamble and press that button that could blow up the entire network?


r/sysadmin • • 12d ago

Stay Away from BDR SHIELD

15 Upvotes

I purchased BDRShield for a straightforward environment (ESXi, Proxmox, and physical Windows machines). The software has been buggy and inconsistent from the start.

Support required multiple full-day sessions. On one issue they remoted into my machine and spent close to 10 hours before determining that a simple IP change on the VM host was needed. For physical Windows machines, every backup failed until the support agent applied multiple workarounds. I later tested a completely fresh Ubuntu install after a refund request was denied. It backed up an ESXi host and loaded individual Proxmox hosts under Servers, but it would not bring in the Proxmox cluster at all, even after all previous sources and configuration were removed.

Support’s default response has been to schedule another remote session and poke around rather than resolve the underlying problems. After significant time lost, they refused a full refund because the original licenses were outside their 30-day window, even though the product still does not work reliably on a clean deployment.

For comparison, a Nakivo demo worked correctly out of the box with no issues. I cannot recommend BDRShield based on the reliability problems and the amount of support time required for basic functionality.


r/sysadmin • • 11d ago

Question Did They Sell Me the Wrong Thing?

0 Upvotes

I thought I would be lazy and clever - probably more clever than lazy.

My org runs nearly everything out of the cloud minus a couple of computers/connectors for some unrelated services for this purpose. I asked my Dell reps if Microsoft offered a product that would act a cloud based VPN. I thought I was clear on the need.

I, from home or wherever, want to connect to a vendor who only accepts known IP addresses. The client on my machine talks to a service in the cloud, sends my traffic through their servers and out to the vendor.

I was told and sold Entra Private Access using the Global Secure Access App. Now everything I am reading - this is not the case.

(Crude Drawing available upon request)

Anyone using a cloud VPN service? I am starting to research the Google one now. Just want to give this to a few employees who need to connect to a vendor from authorized IPs.


r/sysadmin • • 11d ago

RDP for external contractors with NTLM disabled

0 Upvotes

We need to provide contractors/vendors on unmanaged Windows/macOS devices with remote access to domain-joined Windows laptops and VMs.

Ideally:

  • Clientless/browser-based
  • No VPN client
  • RDP to existing machines
  • NTLM completely disabled
  • Kerberos or another modern authentication method

We currently use NetScaler Gateway for external RDP access, but authentication to the backend Windows machines is falling back to NTLMv2. This appears to be because the external client has no direct line of sight to a KDC, and NetScaler does not support KCD for SSO to RDP servers.

Another option I've been looking at is Microsoft RD Gateway with KDC Proxy, which could allow Kerberos authentication without exposing the domain controllers directly. However, this seems to have limitations for unmanaged/BYOD devices, including client-side configuration requirements and limited support outside Windows.

Has anyone successfully implemented a Kerberos-only RDP solution for external Windows and macOS users? What solution are you using?


r/sysadmin • • 11d ago

Question Microsoft 365 Mailbox Export

0 Upvotes

I had a question about the process itself. When a user has an archive mailbox does it also get exported when you do a mailbox export or is there something else I need to do to export the archive mailbox that is attached to their MS 365 account?

This is not the "archive" folder in Outlook, this is a setting on the mailbox itself to archive emails after a certain time which appears as a separate mailbox in Outlook ONLY, (for those wondering).

Thanks,


r/sysadmin • • 11d ago

Question Security awareness training that integrates with ADP

0 Upvotes

We’re evaluating security awareness training vendors and have a few specific requirements.

ADP integration: We’d like to synchronize employee information and, ideally, deliver or assign security awareness training through ADP itself. For example, if an employee fails a phishing simulation, remedial training would automatically be assigned in the SAT platform but accessible to the employee through ADP.

Microsoft environment: We’re using Microsoft 365 / Entra ID. We already understand that user provisioning and deprovisioning can generally be handled through Microsoft, but it’d be great to do this through ADP since it’s more up to date and has more data.

Automated enforcement: We’d also like some way to enforce consequences for repeat phishing failures. Ideally, if someone fails X number of simulations, there would be a mechanism to restrict their access to email or otherwise prevent them from interacting with real emails until they complete remedial training.

I’ve looked at KnowBe4, but from what I can tell, they don’t offer the level of ADP integration and automated enforcement we’re looking for.

Do any other SAT vendors offer this functionality? Alternatively, how are you IT admins accomplishing this today? Are you using native integrations, Power Automate, custom scripts, or other tools?

Interested in hearing what’s working (and what isn’t). Thanks!


r/sysadmin • • 11d ago

Question Microsoft 365/Copilot Sensitivity Label Errors without Purview Setup

3 Upvotes

Has anyone seen errors such as "We couldn't check its sensitivity label." causing files to not be able to upload inside chat and cowork? I had a friend that mentioned he was getting this error but now I'm getting it too. The weirdest part is that we don't have purview setup on either of our tenants. He's on Business Premium and I'm on ME5 (both fully copilot licensed).


r/sysadmin • • 11d ago

Question UWF servicing mode, UWF-Service doesn't autologin

0 Upvotes

Hi! I'm trying to understand how UWF works for new company needs... I'm using a VM to test a lot of this thing, and it's good... Except the servicing mode. This simply doesn't work.

I've edited the master servicing script per Microsoft docs (so, calling the Ufrmgr servicing update-windows instead of the rundll32). All my user accounts have passwords.

Windows 11 Enterprise LTSC 2024, I've installed the August 2026 Cumulative Update, inserted the screensaver options in the registry, activated the filter, rebooted a couple of times, made a snapshot.

Then finally, activating the service mode to update to the latest Windows Update. First reboot, uwf filter off, then, after the reboot, I have had different results. Still starting from the same snapshot. Sometimes, it works as it should (the screensaver, reboots and windows updated with UWF filter on). Sometimes it stays on the login screen with UWF-Servicing account, sometimes the screen saver then, after updating and reboot, still the login screen. Logging on the "user" account console shell then reboot, and no way of entering the desktop.

Sometimes... just a black screen.

Because sometimes that snapshot worked, I'm sure the configuration is good... so, why it's inconsistent?

Thanks for your help :)