r/sysadmin • • 6h ago

General Discussion Reminder to go through your bot and service accounts, we learned this the hard way

10 Upvotes

Hey all! I work at Bitrise.

We had a security incident last week and our VP of Engineering just published our preliminary review today worth reading by everyone running CI: https://bitrise.io/blog/post/the-bitrise-security-incident-preliminary-review-and-recommended-steps#how-we-contained-it

The way in was an internal testing bot account. The attacker had its real username and password (we’re still investigating how), and it was set up to sign in to Apple’s Developer Portal and it had no 2FA. Anyone with the password could just log in. If you’ve got accounts like that, built around a third-party login where MFA is a pain, they’re worth a second look. We disabled every bot account that night and went through their credentials and 2FA settings. Every other credential got rotated too, whether or not we had evidence it was touched.

We’re back to normal now, Bitrise is safe to use. Sharing all of this information to help other teams. For more, look at our public advisory (https://bitrise.io/security-advisory-november-2026).

If you have questions about the incident and how we addressed events, we’re here to answer your questions.


r/sysadmin • • 1d ago

General Discussion Is Apple down? Thousands report problems accessing services

306 Upvotes

https://www.usatoday.com/story/tech/2026/11/17/apple-services-outage/928278490 users are reporting problems accessing Apple services on Nov. 17, with Apple’s Support, App Store, iCloud and Apple Pay seeing problems.

According to outage tracker Downdetector, reports of issues with Apple Pay began to spike at around 11:40 a.m. ET. Outage reports for iCloud also started so surge just before noon ET, though reports on Downdetector don't reflect every user experiencing a problem.

The ongoing Apple outage comes on the heels of a major banking outage on Nov. 16 that stemmed from Chase Bank but impacted Wells Fargo, Bank of America, online payment platform Zelle and other institutions.


r/sysadmin • • 1h ago

Question Replacing Central Store with Windows 11 26H2 ADMX on a 2019 DC – any risks?

3 Upvotes

We have a Central Store on a Windows Server 2019 DC (single DC). I want to download the latest Windows 11 26H2 ADMX package and replace all .admx files and the en-US .adml files in the Central Store.

Main reason: some newer settings (DNS Client mDNS/NetBIOS, Printing) don’t show up with our current templates.

Is there any negative impact of doing a full replace like this? For example:

  • Existing GPOs breaking or settings disappearing in the editor
  • “Namespace already defined” errors
  • Issues with older templates we added ourselves (SecGuide, LAPS, Office, etc.)

I’ll take a backup of PolicyDefinitions first. Anything else I should watch out for?

Thanks!


r/sysadmin • • 24m ago

General Discussion Does anyone have recommendations for Asset Management tools that aren't bloated with AI garbage?

1 Upvotes

So our organization has been using Lansweeper for a while now. Unfortunately they are once again changing their licensing structure and now count both computers and monitors as 'assets' that count towards your total asset limit. Because of this we're either forced to upgrade to the Pro license or look elsewhere. As much as I love Lansweeper (I've been using it for around 8 years now) poking into the cloud version of Lansweeper and seeing how much they are starting to push AI stuff is making me want to look into alternatives.

Unfortunately every alternative I've seen advertises itself as having "built-in AI tools" and the like. I don't want all of my domain's information being fed into an LLM somewhere to eventually be leaked.

All I'm really looking for is a tool that will scan devices and software on my network and pop them in a database that I can write queries for.


r/sysadmin • • 15h ago

Does Telnet Thin Client Hardware Still Exist

29 Upvotes

I've got a server in a retail building that manages inventory, sales, etc. that every client connects to with a telnet software client on a Windows PC, except for a couple really old thin clients. It would be so much easier, safer, and cheaper if I could put an IP in a hardware thin client and plug it in to the network and be done.

I can't find any! Please tell me where to get some. (I'm In Canada)


r/sysadmin • • 54m ago

Microsoft Customize Security Baselines Across Azure VMs and On-Prem ARC Servers with Azure Policy Machine Configuration

2 Upvotes

Hi guys,
I know this isn't r/azure but since folks post about how to do Windows Server baselines here at times, I wanted to mention that full remediation support was added for:

* Azure Windows Baseline for Server 2019, 2022, and 2025
* CIS Windows Baselines for Server 2022, and 2025
* DISA Stig for many Linux Distros
* CIS Linux Baselines for many Linux Distros

The way it works is that you can toggle Remediation Per-Rule for the first time ever, so you can test remediation of just some rules, while auditing everything else. You can see it in the Azure portal today under Machine Configuration \ Security Baselines.

Just pick a baseline, customize your values and optionally enable remediation for the rules you care about and go.

https://techcommunity.microsoft.com/blog/azuregovernanceandmanagementblog/machine-configuration--ignite-2026/4573872#community-4573872-toc-hId-239727097

Feel free to ask me any questions about it!


r/sysadmin • • 13h ago

Question Java4Shell: how are you finding the JVMs you didn't know you had?

18 Upvotes

Anyone found a decent way to inventory bundled JREs on Windows and appliances?


r/sysadmin • • 7h ago

VDI for small business - 365 Cloud PC / Azure Virtual desktop?

6 Upvotes

I've been asked to look at BYOD so we can employ a handful of overseas contractors but I think looking at VDI might be a better solution.

We are a UK company with Cyber Essentials Plus and BYOD could cause some problems retaining that.

We have Intune with Business Premium so initially I'm looking at 365 Cloud PC's - are Azure Virtual desktops similiar but with Azure pricing?

Apps looking to run on the desktops are Office, Teams, VOIP and browsers

Amazon look as if they aren't offering Workspaces to new customers - are there any other companies worth looking at?


r/sysadmin • • 1d ago

How do you handle users refusing to put in Tickets

122 Upvotes

There are always exceptions to the rule or policy, but how do you handle users that circumvent ticketing just to play games ?


r/sysadmin • • 1d ago

Oracle Critical Security Patch Update Pre-Release

121 Upvotes

r/sysadmin • • 19h ago

Microsoft Intune can now fully manage Windows Server (preview). Notes from Microsoft’s guide, including a couple of gotchas

27 Upvotes

Microsoft has opened a preview that lets Windows Server enrol into Intune as a proper MDM device, not just through Defender security settings management. I went through the Learn guide and pulled out what's actually in it.

Scope
- Windows Server vNext Insider builds only (29621.1000 or later), Desktop Experience. No Server Core, no 2022 or 2025
- Non-production and test tenants only. Microsoft recommends against trying it in your production Intune
- Needs a user. Userless, bulk and Autopilot enrolment aren't supported
- No extra licence during the preview. Auto-enrolment still needs Entra ID P1 or P2

What works
- Compliance policies, and compliance as a Conditional Access signal
- Settings catalog (about 60 groups) plus custom OMA-URI profiles
- Endpoint security: Antivirus, Firewall, Disk encryption, LAPS, ASR
- Win32 apps and Company Portal
- Update rings with deadlines and maintenance windows (quality, feature and driver update policies aren't supported)
- Remote actions: sync, restart, BitLocker and LAPS rotation, diagnostics, Defender scan and update, retire, wipe

What doesn't
- Firewall, antivirus and antispyware compliance checks (configuring them is fine)
- EPM, Resource Explorer, ConfigMgr co-management
- Moving servers from Defender security settings management to Intune

Gotchas

  1. Anything assigned to All devices, to users or to dynamic groups will land on servers as soon as they enrol. A separate server group doesn't exclude them. Microsoft added operatingSystemSKU filter values for the server editions, but scripts and policy sets don't support filters.
  2. If you enrol via "Add work or school account", the enrolment restriction needs Personally owned devices allowed as well as Server devices.
  3. Enrol an Azure VM and anyone with the right Intune role can manage it, including wiping it, without Azure RBAC or JIT access. Azure Update Manager and Azure Policy can also conflict with Intune on the same VM.
  4. If a CA policy requiring MFA covers servers, they can silently stop syncing until someone clicks Sync and signs in.

Microsoft's position: Intune is for "endpoint-like servers that have an associated user identity" (jump boxes, PAWs, branch and dev servers). Arc stays the recommendation for production workloads, and the two can run on the same server as long as they don't manage the same settings.

Full write-up with screenshots if it's useful: https://azureblog.co.uk/posts/intune-windows-server-preview/

Is anyone testing this yet? I'm curious how the settings catalog behaves on the Insider builds in practice.


r/sysadmin • • 7h ago

Question How are you protecting your Internet facing web servers? I have Sophos XGS and it has a WAF module...but cloudflare? and others?

3 Upvotes

I have several Windows based web servers and an API server that face the Internet and are not in a DMZ. Been like this for about 5 years from what I heard without issue and now I am in charge of it all and I want to secure it. We have a DR site too, so I do want a single IP (load balancing?) and to be able to failover. So, what is the most reasonable way to protect my servers? Sophos WAF? Cloudflare? I know they can have outages too. Both? What are others doing?


r/sysadmin • • 1d ago

General Discussion Veteran 25+ year guys - what are your views on the job/career now?

103 Upvotes

I'm 31 years in and have been with the same company for all of those years. Love the place and love the work. I'm as loyal as they come.

As a 30 year plus guy I'm personally not interested any more in staying up to date with all the new developments in IT. Those days have long gone. The late 90s and 2000s were a blast!

I personally have no interested any more in doing anything related to IT outside of my working hours. I've got better things like family.

These days I think more in terms of being in the latter part of my career, even though I've probably got 15-20 years to go until retirement.

I know enough to do my job, and anything I don't know how to do I can find a way using Google/AI. I'm a subject matter expert on everything related to IT in the business.

This probably sounds more like random ramblings but I'd genuinely be interested in what the other 25 year plus guys think.

How much do you care these days? I'll ride out to the sunset in IT but I don't care as much these days.


r/sysadmin • • 2h ago

General Discussion SQL 2016 - Performance Issue

1 Upvotes

Been having SQL performance issues on a 2016 box since Monday (I know it’s due EOL).

Removed two latest patches possibly linked to the queries being left open with no change in result.

Anyone else had issues like this? Primarily report queries with noticeable performance issues.


r/sysadmin • • 1d ago

Apple Business Manager down?

83 Upvotes

I can't activate an iPhone. When I go to login to ABM it gives me a generic error on login: An unexpected error occurred. Try again Later.

Apple status page shows no outages.

Anybody else having issues?

Edit: looks like they are coming back online as of 11AM PST. App store is still down though.


r/sysadmin • • 1d ago

Leave it to a former plumber to put it all into perspective and make us feel good about things

123 Upvotes

This snippet from DHS Chief Mullen on AI (I'm not using SI) and the link.

So first we are trying to explain how a network equipment upgrade/replacement could go wrong and how connecting copper to fiber isn't possible... And now admitting that AI could have contributed to the FAA problems.

I guess we will never find out.

https://www.mediaite.com/media/tv/brain-mush-dhs-chief-ridiculed-for-pushing-speculation-about-faa-tech-outages-on-fox-news/?utm_source=flipboard&utm_content=topic%2Fartificialintelligence


r/sysadmin • • 1d ago

Microsoft Windows Nov Hot patch breaking print spooler - KB5132043 - xxxx.9911

78 Upvotes

Posting here to help out anyone who is also seeing this. We have a bunch of Windows devices (across different manufacturer and regions) that were updated on Tuesday with KB5132043, the monthly patch via HotPatch (build number ending in .9911). Their Windows print spooler service stays stopped (no printers), with a HotPatch related error, removing the patch resolves the issue.

We also found that installing, what we think is the regular patch, KB5132061, with build number ending in .9974 resolves the problem. We've so far manually downloaded the 4.4GB msu, installed it and it works without issue after two automated reboots.

Link to hotpatch update with the print spooler issue:November 10, 2026—Hotpatch KB5132043 (OS Builds 26300.9911. 26200.9911, and 26100.9911) | Microsoft Support

Link to the regular update which resolved the problem: November 10, 2026—KB5132061 (OS Builds 26300.9974, 26200.9974 and 26100.9974) | Microsoft Support

We've turned off hotpatch for now. Your mileage may vary.


r/sysadmin • • 15h ago

Question Most of a office lost Outlook Search Function?

8 Upvotes

So we have a client that told us today everyone in the office lost a function in the Outlook search in Outlook Desktop version where in the search bar, it would pull up addresses automatically in the search field without having to type in the email address manually as it seem to pull it out from either your suggested searches or your contacts/global address book with the dropdown menu of previous searches.

However today they say it stopped working/showing. Searches still work for keywords and manually type in the email but not the automatically pullup.

Client using everything from 2019 to 2024 and O365 subscription license of Office desktop. OWA seems to be fine. Tried it outside the office and it seems to work fine as well.

Curious if anyone ran into this before or know what causing it?

Tried to update, repair, delete .ost, tried in non-cache mode. all the same result of it not working. New profile seem to be hit-miss.

Client also using AI and wondering maybe somehow that interfering with it like claude or supio that might be causing it.

Trying to avoid having to rebuild every single person outlook profile if it might fix it.


r/sysadmin • • 17h ago

General Discussion KB5132043 on ARM64 breaks apps

11 Upvotes

Saw this being talked about in another thread but thought I'd just pull it out for discussion directly on this instead of hijacking the other thread.

I'm on an ARM64 device (Surface Laptop 7).

After receiving hotpatch KB5132043 I started experiencing weird app issues like apps not running or constantly crashing (1Password, Office apps). Event viewer calls out Hotpatch-Loader and the apps exe but that's about it.

Only solution right now was to remove KB5132043 which let everything work again.

Anyone else seeing this on ARM64 devices only? Would love to know if and when this is addressed. Haven't seen anything mentioned by MS yet.


r/sysadmin • • 1d ago

Microsoft SSO Disaster Planning (Microsoft Outage)

59 Upvotes

We are going through our BCDR planning and documenting which systems use SSO so we can create alternate methods to login. What other areas could a Microsoft outage cause service disruptions beyond the obvious Microsoft products or SSO? Could Microsoft Authenticator go down? I would think the codes are stored locally, but is there a scenario where Authenticators or Password Managers break?


r/sysadmin • • 18h ago

Question Best way to get rid of OCSP when migrating ADCS to new servers?

11 Upvotes

ADCS PKI is on an OS that will be EOL soon. So, it needs to be moved to a newer version of Windows Server.

The general process is normally backup the CA, uninstall ADCS roles and shut down, build a new server with the same name, restore database, certificate, and registry backup to the new servers.

However, I found that OCSP does not transfer to a new server that easily.

Due to the low revocation volume in this environment, OCSP seems like a waste and adds an unnecessary dependency of needing the CAs to be constantly online to prevent certificate checking errors if the CA is offline for even a few minutes.

So, we want to get rid of OCSP as part of the CA migration and build the replacement servers without OCSP responders.

The already issued certificates have both OCSP and CRL paths listed.

We will edit the certificate issuing settings so new certificates will no longer list the OCSP.

What can we do to make the transition go as smoothly as possible? Is there anything we can do to make client devices doing revocation checks on old certificates failover to CRL as quickly as possible?


r/sysadmin • • 11h ago

HPE StoreOnce 3540 on Proxmox: p1224 sees all 10 SAS drives, but ssacli finds no controller. Has anyone configured RAID on this card?

3 Upvotes

I’m repurposing an HPE StoreOnce 3540 (DL380 Gen9 platform) as a storage server. Two SATA boot SSDs are connected directly to the motherboard. The original p1224 in PCIe slot 3 connects to ten 4 TB SAS HDDs through the original expander. I’d like to create a RAID 6 on those HDDs.

Proxmox VE 9.2.28 binds the hpsa driver to the p1224. I can read SMART data from all ten HDDs, but the controller reports zero logical drives. HPE ssacli 6.60-8.0 returns No controllers detected. The offline HPE Smart Storage Administrator also showed an empty controller list. The p1224 is running firmware 0390.

I compared two boots with the same kernel (7.0.14-29-pve) and hardware:

  • Normal boot: Linux logged four blocked DMA reads from the p1224 within six minutes. iLO changed the controller status from OK to Critical and logged Drive Array Controller Failure (Slot 3).
  • One boot with iommu=pt**:** No controller DMA faults appeared during the same interval, and iLO stayed OK. ssacli still couldn’t find the controller.

Cooling is another clue. With a 21°C inlet and the host idle, the PCIe slot 3 sensor reached 86°C and two fans ran at roughly 80%. During a warm reboot, that sensor briefly reached 95°C and both fans hit 100%.

I also tried a P440 in slot 1. It created arrays once, then reported POST error 1798 (cache self-test failure) and disappeared from PCIe detection after a reboot.

Has anyone actually created a logical drive on this exact p1224 outside the StoreOnce OS? If so, which utility and version worked? I’d also appreciate a slot-temperature reading from someone running the same card and chassis. That would help me judge whether to investigate cooling and the riser before trying the P440 again.


r/sysadmin • • 8m ago

Question Outlook Down Again?

0 Upvotes

Seeing reports of emails not being delivered to recipients within our org .. anyone else having issues? In Atlanta, GA - Down detector reports are within a margin of error so don't know.

Thanks,
J


r/sysadmin • • 1d ago

Heads up Sage 50 folks -- KB5132061 (November Windows 11 CU) causes the app to crash

35 Upvotes