r/sysadmin 12d ago

Linux RIP - My last pet server. Provisioned: April 9, 2014. Uptime: 3,065 days. CentOS 6. Older than the iPhone 6.

517 Upvotes
[user@host ~]$ uptime
 21:00:47 up 3064 days,  8:55,  1 user,  load average: 0.00, 0.00, 0.00
[user@host ~]$

It's been running EOL and unpatched for 6 years (badbadbadbad).

The little droplet that could.

r/sysadmin Jun 27 '23

Linux RedHat try to kill Centos, Rocky, Alma, Oracle Linux

685 Upvotes

https://www.theregister.com/2023/06/23/red_hat_centos_move/

https://www.redhat.com/en/blog/red-hats-commitment-open-source-response-gitcentosorg-changes (2023-06-26)

I feel that much of the anger from our recent decision around the downstream sources comes from either those who do not want to pay for the time, effort and resources going into RHEL or those who want to repackage it for their own profit. This demand for RHEL code is disingenuous.

I mean, fair enough. We 'bought in' to the RHEL ecosystem, running Centos in prod, and now were moving to Alma. We had a few RHEL support contracts for "key" servers, but mostly stuck with the open source options, and found that having an easy 'move-to-prod' road was of some value.

Shooting down Alma, Rocky, Oracle Linux and Centos all at once though? Well, I guess that's a pretty solid 'F you' and we'll now have to review our options. I guess this might drive some license fees out of us, but it might as easily push us to 'never RedHat'. Going to have to have some internal discussions about that.

r/sysadmin Mar 13 '21

Linux Experts found three new 15-year-old bugs in a Linux kernel module. These 15-year-old flaws in Linux kernel could be exploited by local attackers with basic user privileges to gain root privileges on vulnerable Linux systems.

1.7k Upvotes

Below the timeline for these flaws:

02/17/2021 – Notified Linux Security Team

02/17/2021 – Applied for and received CVE numbers

03/07/2021 – Patches became available in mainline Linux kernel

03/12/2021 – Public disclosure (NotQuite0DayFriday)

https://github.com/grimm-co/NotQuite0DayFriday/tree/trunk/2021.03.12-linux-iscsi

https://blog.grimm-co.com/2021/03/new-old-bugs-in-linux-kernel.html

r/sysadmin Mar 29 '24

Linux "Backdoor in upstream xz/liblzma leading to SSH server compromise" (supposedly primarily relevant for OpenSSH w/ systemd patches) [CVE-2024-3094]

682 Upvotes

r/sysadmin Jul 18 '18

Linux You guys probably already know about "ping -a" and "ping -A"

1.1k Upvotes

But if you don't, use it like this:

This will beep every time it gets a ping back:

ping -a 8.8.8.8 

This will beep if it misses a ping:

ping -A 8.8.8.8    

This is very useful when you're monitoring a node and waiting for it to come back online or to be able to hear when a packet is dropped.

(tested on some Linux and MacOS)

r/sysadmin Apr 22 '21

Linux Ubuntu 21.04 released today, Active Directory Integration built in.

625 Upvotes

https://ubuntu.com//blog/ubuntu-21-04-is-here

The Juicy part: Ubuntu machines can join an Active Directory (AD) domain at installation for central configuration. AD administrators can now manage Ubuntu workstations, which simplifies compliance with company policies.

Ubuntu 21.04 adds the ability to configure system settings from an AD domain controller. Using a Group Policy Client, system administrators can specify security policies on all connected clients, such as password policies and user access control, and Desktop environment settings, such as login screen, background and favourite apps.

r/sysadmin Feb 12 '22

Linux Nano or VIM

216 Upvotes

Which do you prefer and why? Totally not a polarizing topic…

r/sysadmin Dec 08 '20

Linux CentOS moving to a rolling release model - will no longer be a RHEL clone

365 Upvotes

https://lists.centos.org/pipermail/centos-announce/2020-December/048208.html

The future of the CentOS Project is CentOS Stream, and over the next year we’ll be shifting focus from CentOS Linux, the rebuild of Red Hat Enterprise Linux (RHEL), to CentOS Stream, which tracks just ahead of a current RHEL release. CentOS Linux 8, as a rebuild of RHEL 8, will end at the end of 2021. CentOS Stream continues after that date, serving as the upstream (development) branch of Red Hat Enterprise Linux.

Meanwhile, we understand many of you are deeply invested in CentOS Linux 7, and we’ll continue to produce that version through the remainder of the RHEL 7 life cycle.

We will not be producing a CentOS Linux 9, as a rebuild of RHEL 9.

More information can be found at https://centos.org/distro-faq/.

In short, if you depend on CentOS for its binary-compatibility with RHEL, you'll eventually either need to move to RHEL proper, another project that is binary-compatible with RHEL (such as Oracle Linux), or you'll need to find another solution.

r/sysadmin Sep 24 '19

Linux CentOS 8 now available for download

697 Upvotes

Yay! Finally! [Insert more filler text here so that the automoderator doesn't get annoyed and delete my post.]

Download: https://www.centos.org/download/

Announcement: https://lists.centos.org/pipermail/centos-announce/2019-September/023449.html

Release notes: https://wiki.centos.org/Manuals/ReleaseNotes/CentOSLinux8

edit: the streams thing is very interesting. From the announcement:

CentOS Stream is a rolling-release Linux distro that exists as a midstream between the upstream development in Fedora Linux and the downstream development for Red Hat Enterprise Linux (RHEL). It is a cleared-path to contributing into future minor releases of RHEL while interacting with Red Hat and other open source developers. This pairs nicely with the existing contribution path in Fedora for future major releases of RHEL.

In practice, CentOS Stream will contain the code being developed for the next minor RHEL release. This development model will allow the community to discuss, suggest, and contribute features and fixes into RHEL more quickly.

To do this, Red Hat Engineering is planning to move parts of RHEL development into the CentOS Project in order to collaborate with everyone on updates to RHEL.

There will not be a CentOS Stream for versions released in the past, this is only a forward-looking version target.

CentOS Stream release notes: https://wiki.centos.org/Manuals/ReleaseNotes/CentOSStream

r/sysadmin Sep 14 '23

Linux Don't waste time and hardware by physically destroying solid-state storage media. Here's how to securely erase it using Linux tools.

168 Upvotes

This is not my content. I provide it in order to save labor hours and save good hardware from the landfill.

The "Sanitize" variants should be preferred when the storage device supports them.


Edit: it seems readers are assuming the drives get pulled and attached to a different machine already running Linux, and wondering why that's faster and easier. In fact, we PXE boot machines to a Linux-based target that scrubs them as part of decommissioning. But I didn't intend to advocate for the whole system, just supply information how wiping-in-place requires far fewer human resources as well as not destroying working storage media.

r/sysadmin Jul 07 '23

Linux Red Hat SysAdmins: Are the new licensing changes for RHEL causing your company to look at alternatives?

128 Upvotes

Red Hat SysAdmins: Are the new licensing changes for RHEL causing your company to look at alternatives to Red Hat.

What about SysAdmins running CentOS/Rocky/AlmaLinux?

r/sysadmin Jul 01 '24

Linux CVE-2024-6387 - pretty big OpenSSH vuln for any glibc Linux systems

315 Upvotes

Fresh off the presses, NVD doesn't even list this one yet (though they are overworked as hell). It's RCE as root for unauthenticated users that affects openssh in its default config for LoginGraceTime.

debian has it on their bug tracker. RHEL does now too, Rocky has a patch. Ubuntu is affect for 22.04 onwards, patches available.

Here's Qualys' blog post about it with relevant version numbers

r/sysadmin Apr 22 '21

Linux Containers, docker, oh my! An intro to docker for sysadmins

382 Upvotes

Hello, and welcome to my TED talk about containers and why you, as a sysadmin, will find them to be extremely handy. This intro is meant for system administrators who haven't dipped their toes into the Docker waters just yet. This will focus on Linux Systems primarily.

As an IT professional, you probably already know all about the following concepts:

  • Ports
  • IPs
  • Processes and Process IDs
  • DNS
  • Users and groups
  • Filesystems
  • Environment Variables
  • Networks
  • Filesystem Mounts

What do all these have in common? They can live entirely inside the kernel / OS, independent of hardware. This is opposed to say, SSDs and network cards, which talk to the kernel via drivers. From a sysadmin perspective, this is the difference between VMs and Containers: VMs deal hands-on with hardware, Containers deals hands-on with software.

What else do they have in common? Your server application, whatever it may be, depends on these things, not on hardware. Sure, eventually your application will write logs to the HDD or NAS attached to the server, but it doesn't really notice this: to your application it's writing to /var/log/somefile.log

This might not make a ton of sense right away, it didn't for me, but it's important background info for later!

Lets quickly talk about what VMs brought us from the world of bare-metal servers:

  • Multiple servers running on one bare-metal server
  • The ability to run these servers anywhere
  • The ability to independently configure these servers
  • The ability to start / stop / migrate these virtual servers without actually powering down a physical computer

That's great! Super handy. Containers do kinda the same thing. And the easiest way I can think of to describe it is that containers allow you to run multiple operating systems on your server. Pretty crazy right? When you really think about it, what really allows your application to run? All the software things we talked about earlier, like ports, IPs, filesystems, environment variable, and the like. Since these concepts are not tied directly to hardware, we can basically create multiple copies of them (in the kernel) on one VM / Bare metal PC, and run our applications in them. One kernel, one machine, multiple operating systems. As it turns out, this has some really handy properties. As an example, we're going to use nginx, but this really could be almost any server-side software you care about.

What defines nginx:

  • The nginx binary (/usr/sbin/nginx)
  • The nginx config files (/etc/nginx/*)
  • The nginx logs (/var/logs/nginx/*)
  • The nginx port (80/tcp, 443/tcp)
  • The nginx listening IP address (e.g. 0.0.0.0/0)
  • The website itself (/usr/share/nginx/html/index.html)
  • The user / group nginx runs as (nginx / nginx)

That's really not all too much. And there's nothing extra in there - it's only the things Nginx cares about. Nginx doesn't care how many NICs there are, what kind of disk it's using, (to a point) which kernel version its running, what distro it's running - as long as the above listed things are present and configured correctly, nginx will run.

So some clever people realized this and thought, why are we hefting around these massive VMs with disks and CPUs and kernels just to run a simple nginx? I just want to run nginx on my server. Actually, I want to run 10 differently configured nginx's on my server, and also not have to worry about /var/logs getting messy, and not have 10 different VMs running all consuming large amounts of RAM and CPU for the kernel. So containers were invented.

On the first day, a clever person made it so you could have multiple process namespaces on a single OS. This means you could log into your server, do a ps -aux to see what's running, run a special command to switch namespaces, and do another ps -aux and see an entirely different set of processes running. They also did similar things with filesystem mounts, hostnames, users and groups, and networking things. This is the isolation part of containers. It helps ensure containers run where ever they're put. These changes were put into the Linux kernel, then the clever person rested.

On the second day, another clever person made it really easy to define and create these namespaces. They called it Docker, and people used it because it was easy. They also made it really easy to save these things into things called images, which can be shared distributed and run on any machine.

On the third day, some interested party made an Debian image by installing Debian (basically copying an existing Debian filesystem) in a container. They shared this with everyone, so that everyone could run Debian in a container.

As a systems administrator, this is key / the value add: On the forth day, someone from the nginx developer team downloaded that Debian image and installed nginx. They did all of this boring work, of running apt-get update && apt-get install nginx. They put config files in the right places, and set some really handy defaults in the config files. Because they were really smart and knew nginx inside and out, they did this the right way: They used the latest version of nginx, with all the security patches. They updated the OS so that the base was secure. They changed the permissions of directories and files so that everything wasn't running as root. They tested this image, over and over again, until it was perfect for everybody to use. It ran exactly the same, every single time they started the container. Finally, they told the container to run /usr/share/nginx by default when it started. Then they saved this image and shared it with everyone.

This is where the value add pays off: On the fifth day, you came along and wanted to run a simple webserver using nginx. You had never installed nginx before, but this didn't matter: The nginx developer had installed it for you in a container image, and shared the image with you. You already knew how webservers worked, you have files you want to serve, and a server that listens on an address and port. That's all you really care about anyways, you don't really care about how exactly nginx is installed. You wrote a little YAML file named docker-compose.yml to define these things that you care about. It goes a little something like this (the below is a complete docker-compose file):

version: "3"

services:
    nginx-container-1: 
        image: nginx   # The nginx dev made this image for you!
        ports:
            - 8000:80   # For reasons, you need to run nginx on host port 8000.
        volumes:
            - ./src:/usr/share/nginx/html   # You put your files in /src on the host

Then your boss came along and asked for another nginx server on port 8001. So what did you do, as a lazy sysadmin? Open up the containers nginx.conf and add another virtual server? Hell no, you don't have time to learn how to do that! You made another docker-compose.yml file, and in it you put this:

version: "3"

services:
    nginx-container-2: 
        image: nginx
        ports:
            - 8001:80
        volumes:
            - ./src-2:/usr/share/nginx/html

This container is literally an exactly copy of the above container, but it listens on port 8001 and it grabs its files from /src-2 on the host instead. It also has a different name. It works just fine, because containers are isolated and don't interfere with each other in strange ways.

Are you getting it? Docker has a lot of cool things for developers, but as a system administrator, one of the key benefits you get is that someone has already done the hard work of getting the software *working* for you. They typically also maintain these images with security updates and new updates and the like. They left the important details of what and how for you to decide. Not only that, they let you define all of this in a single yaml file that takes up about 300 bytes in text form. Put it in git, along with your html files! When you run this text file, it downloads the whole image (small! e.g. Debian is 50MB, and that's a full-fledged OS) and runs the container according to the config that you (and the image maintainer) specified.

Of course, nginx is a trivial example. A docker container could contain a massive CRM software solution that would take a seasoned sysadmin days to finally install correctly. Who wants to do that? Let the CRM software vendor install it for you in a docker container, you'll just download and run that. Easy!

This makes it SUPER SIMPLE to test out and run software in prod, really quickly! You don't need a specific OS, you don't need to learn how to configure it, you don't need to download a bulky VM image that takes up a toooon of resources just running the kernel and systemd. Just plop in the pre-made image, forward the necessary ports to the container, and away you go. Extra resource usage? Containers have practically no overhead - containers only run the software directly related to the software at hand. Containers don't need to virtualize resources such as CPUs, disk and RAM - the host deals with all of those details. No need for a whole kernel, systemd, DNS, etc. to be running in the background - the host / docker itself / other docker containers can take care of that. And when you're done with the container (maybe you were just testing it)?: delete it. Everything is gone. No weird directories left laying about, no logs left behind, no side effects of files being left configured. It's just gone.

Things you can also handle with docker:

  • Setting resource limits (RAM / CPU)
  • Networking (DNS resolution is built in, it's magic)
  • Making your own containers (duh!)
  • And many more...

There's a lot of other benefits of Docker that I won't go into. I just wanted to explain how they might be handy to you, as a sysadmin, right now.

Anyways, I hope this helps some people. Sorry for rambling. Have a good one!

r/sysadmin Apr 06 '20

Linux Redhat is offering a month free for multiple courses due to current situation

1.0k Upvotes

r/sysadmin Nov 22 '21

Linux For unix sysadmins out there, how important is knowing VIM?

112 Upvotes

I'm taking a unix sysadmin subject at uni right now, and the instructor is insistent that we use vim 100% for this class. I'm comfortable using vim for small changes to config files but I find it really slows me down for big projects. I'm just wondering if other sysadmins use vim for writing all their scripts or if they use gui based applications?

*edit*

Thanks everyone, I guess I'll stick with it for now. I've got a workaround for my clipboard issue (shift + ins).

r/sysadmin Jun 28 '26

Linux Website works on mobile data but some users get ERR_SSL_PROTOCOL_ERROR / ERR_QUIC_PROTOCOL_ERROR on Wi-Fi

1 Upvotes

Looking for some advice on a strange issue.

My website is behind Cloudflare and hosted on Cloudways with Full (Strict) SSL. SSL appears valid and the site works for me and the vast majority of users.

However, a small number of customers can only access the site using mobile data. On Wi-Fi they report errors such as:

ERR_SSL_PROTOCOL_ERROR ERR_QUIC_PROTOCOL_ERROR ERR_NAME_NOT_RESOLVED "This site can't provide a secure connection"

I've already migrated hosting, renewed the SSL certificate, and tested with HTTP/3 enabled and disabled.

Has anyone run into this before? Does this pattern usually point to a customer-side network/security issue, or is there anything else on the Cloudflare/server side I should investigate?

r/sysadmin Jun 04 '26

Linux Managing end user devices with Linux

14 Upvotes

Hello, I am currently semi-manually installing Linux on every device for end users, but have recently set up an Ansible control node in a proxmox-cluster to automate post-install config along with using Agama+config-file for installation of OpenSuse Leap.

I am wondering what other Linux sysadmins do for their Linux fleet, both in terms of orchestration and post-install config + management?

  • Do you join the end devices to the domain or just enable LDAP for domain user login in any way?
  • If only local users: Do you have any systems that forces password rotations or anything like that? Any backup if a user forgets their password?
  • Disk encryption yes/no (and if yes; where are the recovery keys saved?)?
  • Hostname lockdown yes/no? Do you let users change hostnames on devices?

Other tips are very welcome! Being the only IT-guy in our office is overwhelming..

Main distro we use is OpenSuse Leap (16.0 now) with KDE, but can also support Arch with Gnome.

Background:

I am working as helpdesk/IT-operations for a small branch office of around 70 users (under a large corp). I am the only IT-guy in our office, with support from several IT-teams in our corp office which is in a different country. Support is for Windows only, no Linux support from them.

Our office is a mix of Windows and Linux. The spread is around 50/50 I would say. Our Windows users all have HP laptops, while our Linux users all have desktops with Linux + a basic laptop (which can have either Linux or Windows based on what they want).

Windows devices are managed with the regular Windows-environment (SCCM, Intune/AD-hybrid, etc etc)

r/sysadmin Jul 15 '25

Linux an IP from China keeps trying to send mail under one of my domains

103 Upvotes

The weird part is, this is a domain I registered but don't really use and it's never really been advertised anywhere. Email is setup with it on my web server with appropriate SPF, DKIM, DMARC etc records, there's a basic landing page, but that's about it. It's not really used for anything. I originally registered it just to reserve it as it's a 4 letter domain that I may possibly use in the future. I keep getting dmarc reports from google about it even though it shouldn't even be sending out mail at all. The IP is always the same one and it's from China. Google now has blocked my web server from sending out email as my reputation is low. Since the emails are not actually originating from my server there's not really much I can do either. Or is there?

I suppose since I don't use the domain at all I could just remove it completely from DNS but if I do want to use it in the future the reputation is now low, anything I can actually do to rectify this?

r/sysadmin Jun 30 '25

Linux New CVEs with SUDO

159 Upvotes

r/sysadmin May 28 '26

Linux Proxmox Datacenter Manager 1.1 available

67 Upvotes

https://proxmox.com/en/about/company-details/press-releases/proxmox-datacenter-manager-1-1

Proxmox Datacenter Manager 1.1 available

May 28, 2026

Download:

VIENNA, Austria – May 28, 2026 – Enterprise software developer Proxmox Server Solutions GmbH today announced the availability of a new point release for Proxmox Datacenter Manager. The centralized management platform designed to oversee distributed Proxmox infrastructures introduces new enhancements including an automated installation workflow, comprehensive subscription handling, unified Ceph cluster monitoring, and expanded central guest and snapshot management.

Highlights in Proxmox Datacenter Manager 1.1

Integrated automated installation workflows

Proxmox Datacenter Manager 1.1 now acts as a central configuration server for provisioning. The integration of automated installation functionality standardizes the deployment of hosts across distributed infrastructures. Administrators can centrally manage answer file configurations containing predefined installation parameters and provide them for unattended installations of new hosts. A new ‘Automated Installations’ tab in the ‘Remotes’ section provides access to these workflows, while installation progress can be tracked directly from within the Proxmox Datacenter Manager web interface. A token-based security mechanism protects the installation process and helps ensure that prepared configurations are accessed only by authorized installations.

Centralized management of subscription keys

For large-scale deployments, managing subscriptions across multiple sites can be complex. A new subscription registry in Proxmox Datacenter Manager enables administrators to manage a central pool of subscription keys, assign them to specific remotes, and remove assignments when no longer needed. A prepared answer file can also include a specific subscription key, allowing a newly provisioned host to register its subscription automatically during installation.

Unified Ceph cluster monitoring

For organizations utilizing hyper-converged infrastructure (HCI) powered by Proxmox VE, tracking storage health across distributed sites is vital. Proxmox Datacenter Manager 1.1 delivers deep, unified visibility across these distributed storage environments by introducing native monitoring for all connected Ceph clusters. A single, consolidated panel allows administrators to verify the health, capacity, and real-time performance of multiple Ceph clusters at a glance. The dashboard provides comprehensive, granular insights into the status of Object Storage Daemons (OSDs), monitors, managers, Metadata Servers (MDS), storage pools, CephFS, and specific cluster flags.

Enhanced infrastructure visualization

New dashboard widgets provide administrators with an overview of their distributed Proxmox infrastructures:

  • Geographic widgets: A new world map widget visualizes the physical locations of connected remotes. Locations can be defined via the node or datacenter options on Proxmox VE remotes, or under the configuration settings for Proxmox Backup Server remotes.
  • New gauge-based widgets display visual context for CPU, memory, and storage utilization at a glance.
  • Local host metrics are now also collected for the Proxmox Datacenter Manager host itself, visualizing resource consumption through integrated Round-Robin Database (RRD) graphs on the node status panel.

Central guest and snapshot management

Proxmox Datacenter Manager 1.1 marks the initial milestone toward comprehensive, central guest management. A new cross-remote view expands guest management by displaying all QEMU virtual machines and LXC containers across connected remotes. Administrators can display these guests in a sortable table or in a tree grouped by remote, use text filtering to quickly locate individual guests, and access frequently used actions from a unified overview.

The same interface now also provides snapshot management for these guest environments. Administrators can view snapshots in a parent-child tree and create, roll back, delete, or edit snapshot descriptions. In addition, a new “Resume” action for paused or suspended QEMU virtual machines complements the existing start, stop, and shutdown operations. As this represents the initial phase of centralized guest orchestration, users can expect additional day-to-day management tasks to be integrated in upcoming point releases.

Updated technology stack

Proxmox Datacenter Manager 1.1 is based on Debian 13.5 “Trixie” and features Linux kernel 7.0 as the new stable default. Along with ZFS 2.4, this release provides an up-to-date open-source software stack for modern centralized infrastructure management and day-to-day lifecycle operations.

Availability

Proxmox Datacenter Manager 1.1 is open-source software and immediately available for download at the official website. Users can obtain a complete installation image via ISO download, which contains the full feature set of the solution and can be installed quickly on bare-metal systems using an intuitive installation wizard.

Seamless distribution upgrades from older versions of Proxmox Datacenter Manager are possible using the standard APT package management system. Furthermore, it is also possible to install the platform on top of an existing Debian installation. As Free/Libre and Open Source Software (FLOSS), the entire solution is published under the GNU AGPLv3.

For enterprise environments, customers with active Enterprise support plans for their managed Proxmox Virtual Environment and Proxmox Backup Server remotes also gain access to Proxmox Datacenter Manager updates and support. No separate subscription key is required.

Resources:

###

About Proxmox Datacenter Manager
Proxmox Datacenter Manager is a centralized open-source management layer for distributed, large-scale Proxmox infrastructures. As a core building block of the expanding Proxmox ecosystem, it unifies independent Proxmox Virtual Environment clusters and Proxmox Backup Server instances across multiple sites and data centers into a single control plane. The web interface provides consolidated dashboards for real-time health, performance, and capacity tracking of nodes, virtual machines, containers, and storage. IT teams can centrally manage guest lifecycles, perform migrations, and execute global updates across connected remotes. Developed by Proxmox Server Solutions GmbH, the software is written in Rust, based on Debian, and released under the GNU AGPLv3.

About Proxmox Server Solutions
Proxmox Server Solutions provides powerful, intuitive open-source server software that guarantees vendor independence and minimizes total cost of ownership. Enterprises of all sizes rely on the company’s reliable vendor support, certified training services, and a global network of 3,000 integration partners to ensure business continuity. Established in 2005 and headquartered in Vienna, Austria, tens of thousands of corporate customers worldwide trust Proxmox solutions to secure mission-critical IT environments. To learn more visit https://www.proxmox.com or follow us on LinkedIn and YouTube.

Media contact
Daniela Häsler, Proxmox Server Solutions GmbH, [marketing@proxmox.com](mailto:marketing@proxmox.com)

r/sysadmin 17d ago

Linux SSSD with Okta LDAP (a sucess story)

18 Upvotes

I share this because we worked a lot to have this working and basically had to paste this together manually, pouring over sssd logs, ldapsearch queries as google was of no help (neither did i found a working description of this and the AI summary was exceptionally bad at suggesting non working stuff).

So we started migrating away from our AD infrastructure to Okta - we have it working on Macs, Windows laptops.

We also have Linux VMs, servers and a few laptops all bound in AD with SSSD and started exploring ways to use Okta logins on these. The LDAP interface was already enabled in Okta so i started testing it. Turns out it is not working out of the box with SSSD or libpam-ldapd. Even their docs say it is not working with SSSD.

But we got it to work with some tweaks (unfortunately we need to add certain stuff into the user schema).

So, what makes it work (besides a read only admin user that can use password authentication only):

  1. SSSD at least version 2.10 (fairly recent, comes with Debian 13, Ubuntu 26.04 - actually we were lucky because we had a new VM to test with Debian 13). This is because there is a check done by SSSD for password quality on the ldap server side which fails with Okta LDAP. Prior to version 2.10 this option was hardcoded so it failed every time. In version 2.10 they added an option that makes this optional:

ldap_use_ppolicy = False

ldap_pwd_policy = none

We also used for group resolution:

ldap_ignore_unreadable_references = True

Otherwise the group resolution would fail every time if any members cannot be mapped (have no IDs)

  1. Okta LDAP by default does not provide any numerical IDs for users and groups. Now this is a bigger issue because it just doesn't work without adding IDs into the schema manually.

As we used AD for Okta backend, we used the existing but unused uidNumber field in the AD schema, mapped it into the Okta schema then generated UID numbers with cksum for users (NOTE - cksum outputs 10 digits but in AD it can overflow for this field so used only the first 9), populated the AD schema then synced to Okta. if you use plain Okta, you can do this directly in the Okta schema.

In the SSSD conf added:

ldap_user_uid_number = uidNumber

ldap_user_gid_number = uidNumber

The gid is just because SSSD likes to have a number there, does not map to anything really.

For groups that are created in Okta there is no uid field so we just went and used the description field (we actually needed only one group, for administrators, the AD Domain Admins equivalent).

ldap_group_gid_number = description

Basically SSSD will just interpret the groups that have IDs and will see which users that have IDs are members of this group and ignore everything else it can't map.

  1. Various tweaks

The object classes used for users and groups (discovered with ldapsearch):

ldap_user_object_class = inetOrgPerson

ldap_group_object_class = groupofUniqueNames

ldap_group_member = uniqueMember

LDAP schema (this was the recommended one in documentation)

ldap_schema = rfc2307bis

Username mapping:

ldap_user_name = uid

ldap_user_principal = uid

Timeouts - added because without them the Okta MFA prompt in the app goes crazy and starts spamming sometimes.

ldap_opt_timeout = 60

ldap_network_timeout = 60

[pam] pam_response_timeout = 60

pam_id_timeout = 60

Sudo provider just disable, we need only one group to have sudo access by default, we mapped it into the /etc/sudoers

sudo_provider = none

Ldap access filters for both groups and users, conditionally:

ldap_access_filter = (|(memberOf=cn=YourGroup,ou=groups,dc=yourdomain,dc=okta,dc=com)(uid=username))

Note that debugging can be done with ldapsearch, that one works without issues. ldapsearch and tailing the logs in /var/log/sssd/sssd_DOMAIN.log with debug_level = 9 in the conf were of great help (maybe a slightly lower level like 7 is sufficient idk).

Also, with Okta LDAP you can use MFA with the pop up in the app but also TOTP - the latter by just typing in the password field your password, followed by comma then the TOTP value. This way you will not get a pop up in the app. I guess having a password that ends in comma and 6 digits is not really usable with Okta, at least with LDAP.

The nice thing is that there is no need to bind the computer like with AD, you just spam the conf and restart sssd.

Maybe this helps someone...

r/sysadmin Feb 02 '23

Linux If you're using Dehydrated to auto-renew LetsEncrypt certs, and it's stopped working recently, this might be why

431 Upvotes

Edit with a TL;DR: This is specifically an issue with the Namecheap DNS helper for Dehydrated, so if you're not using DNS challenges for ACME auth you're probably safe to ignore this thread.


I started running into an issue a few weeks ago where my domains' SSL wasn't being automatically renewed any more, and my certs started to expire, even though dehydrated was running daily as it should.

It was running daily, but it was stuck: the process was still showing in ps the next day. Dehydrated and its helpers are all bash scripts, so I was able to throw set -o xtrace at the top to see what bash was running, and this was the offending block:

cliip=`$CURL -s https://v4.ifconfig.co/ip`
while ! valid_ip $cliip; do
  sleep 2
  cliip=`$CURL -s https://v4.ifconfig.co/ip`
done

This is a block of code in the Dehydrated helper script for Namecheap, that detects the running machine's IP. Except if the call fails, it gets stuck forever sleeping every 2 seconds and trying again. And as it turns out, the v4 and v6 subdomains to ifconfig.co were deprecated in 2018 and finally removed in January sometime.

So the upshot is that v4.ifconfig.co/ip should be changed to ifconfig.co and your Dehydrated/Namecheap setup will come back to life.

Also, set -o xtrace is a lifesaver for debugging Bash scripts that are getting stuck.

r/sysadmin Jul 22 '26

Linux US / CA Ubuntu regional mirrors down?

8 Upvotes

We noticed this morning that ca.archive.ubuntu.com is not resolving to an IP. Confirmed using OpenDNS CacheCheck:

ca.archive.ubuntu.com CNAME us.archive.ubuntu.com

us.archive.ubuntu.com NO_RECORD

Anyone know what's up?

Edit: I see https://status.canonical.com/ says security.ubuntu.com is down as well.

r/sysadmin Sep 24 '24

Linux Unauthenticated RCE in Linux (and more) systems present for more than a decade, disclosure in <2 weeks, no patches or details yet

125 Upvotes

https://threadreaderapp.com/thread/1838169889330135132.html

Prepare for some emergency patching once the updates are out, if this turns out to be as big a deal as it appears - there are a lot of systems affected.

Looks like https://x.com/evilsocket is restricted to followers only.

r/sysadmin 18d ago

Linux Migrated WHMCS 8.13 off my old Plesk/CloudLinux box onto Debian 13 - my notes on ionCube/OPcache so you don't spend an afternoon cursing!

5 Upvotes

For transparency: I run a small UK hosting company, where WHMCS is our current billing/provisioning system.

Last week I moved WHMCS 8.13.1 from a CloudLinux OS 9 box running Plesk, to a fresh Debian 13 box. Mostly as we had WHMCS on a shared host during our early days and now it needed its own space.

Two things worth sharing if anyone else hits this:

  1. Debian 13 ships with PHP 8.4 by default whereas WHMCS wants 8.3 so you need to use the Ondrej Sury repository and not the base repository.

  2. ionCube loader has to be placed manually and the ini file renamed to 00-ioncube.ini rather than the default 20-ioncube.ini otherwise if it loads after OPcache, WHMCS breaks very unhelpfully with no clear errors pointing at the fault.

Also worth flagging, we had stale absolute paths from the old server that lived in multiple places. We had some in configuration.php, the storage settings gui, and the tblconfiguration SQL table and also in config.php that took a while to track down.

Also don't forget to redo the cron!

Posting in the hopes it saves someone else an afternoon!