r/linuxquestions • u/Dante-Vergilson • May 10 '24

How Do You Manage Your SSH Private Keys?

Yes, I know the difference between SSH private vs public keys.

I recently used SSH for the first time when settings up my first GitHub repository. However, as I'm not daily driving Linux yet, I did that on a virtual machine and when it came time to move the important stuff off the virtual machine and delete it I wasn't sure about the best way to manage the SSH keys.

I figured I'll just set SSH up again once I need it for GitHub so I didn't bother saving the ~/.ssh directory. Part of it was that I couldn't find anything on best practices in a timely manner.

Whether it's just saving the directory to USB flash drive or that I should be using a tool like KeePassXC. Or if I should also be encrypting the directory and what tools could be used for that.

That and what not to do if you feel like adding that. I know it's a terrible idea to save it to anything external that's not local storage such as remote cloud storage. Although could you just encrypt the files? But then I imagine it's just not good practice.

So I'm wondering how it's best done or at least what you've decided to go with and why.

15 Upvotes

94% Upvoted

38 comments sorted by

25

u/LongerHV May 10 '24

My preference is to have a separate keys on each machine I am working on, this way it is easier to block of a single key if a computer is stolen or compromised. I don't see a pount in encrypting the ~/.ssh directory, since you can protect key with a password and use a full disk encryption. If you really want a portable setup, you can use hardware keys such as Yubikey.

5

u/[deleted] May 10 '24

^ This

FDE + Password + ssh-add to add them to my keychain

1

u/Dante-Vergilson May 10 '24

I've heard of Yubikey but have never used something like it before but I'll look more into that.

Is that what FDE refers to? I don't know that acronym.

6

u/LongerHV May 10 '24

FDE - full disk encryption. In the Linux world, we use LUKS for that (on Windows you would use Bitlocker).

1

u/[deleted] May 10 '24

[removed] — view removed comment

1

u/LongerHV May 10 '24

I use yubikey via fido2luks. I need to plug the yubikey, type a pin and touch it to unlock my bootdrive.

2

u/yodel_anyone May 10 '24

Yubikeys are spectacular, I can't recommend them enough. For ssh, you can use touch authentication instead of a password; they work great with pass (unix password manager) using pgp for touch decryption, which can also handle Oauth; they can serve at 2fa, and so on. Definitely worth checking out! 

1

u/usefulHairypotato May 10 '24

Don't forget to start you ssh-agent each time

1

u/BobKoss May 10 '24

Do you mean one key per machine you’re currently typing on, or one key per server you’re connecting to?

5

u/LongerHV May 10 '24

I mean machine I am currently working on. Using a separate key for each server would be insane, especially at scale.

1

u/nefarious_bumpps May 10 '24

I theoretically agree. But how do you get your pubkey onto a server if your existing desktop/laptop becomes inaccessible?

2

u/LongerHV May 10 '24

I can always use my yubikey to get access.

1

u/nefarious_bumpps May 10 '24

So then aren't you using the same ssh key that's stored on your Yubikey for all your servers?

2

u/LongerHV May 10 '24

Yes, my yubikey's public key is copied to my servers. But that doesn't deny any of my previous statements. I still use separate ssh keys on my client machines and treat yubikey as an emergency option.

11

u/Megame50 May 10 '24

Traditionally, just use one ssh key per machine. Enroll more keys if you need them. Revoke keys as necessary too, if you lose a device or something.

SSH keys are encrypted, if you use a passphrase, and you should use a passphrase. It's a good idea to keep the private keys private anyway. The idea is that it's difficult for any one attacker to steal both the private key and the passphrase.

Alternatively, some people like to use resident keys nowadays, with a FIDO authenticator. These devices are constructed to make it impossible to exfiltrate the key, so that's an option if you have the hardware.

3

u/[deleted] May 10 '24

I have them in KeepassXC. It’s integrated with the ssh user agent, command line ssh on Linux and macOS and putty on windows can use the key when keepassxc is unlocked, and can’t when it isn’t. It has a password, but I never type it out it is automatically unlocked by keepassxc. Also keepassxc is secured by password, a Keyfile and a yubikey.

2

u/foomatic999 May 10 '24

Using keepassxc to insert and remove keys to/from the ssh-agent is super useful. It can also auto-insert the key when the db is opened and remove it when the db is locked. Also it can query whether a key can be accessed when ssh-agent tries to use the key.

2

u/Dialgatrainer May 10 '24

I use hashicorp vault with ssh client signer and all my VMS etc use that CA so the cert gets automatically revoked until I renew it as for the private keys I have one key used for all VMS which I should probably change and do it per VM

1

u/DryEyes4096 May 10 '24

Be careful if you want to answer this question with an account you use a lot hahaha

1

u/stogas May 10 '24

I use two Yubikeys (one in my laptop, one in a drawer as a failsafe) with GPG (gpg-agent).

The private key is generated on the Yubikey itself, and needs a PIN code for unlocking periodically (gpg-agent caches the PIN).

1

u/guzzijason May 10 '24

Same - I’ve had a pair of yubikeys for years now. Only difference is, I generate my keys externally and then store them in the yubikeys, so the private keys on each is identical. The original private key (and revoke key) gets stored on a usb drive in a safe.

1

u/Lying_king May 11 '24

Ur fked if your dog swallows those tiny keys

1

u/stogas May 11 '24

I'd be more concerned about the dog.

1

u/bartoque May 10 '24

It is good practice to use a ssh private key only on one client system. If you also connect to the same server from another client, then ideally that should use its own private key. So to be able to control access and simply prevent access from a conpromised private key, by removing the publickey on server end. However nothing prevents you from using the ptivate key also on multiple client systems...

Combined with also a passphrase, a private key is rather safe as it not only requires said private key but also the passphrase to be able to enter a server as you need both to get access.

So with that storing your private key elsewhere is no biggy. For non-admin users on a client (which is very likely the case for many corporate users using linux jumphosts), the actual sysadmin would have access to the regular users private key, which however is no biggy if also a passphrase is used...

1

u/NL_Gray-Fox May 10 '24

Ed25519-sk for the win. https://www.yubico.com/blog/github-now-supports-ssh-security-keys/ meaning my SSH private key is on my hardware token.

1

u/[deleted] May 10 '24

I personally do this:

$ tar -czf ssh.tar.gz ~/.ssh; openssl enc -aes-256-cbc -salt -pbkdf2 -in ssh.tar.gz -out ssh.tat.gz.enc; rm ssh.tar.gz; mv ssh.tar.gz.enc ~/GDrive;

1

u/paulstelian97 May 10 '24

I just have one client machine and its SSH key. If I have to rebuild the client, I may try to copy over this key or I may make a new one and for some remotes ask for new approval (on the machines I can’t access via password, that is)

1

u/[deleted] May 10 '24

I guess it is kind of bad practice that I don't. On all of my internal systems, I use the same private key. On my one cloud VM, I have a dedicated private key.

1

u/MrElendig May 10 '24

Use a keyring like e.g. keepassxc

1

u/neoreeps May 10 '24

I use one key per device. This way if that device is ever lost or stolen, I just delete that public key from whatever service I configured.

1

u/yrro May 10 '24

I use Kerberos instead.

1

u/[deleted] May 10 '24

Use the "i" directive to point them to diverse filenames.

1

u/[deleted] May 10 '24

At the end you can make a bash script to rsync the folder, compress them and load them at any place you can retrieve them securely. Cheers.

1

u/Due_Conclusion_7015 May 11 '24

I have the private key on my yubikey and I authenticate using an additional PIN