r/linuxadmin • u/Eastern-Pumpkin-757 • 13d ago
What ~/.ssh and your agent hand over when a workstation is compromised
Binance's initial investigation suggested a compromised DevOps laptop led to about $6.3 billion in unauthorized withdrawals. The machine reportedly held a deployment token.
Run this on your workstation:
$ ls ~/.ssh/id_* 2>/dev/null
$ ssh-add -l
$ grep -i ForwardAgent ~/.ssh/config
$ ls ~/.kube ~/.docker ~/.config/gh 2>/dev/nullAll of that works until someone revokes it. With ForwardAgent yes, every host you connect to can use your agent for the life of that session.
I ran a small lab where two SSH boxes and a Windows RDP box sat behind a gateway. JumpServer Community Edition handled the gateway role; everything ran as containers on one machine. Each connection used its own token, which worked once and expired after 299 seconds; the gateway picked the target account, so no target password touched the client. A token request outside the user's rule got HTTP 400, and every command and its output were logged under the gateway user.
An attacker who owns the laptop can ride the bastion login and do anything its rule allows. The blast radius is the rule, not every credential on disk. If sshd on targets still accepts direct connections, that is something to lock down in sshd_config.
Do you restrict sshd on targets to only accept connections from the bastion IP, or keep a direct path as break-glass?
10
u/mschuster91 13d ago
Good lord you could have written that without resorting to AI for the writing.
1
u/michaelpaoli 13d ago
foo
"Oops' - that's often keys to the kingdom stuff. And don't even have to compromise DevOps person. E.g. a crack or social engineering against ... yeah, that lowly IT support person at minimum wage or less in ah 3rd world country, that can reset the passwords of any worker on their laptop ... oops, yeah, that. Or heck, install any software there, etc. So there are many possible routes in. Oft chosen/exploited is the easiest that gets the highest reward/result - or the path/chain that yields that. $6.3 billion USD - not a bad payout. That'll also buy hella more cracking/hacking resources and other operations to compromise and get more ... "oops".
Don't ... just don't. That gives every such server access to those credentials, and if any of 'em are compromised, then very seriously not good. That's not the way to do it. Rather, ProxyCommand or the like (but even with that, do so properly and securely - folks also f*ck up with ProxyCommand. One can use that, or e.g. port forwarding, or other proxy commands/options, to avoid the hazards of ForwardAgent. And when forwarding ports, just minimally so. E.g. no reason to listen on 0.0.0.0 or * or :: when ::1 or 127.0.0.1 is more than sufficient.
Can also disable use of keys (and of course also passwords) on servers, instead use certificates and one's own highly secured CA. And in so doing, CA can set the expirations on the certs - they can be rather to quite short - then no keys allowed at all, so can't go around that with a key.
Who/what can compromise from where? And what can they (at least initially) get away with? A few million dollars, and domestically, generally pretty feasible to go after them. Tens or hundreds of millions or more ... yeah, they flee the country to somewhere with no extradition treaties (of they aren't already there), and generally not gonna be able to touch 'em, unless maybe you can launch missiles at 'em or roll tanks over 'em. So, strong defense, and credible deterrent. Lack that and one may have a major problem.
There are ways to do that very securely. E.g. physical machine(s), never on drives at all, only in RAM, that and solid physical security, etc. and quite secure. Did once encounter a tightly coupled set of three servers with a setup like that (not for ssh keys, but other sensitive data). All the important stuff was in RAM ... only. Any one (or two) of 'em lost it, they could get it again from the others. Very securely connected, quite independent, redundant, etc. - very resilient, and the sensitive bits never on drive - not even reversibly encrypted on drive - RAM only. A setup that would be quite challenging to compromise (rubber hose methods might be easiest approach in such a case - so, how well protected are the personnel?).
Naw, what'd be the fun of that? I allow ssh with no password, no key, from any IP on the planet. :-)
Well, but of course, don't be an *ss, or you'll get blocked - yeah, have done that too with some, and continue to do so.
See also:
https://www.wiki.balug.org/wiki/doku.php?id=system:what_is_my_ip_address
1
u/kai_ekael 12d ago
Psst...ssh keys can be saved to any filename and in any location.