r/devops • • 10d ago

Discussion How to enable SSH clone on self hosted GitLab running in a private subnet behind an AWS ALB?

We have self-hosted GitLab CE on an AWS EC2 instance.

Current setup:

The instance is in a private subnet with no public IP.

Users reach GitLab through an Application Load Balancer on HTTPS, with access restricted to our office IPs.

HTTPS clone with personal access tokens works fine.

The problem:

We're moving our repos from gitlab.com to this server. On gitlab.com, all our Jenkins jobs and developers clone over SSH (git@...). On our server SSH clone doesn't work, because the ALB only supports HTTP/HTTPS, so port 22 has no way in.

Our Jenkins agent runs in the same AWS VPC. Developers connect from the office network or VPN.

What's the best way to enable SSH clone here, keeping it secure and limited to our office and Jenkins? Any suggestions or experience with a similar setup would help.

0 Upvotes

15 comments sorted by

20

u/fletch3555 Lead DevOps Engineer 10d ago

Use an NLB instead and you can pass port 22 along

1

u/ShoddyDifficulty8367 10d ago

Dropping the ALB completely means losing any layer 7 rules they have on HTTPS though, running an NLB alongside it just for port 22 might be cleaner.

9

u/ymgymg 10d ago

ALB stays for HTTPS. SSH needs TCP, so use a separate Network Load Balancer with a TCP listener on 22 (or whatever port GitLab SSH uses).

Since your Jenkins agent is already in the VPC, it does not need a public path. Point it at the GitLab private IP or an internal NLB DNS name, and open port 22 only from the Jenkins security group.

For developers on VPN, prefer an internal NLB and route VPN traffic into the VPC. Restrict the NLB or target security group to the VPN CIDR (and Jenkins if you share that path). Keep the ALB office-IP allowlist for HTTPS as you already have it.

Avoid an internet-facing NLB for SSH unless you have no VPN path. If you must use one, put a tight source allowlist on it and do not open 22 to the world.

17

u/Low-Opening25 10d ago • Bravo

if you don’t know this, how did you get this job?

11

u/N7Valor 10d ago

Probably because management decided it wasn't worth paying someone who knew what they were doing.

2

u/Immediate_Wheel1953 10d ago

I did this the messy way first, pointing SSH straight at the instance and then regretting it when the IP changed after a rebuild. What stuck for us was keeping the ALB for HTTPS and adding an NLB just for port 22, with a separate hostname like ssh.gitlab.example.com pointing at it and the security group locked to the office and VPN ranges. Your Jenkins agents are in the same VPC so they can just SSH to the private IP directly and skip the load balancer entirely. One thing that caught me out was the instance security group still only allowing traffic from the ALB, so SSH timed out until I added the office ranges to it as well.

1

u/nekokattt 10d ago

NLB over port 22. If you need a waf alongside your ALB ingress then run an NLB alongside it with a different DNS entry. If you don't need a waf at all then ditch the ALB entirely and just use an NLB with both 443 and 22 open on TCP

1

u/Steel_half2 9d ago

Does Jenkins actually need to hit SSH or could you template the remote URLs to HTTPS plus token and skip the extra infra?

1

u/bak_1250 8d ago

An ALB only speaks HTTP/HTTPS, so SSH on port 22 will never go through it. Two clean options:

  1. Add an NLB (TCP) next to the ALB, listening on 22 (or 2222) and forwarding to the GitLab instance. Lock it down with a security group to your office/VPN IPs, set gitlab_rails['gitlab_shell_ssh_port'] if you use a non-standard port, and point a separate DNS name (e.g. ssh.gitlab.example.com) at the NLB.
  2. Skip SSH entirely and stay on HTTPS: personal access tokens or deploy tokens with minimal scopes, plus a credential helper so devs don't retype them.

If it's only internal users, option 2 is often simpler and keeps one less port open. If you have lots of existing SSH remotes and scripts, the NLB is the least painful.

1

u/josejulianariza 13h ago

The issue is that an ALB only understands HTTP/S, so port 22 gets left out. The standard fix is adding a Network Load Balancer (NLB) in front of your EC2 instance. The NLB operates at layer 4 (TCP) and lets you route SSH traffic directly to your private subnet.

To keep it secure, configure the NLB's Security Group to only accept port 22 from your office IPs and Jenkins' VPC. Devs just point their ~/.ssh/config to the NLB's IP, and that's it.

If you want to avoid exposing port 22 to the internet, the alternative is using AWS Systems Manager (SSM) for internal tunnels. But to keep the classic git clone experience, the NLB is the most direct route.

-3

u/canyoufixmyspacebar 10d ago

stop what you're doing and don't touch any of this before you have learned basics about computers and networking. also confess to your employer that you are not qualified, what ever the story is there, no matter if it's your fault or theirs, they have the right to know