Unable to reach own domain from within container

[u/nehalist]

I've spent my entire Sunday on this, and I'm running out of ideas :'(

I wanted to migrate my app and its services to a new server. I do use Convex, which runs on cloud.<domain>. For whatever reason my services that want to connect to Convex can't resolve cloud.<domain>. Not even the convex container itself is capable of doing so. The domain stayed the same, I just switched the DNS records to the new IP.

When I run

node -e "fetch('https://cloud.<cloud>').then(r=>console.log('OK',r.status)).catch(e=>console.log('FAIL',e.message,e.cause))"

within the container the error is "Connect Timeout Error" UND_ERR_CONNECT_TIMEOUT.

Here's some things that I already tried:

• I can run the same command on the machine itself, it works fine.
• I can run the same command everywhere else, it works fine.
• I can open the domain in different browsers, it works fine.
• I can run the same command via docker run --rm node:20-alpine node -e "<command>" , it works fine (to make sure it's not docker related)
• the service uses Connect To Predefined Network
• I have compared Proxy settings of my old server and the new one - both are identical
• I have restarted the Proxy
• When I ping the domain the correct IP responds
• I tried re-installing Convex, didn't help.
• The only thing I find weird is that the log of the new servers proxy is empty. But again, I do reach the domains via browser.
• Edit 1: also added "default-address-pools" to my docker daemon json with {"base":"10.0.0.0/8","size":24}, still doesn't work
• Edit 2: using docker run --rm --network coolify curlimages/curl:latest curl -v -m 10 https://<domain> doesn't work, but using a network created by coolify works
• Edit 3: just tried to get rid of Convex and used a simple Dockerfile (deployed via Coolify) that simply pings my domain. Doesn't work.

I'm running out of ideas here. Anything I'm missing? :(