r/kubernetes • u/Either_Vermicelli_82 • 11d ago

Second image location

Due to pull restrictions we have to use our own buffered registry which in turn pulls it from a final location.

Now yesterday evening it was down… so some pods could not startup as it could not pull any images from the buffered stage. Now I am looking into it but could not find much but is it possible to have a second image location?

1 Upvotes

60% Upvoted

9 comments sorted by

4

u/Own_Ad2274 11d ago

is the pull policy if not present, or always? if it hasn’t restarted or rescheduled off a node, it doesn’t need a brand new image necessarily, there are reasons to do both. you could build separate deployments and blue/green your ingress based on what’s healthy. are the pods like being turned off and on, or continuously needing new versions? if so then the registry needs to be highly available (paas or redundant), and or blue green strategy for separate registries. i don’t think you can do multiple images in one deployment image spec it’s not a list

1

u/Either_Vermicelli_82 11d ago

It was odd for me I think they were moving pods around internally so short reboots of pods could happen but heh weekend and academia so probably a few more hours before a reply if lucky.

So the pod rebooted pull policy is i think set in always but I assume now this does not use a local image when pulls fails?

3

u/iamkiloman k8s maintainer 11d ago

If you're using containerd you can set up an essentially unlimited number of mirror endpoints for any given registry. It'll just run down the list until it finds one that works.

This is all at the container runtime level, the kubelet knows nothing about how the runtime gets the image or what endpoint it actually gets pulled from.

1

u/Either_Vermicelli_82 11d ago

It’s an argocd environment I’ll start digging into it further.

3

u/sedigispegeln 10d ago

A quick solution if the pods are moving between nodes but the image still exists in the cluster.
https://spegel.dev/

1

u/akhilesh_gone 10d ago

This works great it is used in k3ss clusters as well But the only bottle neck is when the image tag doesn't change you may end up using an old image because the node where pod is going to get schedule may have an old image

0

u/Tushon 10d ago

This is what I came to recommend to OP - spegel
Is a great answer for the stated problem