private registries

[hellt]

Currently clabernetes is doing fine when images used in manifests point to:

1. Public images
2. Private images contained with the private registry that can be reachable from the cluster

The problem is that most VM-based images users build with clab are local. Think of SR OS image that is privately stored in local docker image store of a user.

Clearly, when people are adventurous enough to explore k8s waters they might as well invest in running their private harbor or any other registry, but we need to make this easy for those who can't do it on their own.

I was thinking we need to brainstorm how to support c9s-registry pod inside k8s cluster that would store the user' images and as such make them available to c9s/clabverter.

It seems we can automate creation of a docker-registry pod for the cluster in question, this registry can have dummy user/pwd pair that users could use to push images they want to use in their labs.

There are many tutorials how to add registry pod to a cluster, this one looks ok.

The registry can be ephemeral to start with (no pv/pvc issues), and then it seems we need traefik to expose it so that users could push images to it - see this.

It is an open question how we automate this, maybe for starters it can be just a bash script for us to use in the first place.

The workflow is then:

1. deploy registry
2. push images you want to use in a lab
3. deploy Clabernetes CR

Ideally all of this can be handled by clabverter, offering a truly turnkey experience. But this is for laaaater.

[hellt]

Option 1 - Pulumi/TF based installation

It seems reasonable to have a repo that would show how to get the registry installed with some trivial pulumi/tf code (and maybe a pinch of ansible)

Then the traefik ingress manifest would expose this outside the cluster for people to manage the registry.

[jbemmel]

Related to this, the documentation says The only difference is that you need to specify the image in the topology file as a fully qualified image name, including the registry name

Perhaps it would make sense to have an option in clabverter to specify a default registry to use, when the image URL in the topology doesn't include one?

[jbemmel]

For me, conceptually this is a deployment choice and not a part of the topology. I appreciate that we wouldn't want to complicate clabverter any more than necessary - but telling the system where it should pull its images from belongs there, imho

I mean: I'd want to point clabverter to a topology in github, and say "deploy that topology using my local registry over here to pull images from"

[carlmontanari]

wouldn't it apply for clab normal too though? like image is just "srlinux:latest" and "ceos:latest" then the default registry could be "tacocat.com" and then clab knows to pull "taccat.com/srlinux:latest" or something?

ah ok, I see your edit now. yeah, I totally hear what you're saying and it makes sense. just I also feel (possibly incorrectly?) it could be useful for clab too, and if that is true then just doin it in clab seems better (and less work for me :p)

[hellt]

I feel that whenever a user is "serious" about their c9s game they tune the topology to the deployment environment; meaning setting the fqdn of the images as they should be set.

The bridge between clab <-> c9s is a bridge for newcomers, someone who never authored a CR in their life and want to try this out. But whenever c9s becomes a deployment model of choice, then the topology should veer more into c9s way of doing things.

If anything, the topology may use env vars (supported by clab today), e.g:

kinds:
  nokia_sros:
    image: ${REGISTRY:somelocal.reg.io}/sros:24.3.1

[carlmontanari]

If anything, the topology may use env vars (supported by clab today), e.g:

lol damn, now I gotta add custom env support for launchers 🤣 (though this is probably overdue and should be done anyway if I didnt already!)

[jbemmel]

Yes I was going to suggest env variables (clab/bash style) anyway. That will work for both clab proper and clabverter

There are more places where this will come in handy - but the repo URL is definitely one