Add distribution spec project proposal #35

caniszczyk · 2018-01-23T15:54:37Z

based off initial discussion here: https://docs.google.com/document/d/15y0SBrrDFIEM7pnU-Oe3Y6pq-eTZfo0mk-k33cS2hR4/edit#

Signed-off-by: Chris Aniszczyk caniszczyk@gmail.com

Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>

duglin · 2018-01-23T16:10:28Z

proposals/distribution.md

+
+This proposal covers the distribution API spec, and while it does not cover the code for the docker-registry, that implementation is considered the reference implementation. There are other implementations of this protocol, not all are open-source though (Google gcr.io, Amazon ECR, CoreOS Quay, Gitlab registry, JFrog Artifactory registry, Huawei Dockyard, etc).
+
+In the past when the topic of having an OCI specification around the distribution of container images, it was deferred as "let’s get the image format defined, mean while the industry will settle on a distribution standard". Fast forward, OCI image format is out and adopted, and the Registry v2 is the defacto standard. There is and will be use-cases for alternate methods and the future will likely hold creative ways to push, fetch and share container images, but right now this promotion serves to acknowledge by the OCI the current industry standard of distributing container images.


I think you might be missing a was discussed before the first comma (...of container images was discussed,)

s/mean while/meanwhile/

duglin · 2018-01-23T16:13:18Z

minor typos, but LGTM

wking · 2018-01-23T16:04:54Z

proposals/distribution.md

+* Vincent Batts <vbatts@redhat.com> (@vbatts)
+* Derek McGowan <derek.mcgowan@docker.com> (@dmcgowan)
+
+Additional Maintainers to consider:


The TOB needs to pick an initial maintainer set. I think these suggestions are for the TOB, and should be reviewed and either added to the initial maintainer list or dropped before the vote. A three-person initial maintainer set can add other maintainers on their own, so the TOB can drop tge whole consideration list and punt to the initial maintainers if it wants.

Same.

How are TOB members supposed to interpret this list? Maintainers are picked based on current and past contributions to a codebase or area of expertise and usually their maintainer vote comes with justification of why they should be a maintainer in a project.

@caniszczyk, were we just dropping this list, and leaving it up to the above-three maintainers to add additional maintainers?

wking · 2018-01-23T16:06:37Z

proposals/distribution.md

@@ -0,0 +1,61 @@
+# Abstract
+
+The Docker registry protocol has become the defacto standard across the container registry world ([https://github.com/docker/distribution/blob/master/docs/spec/index.md](https://github.com/docker/distribution/blob/master/docs/spec/index.md)).


Pin this link to a specific version? Or does Docker intend to keep docs in that master after this proposal?

This is actually the wrong link. The specification is at https://github.com/docker/distribution/blob/master/docs/spec/api.md. The auth and token spec are docker-specific extensions and are not required for a full specification.

wking · 2018-01-23T16:14:43Z

proposals/distribution.md

+
+* GitHub for issues and pull requests
+* The dev@opencontainers.org email list
+* The monthly OCI developer community conference call


Does starting at v2 (as suggested below) get us out of the pre-1.0 weekly meeting recommendation? I'm not sure if the "we're starting at v2" approach was considered when writing those docs (I certainly hadn't considered it).

It was discussed in the google doc

hey @wking here you go! If you follow the link at the top, click on "Comments" in the top right, you can explore some of the discussion.

I'm fine with starting at v2. I'm just not clear on whether monthly meetings are the right target level for a new spec (regardless of the number we put on our initial release). The discussion in this PR suggests folks are going to have lots of ideas. Project-template suggests weekly meetings during pre-1.0 development (presumably because the roadmap is less obvious/established then). And there's no reason that roadmap discussions and such couldn't happen asynchronously (but then maybe we want to drop the meeting recommendation from project-template?). Anyhow, I'd just like to make sure that, if this project starts out with monthly meetings, it was a conscious decision that we didn't expect to need weekly meetings.

right target level for a new spec

The specification is not new. It was already discussed in the open and 100+ comments.

@wking sounds like something to discuss on the next monthly meeting :-)

While there is plenty of discussion here, not such an overhaul needed as to change increase from monthly. I am sure we'll adjust the meeting schedule as needed

vsoch · 2018-01-23T16:44:15Z

proposals/distribution.md

+* Jimmy Zelinskie jimmy@coreos.com (@jzelinskie)
+* Liu Genping <[liugenping@huawei.com](mailto:liugenping@huawei.com)>
+* Vanessa Sochat (@vsoch)
+


For completeness, choose any/all that is needed:

affiliation: Stanford / Singularity

email: vsochat@stanford.edu

stevvooe · 2018-01-23T16:50:28Z

This should probably also include two active PRs:

Simplifies tag listing: New Tags API distribution/distribution#2169
Allows listing of manifests: API to retrieve lists of manifest descriptors distribution/distribution#2199

ArangoGutierrez · 2018-01-23T18:02:32Z

+1

Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>

wking · 2018-01-23T20:52:26Z

proposals/distribution.md

@@ -0,0 +1,65 @@
+# Abstract
+
+The Docker registry protocol has become the defacto standard across the container registry world ([https://github.com/docker/distribution/blob/master/docs/spec/api.md](https://github.com/docker/distribution/blob/master/docs/spec/api.md)).


I still think we want to pin this to a specific version. For example:

https://github.com/docker/distribution/blob/5cb406d511b7b9163bff9b6439072e4892e5ae3b/docs/spec/api.md

That should be an implementation detail of the proposal. The implementors of the proposal should choose the version to adopt so this can be changed without TOB involvement.

I think the problem is that the hyperlink will be broken when the spec is moved to the OCI distribution-spec repo.

I think keeping the hyperlink permanently after the migration is good for recording the history.

So +1 for pinning.

And as long as the TOB-selected version isn't far behind (how far will the spec move during a week of voting?), it should be easy for the new maintainets to catch up on any subsequent drift.

we can use such as language when pointing to a particular version...

wking · 2018-01-23T20:56:07Z

This should probably also include two active PRs…

Do you mean “we'll want to land these in docker/distribution before the transfer” or “we'll want to open new PRs post-transfer to continue these open disussions”? If the former, that seems like “we're not ready to raise this proposal for a TOB vote yet”. If the latter, that seems like something the new distribution-API maintainers can sort out amongst themselves without TOB intervention.

Hi I would like to be part of the Maintainers team: affiliation: Sylabs / Singularity email: eduardo@sylabs.io

Join Maintainers

cyphar · 2018-01-24T00:24:47Z

proposals/distribution.md

+
+In the OCI, having a solid, common distribution specification with conformance testing will ensure long lasting security and interoperability throughout the container ecosystem.
+
+## Proposal


One thing I would like to clarify is whether we will actually be making improvements to the spec over time (as per how we did things in the other specifications) or whether we're just going to freeze on the current v2.0 and do nothing much afterwards -- since according to SemVer we'd need to work on v3.0 for that.

Also, it was my understanding that the submission of Docker Distribution as a specification would be something like "any future distribution spec we agree upon will have support for Docker Distribution" -- so as to unblock the distribution problem without locking us into never making significant progress on Distribution. But maybe I misunderstood?

@cyphar I don't think there is anything in this protocol or proposal preventing "significant progress". The specification has been out and in the open since its inception and we have yet to receive any sizable improvement proposals. Most of the perceived limitations of this protocol are limitations in the client implementation that is part of docker.

There are some good changes that we can make in backwards compatible ways:

The listing PRs I mentioned here.

Stateless transfer. There are only a few small changes required to make this work.

p2p already can work on this protocol.

In addition, to this, there is a lot more that can be integrated on top of this protocol as part of client resolution. Most of the issues around naming, signing and mirroring can be taken care of with no API changes.

However, I think it should be clear that the initial specification activity should be about ensuring that the specification matches the current state of the world without breaking compatibility.

I think what would be good to see is some clear instructions for the non-expert in terms of writing and working on specifications. Coming from this kind of boat, I have the general sentiment that I want to contribute, and I want to learn about the process to improve expertise in this sort of contribution. I would look for something akin to a CONTRIBUTING.md, or even a simple bullet list in a README that goes through logical steps. Like:

Goal: to resolve conflict for technical specifications

Procedure:

receiive / link to protocol for discussion

board comments and asks questions as issues

assignment goes to X

issued discussed and resolved, something else...

head maintainer(s) have final sign off on something

publication via... X

I suppose this is a technical standard for reviewing technical standards, haha. For example, a nice parallel is to look at something like JOSS that has complete orchestration via a little robot integration. The workflow is clearly defined for the reviewer and reviewee, and the assigned editor. The parties involved fill in the gaps in terms of opening and resolving issues, but there is always clear definition. I'm not sure how something like this might fit for our group, but it would be great to think about to get greater community involvement and discussion.

The workflow is clearly defined for the reviewer and reviewee, and the assigned editor.

project-template (referenced from this proposal) is what the OCI uses for this role. See especially here and here.

Thank @wking ! The general roles for a contributor and maintainer are well defined, but I'm wondering about a more specific "hand holding" how to contribute sort of document, even just a bullet list. If it's something that can be observed then I can observe, learn, and write something up. I'm mostly thinking about this to help new contributors such as myself to get into the ropes of the group :)

Sure thing! A specific question before closing the discussion here. For this PR, given that we don't have code (nothing to test or critique), and it's a general text draft, what are the criteria we are using for evaluating it, and how do we know when it's ready to go? I think this is likely something that would be obvious for someone working on a lot of these drafts over time, and I'll just catch on, and it would be helpful if someone could jot down a few notes about these questions.

For this PR, given that we don't have code (nothing to test or critique), and it's a general text draft, what are the criteria we are using for evaluating it, and how do we know when it's ready to go?

Ah, this repo. I thought you meant the coming distribution-spec repo. This repo could use a CONTRIBUTING.md, but the voting semantics are covered in the OCI charter (linked from here). At some point a TOB member will think it ready and put it up for a TOB vote. Until then, we're free to suggest improvements.

@stevvooe

Right. Can you expand on these points (we had a short chat a while ago about them, but I don't think I fully understood what the plan was):

Stateless transfer. There are only a few small changes required to make this work.

p2p already can work on this protocol.

Also, I would still like to have some sort of .well-known identifier for registries so that you don't need to host a separate subdomain to have a registry. This would also be a good place to pin future extensions if it turns out that we do want to add something.

Just on this point:

The specification has been out and in the open since its inception and we have yet to receive any sizable improvement proposals.

The reason for this may be unrelated to whether people want to make changes, or have worthwhile improvements. As you mentioned, a lot of the percieved issues with Distribution are actually because the Docker client doesn't expose those features -- and so people who want to improve distribution may not want to go through improving all layers of the Docker stack to do so.

I guess what I'm saying is "let's not close the door on any improvement discussions", especially since this is now going to be an OCI spec and not a Docker one anymore.

@cyphar what sort of language are you looking for to ensure the door is not closed for improvement discussions?

How about after the interoperability statement adding:

This proposal also provides the container ecosystem with a means to discuss and schedule extensions to the distribution specification.

The specification has been out and in the open since its inception and we have yet to receive any sizable improvement proposals.

The reason for this may be unrelated to whether people want to make changes, or have worthwhile improvements.
[...]
I guess what I'm saying is "let's not close the door on any improvement discussions", especially since this is now going to be an OCI spec and not a Docker one anymore.

To add to what @cyphar stated, I think that the pivot from a Docker spec to an OCI spec changes both the set and focus of the contributors; since the spec is no longer being driven by the Docker product's needs it can attract use-cases that would not have made sense for the Docker product.

@mikebrow That language looks fairly reasonable to me.

yuwaMSFT2 · 2018-01-24T08:59:06Z

proposals/distribution.md

+* Does this include the code of the docker-registry?
+    * No. This is an API specification discussion.
+
+## Related GitHub Issues


What about the proposal of new APIs to the spec? Shall we do it after the vote as future incremental improvements, or shall we raise and discuss them now?

From our experience of running our registry service (Azure Container Registry), there are quite some popular feedbacks/lessons we learned and would like to see if they could be included in the future registry spec:

Richer image management ability, including delete repository, purge unreferenced blobs, etc.

Richer discovery ability where user has lots of images, like list tag by creation time, list manifest by creation time, etc. So basically to support more query parameters in the list APIs.

Richer metadata APIs. Basically user wants to put additional metadata with their image, in additional to a simple tag.

Native multi-tenant support. Something like /v2/tenants/{tenant}/repositories/{name}/manifests/{reference}. Since more and more public cloud services are adopting the registry service, a native support for multi-tenant would be great IMHO.

We would like to learn if there is any opportunity that these feedback could be included in the distribution spec. Thanks!

Image size and format would also be very useful, beyond sniffing with a HEAD request. There is also a distinction between a tag (e.g., latest) and a version (a hash or commit), we've had users asking for both with Singularity Hub/Registry, and then for all these requests, the default should be reasonable of course! Are we going with docker defaults like latest for tag, library for namespace, and other registries should follow suit? @yuwaMSFT2 for tenant would that be akin to the manifests list where you can request a particular OS or architecture? The user client would then always be required to make two calls https://github.com/docker/distribution/blob/master/docs/spec/manifest-v2-2.md#manifest-list (and this is what we are doing currently in Singularity)!

@yuwaMSFT2 @vsoch interesting extensions. See above proposal to add a sentence about discussing and scheduling in extensions without going into specific detail.

@vsoch no the metadata of the manifest is not what I meant for tenant.
It's more to make the repository hierarchical. Like on dockerhub you can have organization, then under the organization you can have repository. Current uri would be /v2/orgname/reponame/, and the registry treat the whole path component "/orgname/reponame" as name parameter in the route matching. It will be good if we can separate it.

@mikebrow this looks good to me!
@yuwaMSFT2 this is badly needed, and I strongly +1. Right now we clump the entire thing that comes before the digest (after the @) and tag (after the :) and the image name (e.g., ubuntu) as the namespace. But given registry urls and the potential for local registries to decide to use a custom namespace, parsing this string has been more than challenging). Being able to separate these two things? 🙏 !!

@yuwaMSFT2 A lot of the "extensions" you've referenced are really about content management, rather than content distribution. This API has almost always been about content distribution. Adding these extensions to the specification provides little room for vendors to implement these kinds of functionality in a way that fits their platform.

Right now we clump the entire thing that comes before the digest (after the @) and tag (after the :) and the image name (e.g., ubuntu) as the namespace.

@vsoch None of this is a part of this specification. Even the hub behavior @yuwaMSFT2 was referencing is the hubs implementation. This specification allows arbitrary paths. Other things, like latest, are completely up to the client implementation and the registry doesn't care and this specification doesn't even mention.

Let's make sure this exercise is critiquing the specification and not Docker's implementation of it.

@yuwaMSFT2 agreed about content management vs. distribution, I've never distinguished those two before.

In that the uri is serving as an entry point into a registry, even if it doesn't conform to a specific namespace, tag, arguably I should be able to (knowing the API conforms to the registry) programatically predict the base uri to be calling based on this alone. It could be as general as a general flow of regular expressions to follow, or as specific as a discrete set of different kinds (e.g., abstract uri vs. an arbitrary path). It would be a stronger specification to have some level of predictibility here, otherwise I still need to write a custom thing (after checking!) for each registry endpoint to even interact with most endpoints.

agreed about content management vs. distribution, I've never distinguished those two before.

The two systems have vastly different requirements. For example, even though the Hub looks like its the same system, they are logically separate between the registry and the hub ui, in practice. This prevents the registry deployment from getting complicated. All of the UI and management of images gets implemented in a separate system, allowing them to grow in features and functionality.

I'm still not quite understanding what you're talking about regarding URIs: the format that you're talking about parsing has nothing to do with this specification. Different systems may implement completely different behaviors and formats, depending on their opinion.

@vsoch @yuwaMSFT2 I expanded a bit on the divide between content distribution and management in #37 (comment). I'll repost here, as there may have been some confusion.

This API had everything in it required to integrate with a container runtime (ie. pull an image, figure out what tags are in a repo, etc.) and APIs required to integrate with content management systems. This includes listing repositories and tags. There is also a notification API that enables in the registry implementation, but isn't a part of this specification (there may be an argument about pulling this in).

yuwaMSFT2 · 2018-01-24T09:05:35Z

proposals/distribution.md

+
+TL;DR; Move [https://github.com/docker/distribution/tree/master/docs/spec](https://github.com/docker/distribution/tree/master/docs/spec) to [https://github.com/opencontainers/distribution-spec](https://github.com/opencontainers/distribution-spec)
+
+This proposal covers the distribution API spec, and while it does not cover the code for the docker-registry, that implementation is considered the reference implementation. There are other implementations of this protocol, not all are open-source though (Google gcr.io, Amazon ECR, CoreOS Quay, Gitlab registry, JFrog Artifactory registry, Huawei Dockyard, etc).


Is authentication in the scope of the image distribution?

Also, shall we call out image signing (trusted image)?

Image signing would be very useful for Singularity containers.

@yuwaMSFT2 Unless I'm reading the current spec wrong, authentication is currently included.

Is authentication in the scope of the image distribution?

The authentication for the registry protocol is just http-based authentication. There is a token authentication specification, which may be in scope, but there a lot of ways to actually implement it that may be context-specific.

Image signing would be very useful for Singularity containers.

This is not in scope. All resources are content-addressable and can be signed in external systems. Early versions of the specification and implementation had integrated signing, but there were a lot of problems with it. In practice, the enforcement needs to be in the hands of the client.

This doesn't mean the registry couldn't be used to store signature blobs, but that would require more thought. In practice, we've found the best approach is to decouple this.

For the token authentication - the fact that it is challenging suggests it's even more important to figure out. I would guess that more rather than fewer would want to use token auth, and perhaps the specification can have a default and then fall back cases for a client to implement that are guaranteed to work for most. Without that, we are in a situation of needing a custom implementation for each one, and that defeats the purpose of having a standard. Maybe we could try, and see how far we get? We can always fall back to sticking with just the basic.

And agreed about the signing, given that it's likely very different! It would be good to come back to this at some future point (and as it's more commonly done and discussed).

There is a token authentication specification, which may be in scope, but there a lot of ways to actually implement it that may be context-specific.

I'm in favor of moving over enough of the auth spec to allow clients to authenticate with Docker's Bearer approach without needing to leave the new project's specs. I've filed #37 against this PR with the changes I think we need for that and more detailed motivation.

The issue with this approach is that each auth provider will then have to issue docker-style tokens to interact with the registry. With a more open approach, each registry can choose which providers are integrated. There is also the issue of the access control model: the model used in docker tokens is specific to the way docker implements the registry. Other providers may want to choose to have different token models.

From the perspective of the client, most of this is opaque. They pull an image and authorize the pull through whatever flow makes sense for the context and provide an opaque value for an http header.

wking · 2018-01-24T19:20:06Z

nit: This proposal needs a README entry here.

samuelkarp · 2018-01-26T01:10:42Z

AWS is happy to support the adoption of a specification for distribution and I'm happy to serve as a maintainer.

Since the Docker registry protocol has become a de facto standard across implementations of container registries, it does make sense to start there. However, I think it's important to recognize that there are other OCI users than Docker and that this specification should evolve over time to serve them.

Signed-off-by: W. Trevor King <wking@tremily.us>

Docker's use of Bearer requires information beyond what's covered in RFC 6749 and 6750 [1]. So folks writing a client that will interact with a Docker registry that uses that auth approach will need a "Docker registry's 'Bearer' additions" spec to follow. While I prefer off-the-shelf RFCs for HTTP auth, the Docker registry additions are small enough, and widely used. This change adds the client side of their specification to the new distribution-spec project. The docker/distribution repository also includes docs for scope [3] and the JWT token semantics [4]. The scope docs are borderline useful for clients, but I've left them out because clients can extract the required scope from WWW-Authenticate in 401ed responses: $ curl -IH 'Accept: application/vnd.docker.distribution.manifest.v2+json' https://index.docker.io/v2/library/docker/manifests/1.12.1 HTTP/1.1 401 Unauthorized Content-Type: application/json; charset=utf-8 Docker-Distribution-Api-Version: registry/2.0 Www-Authenticate: Bearer realm="https://auth.docker.io/token",service="registry.docker.io",scope="repository:library/docker:pull" ... Clients can consider them opaque, so I've left them out of the distribution-spec project for now. If distribution-spec maintainers feel that clients could benefit by explicitly crafting their own scope strings, they can pull in the scope specification after the project forms. JWT token semantics [4] are part of the interface between the auth server and the registry. Clients can consider them opaque, so I've left them out of the distribution-spec project. Also pin the docker/registry links to a specific version so the links will survive future docker/registry changes (including removing the docs after the OCI picks them up). As long as the TOB-selected version isn't far behind (how far will the spec move during a week of voting?), it should be easy for the new maintainets to catch up on any subsequent drift. The signing scope language is from Stephen in [5]. The discovery scope language is from Derek [6]. [1]: xiekeyang/oci-discovery#64 (comment) [2]: https://github.com/docker/distribution/blob/5cb406d511b7b9163bff9b6439072e4892e5ae3b/docs/spec/auth/oauth.md [3]: https://github.com/docker/distribution/blob/5cb406d511b7b9163bff9b6439072e4892e5ae3b/docs/spec/auth/scope.md [4]: https://github.com/docker/distribution/blob/5cb406d511b7b9163bff9b6439072e4892e5ae3b/docs/spec/auth/jwt.md [5]: opencontainers#35 (comment) [6]: opencontainers#34 (comment) Signed-off-by: W. Trevor King <wking@tremily.us>

These were added based on Stephen's recommendation [1]. But the new distribution-API maintainers can port those pull-requests over after the project is formed. Or the docker/distribution maintainers can land them before the new project is formed. Either way, there's no need to involve the TOB at this level of detail. [1]: opencontainers#35 (comment) Signed-off-by: W. Trevor King <wking@tremily.us>

dmcgowan · 2018-01-26T22:15:31Z

However, I think it's important to recognize that there are other OCI users than Docker and that this specification should evolve over time to serve them.

I think we are all OK with evolving and making improvements over time. It is going to be a critical role of the maintainers to realize the impact that any change to this specification can bring. Having spent the last 3 years implementing both sides of this specification, I have seen how difficult it can be to both get clients supporting new features and getting registries to fully implement the specification. Inevitably what happens is clients target a subset of registries and registries target a subset of clients. My hope is that we can first address this by moving to OCI, giving the specification the authority and visibility it needs. After that, we can make extensions which have been proven out in the wild, and as a community address the variability of implementations and tendency to implement the lowest common denominator.

vsoch · 2018-01-26T22:27:56Z

@dmcgowan I really like this strategy! And +1+1 for the mental visual of specifications galloping around in the wild! 🏇

stevvooe · 2018-03-06T22:57:44Z

proposals/distribution.md

+
+* “Retrieving image content by its content-addressable hash”.
+    Docker's registry API already provides [endpoints for fetching manifest objects by digest][get-manifest].
+    Docker's registry API does not currently provide endpoints for fetching [image-index][] objects by digest, but this is the project where that will happen.


Missing link here.

Missing link here.

This is the “implicit link name shortcut”, and GitHub renders it fine.

I think you can forgo the [] and just leave it [image-name] and it'll continue to work, as well.

stevvooe · 2018-03-06T22:58:05Z

proposals/distribution.md

+[get-manifest]: https://github.com/docker/distribution/blob/5cb406d511b7b9163bff9b6439072e4892e5ae3b/docs/spec/api.md#pulling-an-image-manifest
+[iana-auth]: http://www.iana.org/assignments/http-authschemes/http-authschemes.xhtml
+[image-spec]: https://github.com/opencontainers/image-spec/
+[image-index]: https://github.com/opencontainers/image-spec/blame/v1.0.1/image-index.md


Any reason for the blame links?

heyo! I think this was explained a bit up here --> #35 (comment) (I had the same question!)

By that explanation this link does not need to be a blame. The other blame link to the same doc has an anchor, but it might be wise to consider not using blame and rather pointing to the parent section so the link is navigable to the sub sections.

By that explanation this link does not need to be a blame.

Good point. I've filed #47 against this PR to fix this link.

The other blame link to the same doc has an anchor, but it might be wise to consider not using blame and rather pointing to the parent section so the link is navigable to the sub sections.

I prefer linking directly to manifests instead of linking to the whole section (~85 lines of Markdown), but if @caniszczyk and/or the TOB prefer section links I'll survive.

jonjohnsonjr · 2018-03-06T23:44:54Z

@stevvooe thanks for the link to Liz's blog post, that's a great resource.

I think we're in violent agreement re: the problems with schema1 👍

We only need blame views when the Markdown does not provide a specific-enough anchor, and this is a link to the whole file. Signed-off-by: W. Trevor King <wking@tremily.us>

Stephen points out that the response code list was missing 403 and that the line is fairly low-level [1]. I agree with Derek [2] that the RFC 7235 reference is sufficient to express the idea, and the less we say, the less likely we are to be wrong ;). [1]: #35 (comment) [2]: #35 (comment) Signed-off-by: W. Trevor King <wking@tremily.us>

stevvooe · 2018-03-07T04:01:43Z

violent agreement

The best kind of agreement! :D

stevvooe · 2018-03-07T04:10:23Z

proposals/distribution.md

+    * In/Out/Future: In scope
+    * Status: In progress (see opencontainers/distribution-spec)
+    * Description: Define a protocol for creating, retrieving, updating, and deleting objects defined in the [image specification][image-spec].
+        Listing repositories (like [`/v2/_catalog`][catalog]) is a multi-[image-index][] action, which is out of scope for this entry.


Again, there is not relation to "multi-image-index" and the catalog endpoint. Please avoid making incorrect statements.

Again, there is not relation to "multi-image-index" and the catalog endpoint. Please avoid making incorrect statements.

How would you prefer to phrase it? Do you agree that /v2/_catalog is a registry-level operation? Do you agree that it may return multiple repositories? Do you agree that repositories may contain multiple images?

I have no clue what is being said here. It literally makes no sense. The specification makes no reference to a "multi-image-index" and we don't have anything called that in the image-spec. It is just made up.

Is this possibly confusion caused by a misunderstanding of ManifestLists?

These are unrelated concepts.

Rereading the sentence, it sounds there's an assumption that the registry implements a 'multi-image index' (e.g. an indexed repository table in a relational database) used to serve the catalog endpoint. This document should avoid making any assumptions about the registry implementation.

Can we rephrase this to say something along the lines of:

Listing repositories is a registry-wide operation; the implementation of which is out of scope of this document

@jzelinskie That sounds great!

I've submitted this change as part of #49.

distribution: Remove IANA auth-scheme sentence

caniszczyk · 2018-03-07T22:57:08Z

On today's OCI dev call we came to consensus to call the project vote on Monday March 12th for the @opencontainers/tob to make a decision. I'll work with @crosbymichael on the voting procedure

http://ircbot.wl.linuxfoundation.org/meetings/opencontainers/2018/opencontainers.2018-03-07-22.00.log.html

caniszczyk · 2018-03-12T15:55:16Z

I kicked off the official project creation vote here for the OCI TOB to approve:

https://groups.google.com/a/opencontainers.org/forum/#!topic/tob/dU6uXM__ilU

cc: @opencontainers/tob

stevvooe · 2018-03-12T22:07:06Z

proposals/distribution.md

+
+## Related GitHub Issues
+
+* Simplifies tag listing: docker/distribution#2169


Let's add a statement here about what these PRs mean. Are these additions required by the proposal? They have been vetted and implemented, so I would think they are a good candidate.

distribution: Change from blame to rendered URI for image-index

Tag listing should mention the endpoints and external method relying on content addressability. Image repositories should not be referred to as indexes, but rather as image repositories. Signed-off-by: Derek McGowan <derek@mcgstyle.net>

Signed-off-by: Stephen J Day <stephen.day@docker.com>

…p-tag-listing Clean up language around tag listing and repository naming

wking · 2018-03-28T19:29:41Z

There's a second run at the vote going on here.

vbatts · 2018-04-02T14:47:23Z

LGTM

caniszczyk · 2018-04-04T22:02:43Z

The vote PASSED:

https://groups.google.com/a/opencontainers.org/forum/#!topic/tob/sJ7_H72M9FM

Catching up with [1]. [1]: opencontainers/tob#35 Signed-off-by: W. Trevor King <wking@tremily.us>

Catching up with [1]. Also mention the new project in GOVERNANCE.md. I'd still rather drop the parenthetical entirely and link to a place that listed OCI Projects, but we don't have a canonical target for that yet (opencontainers/tob#2) and the current closest instance seems to be the GitHub section in [2] (which doesn't have the "OCI Project" words). [1]: opencontainers/tob#35 [2]: https://www.opencontainers.org/community Signed-off-by: W. Trevor King <wking@tremily.us>

This commit redefines the `_catalog` endpoint as an optional operation. Background on the issue: opencontainers#22 https://groups.google.com/a/opencontainers.org/forum/#!topic/dev/rJ72OtZuhbc opencontainers/tob#35 opencontainers/tob#46 opencontainers/tob#50 Signed-off-by: Atlas Kerr <atlaskerr@gmail.com>

Add distribution spec project proposal

1bde248

Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>

caniszczyk mentioned this pull request Jan 23, 2018

Distribution Specification #34

Closed

duglin reviewed Jan 23, 2018

View reviewed changes

wking reviewed Jan 23, 2018

View reviewed changes

vsoch reviewed Jan 23, 2018

View reviewed changes

Make updates after community feedback

c1a5479

Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>

caniszczyk requested a review from vbatts January 23, 2018 20:43

wking reviewed Jan 23, 2018

View reviewed changes

ArangoGutierrez and others added 2 commits January 23, 2018 16:46

Join Maintainers

6dc227c

Hi I would like to be part of the Maintainers team: affiliation: Sylabs / Singularity email: eduardo@sylabs.io

Merge pull request #36 from ArangoGutierrez/patch-2

e2fc9af

Join Maintainers

cyphar reviewed Jan 24, 2018

View reviewed changes

yuwaMSFT2 reviewed Jan 24, 2018

View reviewed changes

vsoch mentioned this pull request Jan 25, 2018

Support for Singularity compliance with the OCI runtime apptainer/singularity#1235

Closed

README: Link to the distribution proposal

d20c56d

Signed-off-by: W. Trevor King <wking@tremily.us>

This was referenced Jan 26, 2018

README: Link to the distribution proposal #38

Merged

distribution: Add in-scope and out-of-scope wording #37

Merged

wking mentioned this pull request Jan 26, 2018

distribution: Drop "Related GitHub Issues" section #39

Closed

stevvooe reviewed Mar 6, 2018

View reviewed changes

distribution: Change from blame to rendered URI for image-index

0c2ad77

We only need blame views when the Markdown does not provide a specific-enough anchor, and this is a link to the whole file. Signed-off-by: W. Trevor King <wking@tremily.us>

wking mentioned this pull request Mar 6, 2018

distribution: Change from blame to rendered URI for image-index #47

Merged

wking mentioned this pull request Mar 7, 2018

distribution: Remove IANA auth-scheme sentence #48

Merged

stevvooe reviewed Mar 7, 2018

View reviewed changes

Merge pull request #48 from wking/remove-iana-auth-scheme-reference

0cd79c9

distribution: Remove IANA auth-scheme sentence

stevvooe reviewed Mar 12, 2018

View reviewed changes

Merge pull request #47 from wking/distribution-image-index-blame

e07b90e

distribution: Change from blame to rendered URI for image-index

jzelinskie mentioned this pull request Mar 15, 2018

More robust authentication helm/chartmuseum#59

Closed

dmcgowan and others added 3 commits March 20, 2018 17:03

proposals/distribution: remove non-sense about image indexes

2b4308e

Signed-off-by: Stephen J Day <stephen.day@docker.com>

Merge pull request #50 from dmcgowan/add-distribution-proposal-cleanu…

b6ec853

…p-tag-listing Clean up language around tag listing and repository naming

caniszczyk merged commit 3619df2 into master Apr 4, 2018

caniszczyk deleted the add-distribution-proposal branch April 4, 2018 22:02

wking added a commit to wking/oci-project-template that referenced this pull request Apr 4, 2018

.pullapprove.yml: Add distribution-spec-maintainers

cbfea6d

Catching up with [1]. [1]: opencontainers/tob#35 Signed-off-by: W. Trevor King <wking@tremily.us>

wking mentioned this pull request Apr 4, 2018

.pullapprove.yml: Add distribution-spec-maintainers opencontainers/project-template#53

Open

bsatlas mentioned this pull request Jan 4, 2019

spec: catalog operations optional. opencontainers/distribution-spec#40

Merged

caniszczyk mentioned this pull request Jun 10, 2021

OCI artifact manifest, Phase 1-Reference Types opencontainers/artifacts#29

Closed


		This proposal covers the distribution API spec, and while it does not cover the code for the docker-registry, that implementation is considered the reference implementation. There are other implementations of this protocol, not all are open-source though (Google gcr.io, Amazon ECR, CoreOS Quay, Gitlab registry, JFrog Artifactory registry, Huawei Dockyard, etc).

		In the past when the topic of having an OCI specification around the distribution of container images, it was deferred as "let’s get the image format defined, mean while the industry will settle on a distribution standard". Fast forward, OCI image format is out and adopted, and the Registry v2 is the defacto standard. There is and will be use-cases for alternate methods and the future will likely hold creative ways to push, fetch and share container images, but right now this promotion serves to acknowledge by the OCI the current industry standard of distributing container images.

		@@ -0,0 +1,61 @@
		# Abstract

		The Docker registry protocol has become the defacto standard across the container registry world ([https://github.com/docker/distribution/blob/master/docs/spec/index.md](https://github.com/docker/distribution/blob/master/docs/spec/index.md)).

		@@ -0,0 +1,65 @@
		# Abstract

		The Docker registry protocol has become the defacto standard across the container registry world ([https://github.com/docker/distribution/blob/master/docs/spec/api.md](https://github.com/docker/distribution/blob/master/docs/spec/api.md)).


		In the OCI, having a solid, common distribution specification with conformance testing will ensure long lasting security and interoperability throughout the container ecosystem.

		## Proposal


		TL;DR; Move [https://github.com/docker/distribution/tree/master/docs/spec](https://github.com/docker/distribution/tree/master/docs/spec) to [https://github.com/opencontainers/distribution-spec](https://github.com/opencontainers/distribution-spec)

		This proposal covers the distribution API spec, and while it does not cover the code for the docker-registry, that implementation is considered the reference implementation. There are other implementations of this protocol, not all are open-source though (Google gcr.io, Amazon ECR, CoreOS Quay, Gitlab registry, JFrog Artifactory registry, Huawei Dockyard, etc).


		## Related GitHub Issues

		* Simplifies tag listing: docker/distribution#2169

Add distribution spec project proposal #35

Add distribution spec project proposal #35

Conversation

caniszczyk commented Jan 23, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

duglin commented Jan 23, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stevvooe commented Jan 23, 2018

ArangoGutierrez commented Jan 23, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wking commented Jan 23, 2018

cyphar Jan 24, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wking Jan 24, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mikebrow Jan 24, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wking commented Jan 24, 2018

samuelkarp commented Jan 26, 2018

dmcgowan commented Jan 26, 2018

vsoch commented Jan 26, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jonjohnsonjr commented Mar 6, 2018

stevvooe commented Mar 7, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jzelinskie Mar 13, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cyphar Jan 24, 2018 •

edited

Loading

wking Jan 24, 2018 •

edited

Loading

mikebrow Jan 24, 2018 •

edited

Loading

jzelinskie Mar 13, 2018 •

edited

Loading