Specify semantics of resource creation via PUT

[acoburn]

The PUT method can be used to replace or create resources. This is explicitly allowable under both LDP and HTTP.

What does the Solid specification say about PUT to create resources (MAY, SHOULD or MUST)?

Specifically, if a resource is created via PUT, what is the expectation about how this operation interacts with LDP containment?

For example, if a client creates a resource via PUT at /foo/bar/baz, should an ldp:contains triple be added to the container /foo/bar?

What if /foo/bar does not exist? Should the server create the container automatically or should the server reject the operation?

What if /foo/bar is a non-container resource (ldp:RDFSource or ldp:NonRDFSource)?

[csarven]

Just to eliminate any ambiguity, by "create resources", we consider all ldp Resources; (Non)RDFSources as well as Containers:

What does the Solid specification say about PUT to create resources (MAY, SHOULD or MUST)?

The thing is that LDP doesn't recommend containment triples to be updated through PUT ( https://www.w3.org/TR/ldp/#ldpc-put-mbrprops ):

5.2.4.1 LDP servers SHOULD NOT allow HTTP PUT to update a LDPC’s containment triples; if the server receives such a request, it SHOULD respond with a 409 (Conflict) status code.

The reason I'm mentioning is that, if PUT is used, Solid needs to make a decision on whether when a resource is created, it should update the containment triples right there and then, or not. Otherwise, it has to go against the stream (re "SHOULD NOT"). Putting aside the case where the SHOULD NOT recommendation is ignored, PUT without updating containment triples would entail that the created resources won't be discoverable through the container.

Aside: that doesn't imply that created resource is not discoverable through a property other than ldp:contains, most likely through membership properties.

It is kind of a one way street. I would suggest that to keep PUT as a MAY (as LDP), but add that server if PUT is used, server MUST update containment triples. Otherwise, containment triples are confined to POST. I'm not sure about any downsides to not updating the containment triples other than the challenge of determining the right container.

I suppose that non-overly LDP-centric clients can still operate in PUT mode without having to rely on POST or POST w/ Slug, if they are authorised and the server is willing any way.

AFAIK, LDP implementations generally do support PUT, however I don't know what they do with regards to managing containment triples.

The fundamental issue is that PUT has no context about where it belongs with regards to containers. Either additional hacking is required (eg URI parsing), default to root container as its container, or the server finds the closest container. I don't think the client should mess around with URIs and assume what the server actually claims the URI to be. (Not to mention being subject to potential differences with URI's HTTP header and/or body).

For example, if a client creates a resource via PUT at /foo/bar/baz, should an ldp:contains triple be added to the container /foo/bar?

I find this to be difficult to answer right now because it depends on resolving #35 . Provided that if 1) PUT changes containment triples (as mentioned above) and 2) /foo/bar/ is a container, then yes. If 1) but not 2) (or unknown), I suppose "/ contains /foo/bar/baz"? What I think may be reasonable to expect from clients is that, they identify the container in which they want to PUT a resource into. That eliminates relying on a URI pattern or parsing URI strings. However!!! The server doesn't explicitly know where it should be contained.. other than itself finding the most appropriate spot, possibly checking the URI pattern or throwing it under the root container.

What if /foo/bar does not exist? Should the server create the container automatically or should the server reject the operation?

The server should keep the number of operations to minimum with respect to creating new containers. Answering if/how containment is done, this may be easier to answer provided that we know the closest container.

Aside, FWIW:

$ touch /tmp/foo/bar/baz
touch: cannot touch '/tmp/foo/bar/baz': No such file or directory

What if /foo/bar is a non-container resource (ldp:RDFSource or ldp:NonRDFSource)?

Same as above. Server probably needs to find the closest or most meaningful container and link to the resource from there. However, unpopular opinion but keeping URIs opaque: if /foo/bar is a (Non)RDFSource resource, /foo/bar/baz can be a (Non)RDFSource ie. not caring about it being associated to a filesytem. I realise the obvious advantages to keeping it close to filesystem's directory/file relationship. I'm not strongly opposed to that,.. just sayin'.

More discussion and implementation experience (on different code from different implementers) would be useful here.

If PUT is supported and assuming that its is desirable to have the interaction model for POST and PUT aligned, then clients intending to create an LDPR as well as subsequent requests to it with PUT should be honoured the same way by servers as that of POST: https://www.w3.org/TR/ldp/#ldpc-post-createrdf .

Edit: Mentioned PUT LDPR expectation.

[pmcb55]

Just my initial thoughts (mainly due to 'keeping the server dumb'):

What does the Solid specification say about PUT to create resources (MAY, SHOULD or MUST)?

Like LDP, optional (so MAY).

Specifically, if a resource is created via PUT, what is the expectation about how this operation interacts with LDP containment?
For example, if a client creates a resource via PUT at /foo/bar/baz, should an ldp:contains triple be added to the container /foo/bar?

If /foo/bar is already explicitly an ldp:Container then yes, otherwise, no.

What if /foo/bar does not exist? Should the server create the container automatically or should the server reject the operation?

Server MUST reject the operation. (Client should explicitly create '/foo/bar' first as an 'ldp:Container'.)

What if /foo/bar is a non-container resource (ldp:RDFSource or ldp:NonRDFSource)?

Server MUST reject the operation. (Client should update '/foo/bar' first (via PATCH or PUT) to try and explicitly make it an 'ldp:Container' first.)

[dmitrizagidulin]

Question:

What if /foo/bar does not exist? Should the server create the container automatically or should the server reject the operation?

@pmcb55:

Server MUST reject the operation. (Client should explicitly create '/foo/bar' first as an 'ldp:Container'.)

That seems like a reasonable policy. I do want to point out, though, that the current behavior (and I believe spec) for both PATCH and POST in Solid is to follow the mkdir -p semantics (to automatically create the intermediate containers).
So, at very least, I'd like whatever policy we settle on to be consistent, among PUT, POST and PATCH.

(Personally, I vote for the mkdir -p behavior.)

[dmitrizagidulin]

@acoburn - I hope you don't mind, I changed the title of this issue slightly (so I can point the various scattered PUT-related issues to it)

[dmitrizagidulin]

Related issues:

• solid-spec/201 - PUT should not switch resources between container and non-container
• solid-spec/198 - Is PUT a standard?
• solid-spec/185 - Does PUT /foo/bar add a containment triple for /foo/ ldp:contains /foo/bar?
• solid-spec/30 - PUT for create requires extension of LDP
• solid/71 - container type for container that creates subdomains
• solid/42 - Disable random slug generation for containers

[csarven]

I like @acoburn's questions from solid/solid-spec#198 (comment) so I'm adding them here for a more comprehensive coverage. The reader may want to contrast the following response with what I said in #40 (comment) but note that the intention of that comment was exploratory.

LDP, however, does not specify the exact behaviors of Create-on-PUT, for example:

What container (if any) will now include the ldp:contains triples pointing to this new resource?

None. LDP does not recommend updating containment triples via PUT https://www.w3.org/TR/ldp/#ldpc-put-mbrprops :

LDP servers SHOULD NOT allow HTTP PUT to update a LDPC’s containment triples; if the server receives such a request, it SHOULD respond with a 409 (Conflict) status code.

Given a PUT to /foo/bar/baz in the absence of /foo and /foo/bar, are those intermediate containers created?

Assuming that containment triples are desired, then the reason to not do that is same as above ie. "SHOULD NOT". It also goes against URIs Opacity. Moreover, it is at odds with https://tools.ietf.org/html/rfc7231#section-4.3.4 :

the intent of PUT is idempotent and visible to intermediaries

If the containers are created, what type of container is created? (ldp:BasicContainer would make sense, but there is no text clarifying that point)

Inapplicable. Implementation detail. (ldp:BasicContainer seems like the obvious candidate. There should at least be ldp:Container if a specific LDPC interaction model is not known at that time.)

If those intermediate resources already exist, what happens if they are not containers?

Inapplicable. Note related recommendation https://www.w3.org/TR/ldp/#ldpc-post-createrdf (and if that's adopted for PUT) ie. don't change a resource's interaction model - server wins. Moreover, in the event that the server allows the client to change what's already there, that also goes against LDP's non-normative recommendationto not re-use URIs for a different resource: https://www.w3.org/TR/ldp/#ldp-webarch-uri-reuse

If intermediate containers are created, what happens in the case of conflict in an eventually consistent back-end store with a high level of concurrency?

Inapplicable. Implementation detail.

How does Create-on-PUT square with the requirement to use If-Match (or similar) conditional headers with PUT?

Can you be more specific? LDP says SHOULD for ETag/If-Match.

[JordanShurmer]

Let's support PUT

Here are my thoughts on this issue, especially in light of the #97 and #98 resolution (no orphaned resources, and containment is hierarchical).

I am strongly in favor of Solid servers supporting PUT to create resources. I would even argue for a MUST (something like "Solid servers MUST support PUT to create resources"). I have two main reasons for this:

1. The specs that this is build on say MAY, which can potentially lead to interoperability issues for clients which were written against a server that does support it, not realizing that not all servers will.
2. It is a very common way to create resources in general. Client side web developers are familiar with using PUT to create resources in other apis, and they would expect it here
3. There is no good reason to not support it ;) (I'll address some of the concerns already mentioned)

Addressing the concerns

I see two concerns in the previous comments (both raised by @csarven).

updating containment triples

There is some concern about this requirement from the LDP spec:

5.2.4.1 LDP servers SHOULD NOT allow HTTP PUT to update a LDPC’s containment triples; if the server receives such a request, it SHOULD respond with a 409 (Conflict) status code. ldp

However, I will observe that this requirement is in section 5.2, which is addressing HTTP requests to LDPC, not to resources. Also, 5.2.4 explicitly says "this specification imposes the following new requirements for LDPCs.". This means this requirement is referring to a client making a PUT request on a container's url directly, not PUTing a resource in that container This simply says that a client can't attempt to add or remove things from the container by manually updating the containment triples and updating the container url. It has no bearing on whether the server should update the containment triples when a resource is PUT "under" that container url or not.

PUTing in arbitrary locations

The other concern involves what to do when a resource is PUT somewhere. E.g. should parent containers be created, should a "root" container by updated instead, etc.

I think this has been resolved by #98 enforcing hierarchical containment and #97 prohibiting orphaned children.

When a resource is created via PUT it is by definition contained in it's "parent" in the url.

[kjetilk]

Yes, indeed, I think we are much closer to this now.

Also, I like @michielbdejong 's observation from solid/solid-spec#198 that

You would use If-None-Match: *, meaning only create the resources if no ETags match, i.e. if the resource does not exist.

that's also very helpful, and a feature I hadn't thought of.

[csarven]

However, I will observe that...

The LDP spec is intended to be organised for readability. Document sectioning is not strict in the algorithmic sense. There are many cross-references. Basing things on the structure of sections is fine, but I'd take it with a grain of salt. Because even with that line of reasoning, it is evident that LDP doesn't say anything about PUT creating a resource within a particular container. The client doesn't specify which container to PUT a resource in and the server is not told how to determine where it should be contained in, if at all - implementation detail. Compare that with how a client can make a request to create a new resource in a particular container using POST. As it stands, what's clear with LDP's PUT is that, it can create a resource (with no strings attached to a container) and the resource can be updated.

That is one of the reasons why we went around in circles in LDP, RFCs etc to be certain about what it says and doesn't say, what's allowed and not - in several issues - as well as to figure out the affordance of URIs with slashes. The rough consensus around 97, 98 (and certainly elsewhere) indeed helps a lot to clarify what we perceived to be under specified for Solid's needs.

[kjetilk]

I have been thinking about PUT today, not only on resource creation, but also updates, I think I have a proposal. But first, two notes:

1. All the complexity around containment triple management is addressed by Is Solid containment strictly hierarchical?  #98, i.e. containment triples will exist as a function of hierarchy.
2. In LDP, all writes are optional. My assumption is that Solid without writes is just a Web server, thus, Solid is read-write, and therefore the optional nature of writes in LDP should not apply to Solid.

In my proposal, I will introduce two concepts, exactness and consistency. Now only informally. By consistency, I mean that a request must be consistent with its data, metadata and URI. This relates to #128 in that if it writes to a /, if the client declares an interaction model, that interaction model must be ldp:Container. Also, it relates to #121 , in that the resource type of the request must not conflict with existing types. When writing this down formally, I think that is something we need to set down in clear terms. By exactness, I mean to constrain the freedom the server has to adapt the request. In my current thinking, it boils down to that the server has to accept the request URI or reject the request, it cannot modify the URI.

With that, here's my thinking on PUT:

• Container

  • Creation: MUST be supported, consistent and exact. Requires append on parent priv..
  • Update: MUST NOT be supported. @TallTed promoted this idea in Specify/advise semantics of default Container resources (index.html/.ttl) #69, and I have come to agree. PUT replaces the entire state of the resource, and you can never be allowed to do that, since much of the state is governed by the server. It is also discouraged by LDP. There is a possible opening around HTML representations, but that too has problematic sides. I therefore suggest that PUT is not allowed in updating a container.

• RDF and non-RDF resources

  • Creation: MUST be supported, consistent and exact. Requires append on parent priv. If-None-Match: * MUST be supported, so that a client can ensure it doesn't execute an Update operation when a Creation operation is intended.
  • Update: MUST be supported, consistent and exact. Requires write priv.

• ACL resources

  • Same as above, but with control on parent and control privs respectively.

[csarven]

it cannot modify the URI

Unless you mean MUST NOT, this is covered by SHOULD NOT re-use URI in LDP drawing from AWWW.

re Container Creation: Agree.

re Container Update: Already mentioned #40 (comment) , solid/solid-spec#201 (comment) for different circumstances prior to 69.. So, yes, I agree and glad that we're now in alignment on the intentions ;) If by "PUT not allowed in updating a container" you mean MUST NOT, then I think SHOULD NOT would be preferable as in LDP; primarily for the reason of client staying far clear of containment management. PUT should not appear in Allow header of that resource while it exists.

re RDF and non-RDF resources Creation: Agree.

re RDF and non-RDF resources Update: Agree.

re ACL resources: Agree. We should further review controls.

[csarven]

continued on Container Update:

Recommending against updating containers with PUT means that the homepage case gets past onto POST or PATCH. POST is unclear. PATCH is currently only in SPARQL Update ( #125 ).

If core concern is about avoiding containment, all that needs to be done is checking to see if the RDF graph of the representation contains statement for new containment or altering a containment under server-managed triples, then return 409 and explain why in the message. We don't actually have to prevent PUT updates altogether as it would make it difficult or impossible for some use cases to be realised.

[TallTed]

[@kjetilk] it cannot modify the URI

[@csarven] Unless you mean MUST NOT, this is covered by SHOULD NOT re-use URI in LDP drawing from AWWW.

Uh, @csarven, you left out the introductory parts of @kjetilk's sentence, without which the phrase you kept has a totally different meaning ... which your response still isn't relevant for.

The full sentence (with one added word):

In my current thinking, it boils down to that the server has to accept the [PUT] request URI or reject the request, it cannot modify the URI.

The "re-use" of a URI which is discussed in AWWW and LDP would be using a URI which previously identified one referent to identify a different referent. That is not what would happen if a server took a PUT request and wrote that content to a different location than what was requested.

That discrepancy notwithstanding, LDP forbids such relocation on a PUT (i.e., creating the resource at a different location than the request URI); this is only acceptable on a POST.

---

I'm seeing a LOT of cherry-picking from LDP. (Similar is happening with other external specs, but to a lesser extent.) One sentence (or less) -- which may not even come from a normative section of the LDP spec -- gets taken out of context, and presented as the authority behind a position taken on how Solid should function, while ignoring other parts in the LDP spec which contradict that position.

This is not productive.

Either LDP applies to Solid, in which case it must be taken as a whole, with primary if not exclusive focus on the normative sections of the LDP spec; or LDP does not apply to Solid, in which case, most if not all appeals and references to the LDP spec should be ceased.

[csarven]

Uh, @csarven

Ted, you're absolutely right. I was specifically adding on to the point modifications, changes, reuse..

The "re-use" of a URI which is discussed in AWWW and LDP would be using a URI which previously identified one referent to identify a different referent. That is not what would happen if a server took a PUT request and wrote that content to a different location than what was requested.

Yes Ted, I'm aware of that. I didn't contradict that. I think the term "re-use" is fairly clear and I was responding specifically (as I've even quoted) "modify".

---

I don't think we need to adopt binary decision making with extremes. There are notions in LDP that's thought through and aligned with what we'd like to see in Solid. Some things in LDP are indeed unclear or absurd. We can better understand the gaps and borrow what's useful. We still have to go through that process even if we were to take LDP as a whole.

[kjetilk]

Just to clarify my thinking with regards to LDP is that we shouldn't have it as a full normative reference, since many of Tim's ideas collide with that, but we should have the ambition to make Solid trivially implementable on LDP. So, we're allowed to cherry-pick :-)

[kjetilk]

If by "PUT not allowed in updating a container" you mean MUST NOT, then I think SHOULD NOT would be preferable as in LDP;

OK. I can go with SHOULD NOT, but I don't see the win in terms of interop, that would mostly be in the case someone figures out something clever around it... And I guess they could. You could have "add-only replacements" and that kind of stuff, so indeed, it can be envisioned valid use cases for it.

[acoburn]

Just to add some implementation context....

PUT is supported for Containers in Trellis with the following behavior: first, internally, there is a clear separation between "user-managed" triples and "server-managed" triples; containment triples are "server-managed". So any containment triples that are included in a PUT request are simply ignored. I think this implementation is legit because of LDP 4.2.4.1:

LDP servers may ignore LDP-server-managed properties

where "server-managed" properties are defined as:

The portion of an LDP's triples whose behavior is constrained directly by this specification; for example, membership triples and containment triples

And following from that, I see no problems with allowing PUT on Containers, provided that the containment triples are not modified. I will note that PUT operations on Container resources in Trellis cannot in any way modify the LDP containment triples on that target resource.

[csarven]

Great re Trellis 4.2.4.1 implementation.

5.2.4.1 is a more specific constraint on the container than 4.2.4.1 on the resource.

We can look at the effects when taken individually:

4.2.4.1 potentially allows containment to be affected. This criteria alone will introduce the possibility of discovery and data issues without having defined error routes. It also introduces complexity to make sure that there are no conflicting containment statements.

5.2.4.1 prevents clients from attempting to update containment and gives a specific error. This makes sure that the resource doesn't have conflicting statements with respect to the server-managed triples portion.

4.2.4.1 alone on the container doesn't cut it. 5.2.4.1 is useful but the actual implementation can vary, especially when trying to deal with use case like the the homepage. It is possible that an implementation can return 409 as soon as it observes containment triples in the payload. Yet another implementation would only reach the same result by observing the actual resources that may be affected. This second point is what I had in mind when earlier I said:

if the RDF graph of the representation contains statement for new containment or altering a containment under server-managed triples, then return 409 and explain why in the message.

What that allows is an application to use PUT to update a container as long as there is no conflicting information about containment. If the payload includes containment triples that is a subset of the server-managed triples, the request can succeed.

I'm curious to know if and how does Trellis implement 5.2.4.1.

[kjetilk]

Thanks, @acoburn , that's interesting!

Yes, I can see that this is consistent with LDP, but my concern goes back to the more Fielding-idea of a representation and the expectation in the RFC that

The target resource in a POST request is intended to handle the enclosed representation according to the resource's own semantics, whereas the enclosed representation in a PUT request is defined as replacing the state of the target resource.

In my interpretation, by separating server-managed triples and user-managed triples, that is essentially the resource's own semantics, and therefore falls under what shouldn't be PUT. However, as the RFC also notes, there has always been a certain tension with the interpretation of PUT in that even though the intention is to always replace, you cannot ever rely on observing that state, because the resource state may have changed between the PUT and a subsequent GET. Thus, ignoring server-managed triples in the payload is necessarily a victimless crime. :-)

I'm certainly open to relax the constraint on container update. My main idea with that is that it makes it simpler to implement. But perhaps that too doesn't hold.

So, first, do we have rough consensus on all the other operations at this point? Is container update the only item where there is (slight) disagreement?

[csarven]

Good.

The unclarity on container update with PUT is same as POST.

We agree that PUT is allowed as long as containment information is not changed. I suggest we use LDP 5.2.4.1.

The criteria from 5.2.4.1 can as well be applied to POST (#108) , PATCH (#85), index.html (#69).

[kjetilk]

During test development, I encountered the following NSS behaviour:

a resource /test-put-bc/dahut-rs.ttl exists and is understood by the server as an RDF source and a Turtle representation.

First, I update that with a

PUT /test-put-bc/dahut-rs.ttl
Content-Type: "text/plain";
Link: <http://www.w3.org/ns/ldp#NonRDFSource>; rel="type"

NSS will create a file dahut-rs.ttl$.txt as a result of this operation. Then, I drop the Link header, and again, it creates a file dahut-rs.ttl$.txt.

My understanding is that both these operations should result in a 409, as it is changing the interaction model of the resource in an inconsistent way.

But then, I figured, it could be argued that they are just two different representations of the same resource, and should be allowed, i.e. it is not a change, it just adds another representation.

My intuition says that allowing this would introduce quite some extra complexity, and also is a more radical departure from LDP than we have done before, so it may be hard to implement this on top of a pure LDP server, but I'll bring it up for discussion.

[RubenVerborgh]

The above NSS behavior is unintentional; it is a consequence of missing on-disk conneg.