Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MSC3757: Restricting who can overwrite a state event #3757

Open
wants to merge 26 commits into
base: main
Choose a base branch
from
Open
Changes from 18 commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
ff5fd48
Add 'Restricting who can overwrite a state event'
andybalaam Mar 25, 2022
610f244
Rename MSC3757 to its proper number
andybalaam Mar 25, 2022
bfde329
Clarify wording of comment
andybalaam Mar 31, 2022
344e876
Add unstable room version
andybalaam Mar 31, 2022
ccb7e52
Note that this requires a new room version
andybalaam Mar 31, 2022
1ce7e0e
Refer to MSC3760 as an alternative
andybalaam Mar 31, 2022
6df6109
Re-word link to MSC3760
andybalaam Mar 31, 2022
6e108b3
Update to m.self to match latest thinking on MSC3672
andybalaam Apr 26, 2022
bd4176f
Remove the user of 'unprivileged'
andybalaam May 11, 2022
e352a1d
Add a note about allowed characters
andybalaam May 11, 2022
5e95ff3
Reword proposed auth rule
AndrewFerr Sep 4, 2024
68dc97f
Nitpick: always use formatted text for state_key
AndrewFerr Sep 11, 2024
17890fd
Nitpick: remove trailing whitespace
AndrewFerr Sep 11, 2024
f962bf3
Change recommended room versions to apply on
AndrewFerr Sep 11, 2024
dd9b33e
Mention that _ can't be in any form of server name
AndrewFerr Sep 24, 2024
eb0eed6
Add issue of incompatibility with long MXIDs
AndrewFerr Sep 24, 2024
ac24510
Add issue of underscores in domain names
AndrewFerr Sep 24, 2024
9490cbd
Fix typo
AndrewFerr Sep 24, 2024
486b0cd
Use device ID suffix in location beacon example
AndrewFerr Sep 26, 2024
590ff96
Increase state key size limit & set suffix limit
AndrewFerr Sep 26, 2024
d9b149d
Keep original size limit on unprefixed state keys
AndrewFerr Sep 27, 2024
ae17437
Move paragraph to alternative section
AndrewFerr Sep 27, 2024
8222738
Add alternative of state key arrays
AndrewFerr Sep 27, 2024
63955d7
Add alternative of field for non-state events
AndrewFerr Sep 27, 2024
07d784a
Clarify state subkey/array relevance to user IDs
AndrewFerr Sep 27, 2024
99698ef
Fix formatting of auth rule's numeric list
AndrewFerr Sep 27, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
146 changes: 146 additions & 0 deletions proposals/3757-restricting-who-can-overwrite-a-state-event.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,146 @@
# MSC3757: Restricting who can overwrite a state event.
andybalaam marked this conversation as resolved.
Show resolved Hide resolved
andybalaam marked this conversation as resolved.
Show resolved Hide resolved
andybalaam marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is actually derestricting who can overwrite a state event.

Suggested change
# MSC3757: Restricting who can overwrite a state event.
# MSC3757: Relaxing the restrictions on who can overwrite a state event.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's true for MSC3779, but not as much for this one; the former bypasses PL restrictions for setting state that belongs to the sender (where "belongs to" = the state key starts with the sender's MXID), but this MSC does not.

The restriction proposed by this MSC is to prevent state that belongs to a particular user from being overwritten by other, equally-powerful users.

The only PL restriction that's relaxed by this MSC is for allowing more powerful users to overwrite state that doesn't belong to them, for the sake of having a tool against state graffiti.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right. Still, I find the title a bit confusing. How about:

Suggested change
# MSC3757: Restricting who can overwrite a state event.
# MSC3757: Extending the set of write-protected state keys

or something

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TBH I informally refer to this MSC as "the owned state MSC" despite that being the title of MSC3779 😉

Since one of the distinguishing differences of this MSC over 3779 is the ability for admins to manage others' state, maybe we could call it

Suggested change
# MSC3757: Restricting who can overwrite a state event.
# MSC3757: State event ownership (with admin management)

?


andybalaam marked this conversation as resolved.
Show resolved Hide resolved
## Problem

Currently there are three main ways to limit who can overwrite a state event:

* If a user's PL is greater than the `m.room.power_levels` `state_default` field
* If a user's PL is greater than the `m.room.power_levels` `events` field for that event type
* If a state event has a state key which begins with an `@`, then the sender's mxid must match that state key.
Comment on lines +5 to +9
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I find this a confusing way of phrasing the current restrictions.

I'd say that there are essentially two current restrictions, imposed by rules 7 and 8 in the auth rules.

In short:

  1. The user's PL must not be less than the "required power level" for that event.
  2. In addition, if a state event has a state key which begins with an @, then the sender's mxid must match that state key.

The spec for m.room.power_levels defines required power level, but in summary, it is determined by the setting in m.room.power_levels events field for that event type if present; otherwise, it is determined by the state_default field.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I find the @ prefix restriction a bit questionable, for the record. The only specified state event which uses it is m.room.member, which the auth rules validate state_key well before the restriction is checked. Non-spec usage of the state key already applies tricks like prefixing user IDs with _ to ensure they aren't hit by the restriction, and I'm not really convinced that location sharing/beacons need to pack a user ID into the state key anyways (see https://github.com/matrix-org/matrix-spec-proposals/pull/3757/files#r1103877363 ).

This has me leaning towards removing the @ prefix restriction entirely, but I recognize that's not exactly a helpful opinion for this MSC to tackle. Creating stronger arguments for why the dependent features require the user ID in the state key I think would help, though.


This is problematic if a user needs to publish multiple state
events of the same type in a room, but would like to set access control so
that only they can subsequently update the event. An example of this is if a
user wishes to publish multiple live location share beacons as per [MSC3489](https://github.com/matrix-org/matrix-spec-proposals/pull/3489)
and [MSC3672](https://github.com/matrix-org/matrix-spec-proposals/pull/3672), for instance one per device. They will typically not want
other users in the room to be able to overwrite the state event,
so we need a mechanism to prevent other peers from doing so.
Comment on lines +11 to +17
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MSC3489 and MSC3672 are not prioritized for inclusion right now, which raises questions about why this MSC is put up for FCP if we're not going to use it for a while. A question around whether the implementation is suitably deployed also comes to mind, though I'm not certain enough to raise that as a FCP-blocking concern.

An update to the introduction to better list out all the features which benefit from this MSC feels needed.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll let @AndrewFerr talk more about his use case, but broadly I think its for per-device call state within a room.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

its for per-device call state within a room.

That's correct; more precisely, it's for per-device call membership state (as a user can join a call from multiple devices at once).

Currently, the call membership state is an array, with one element per participating device. The problem is that updating that state array is prone to race conditions, as adding/removing an entry to the array is dependent on the current value of the array.

This MSC allows for using one state event per device, which each device can freely update without risk of any other device (or user, for that matter) overwriting it at the same time.


[MSC3489](https://github.com/matrix-org/matrix-spec-proposals/pull/3489) originally proposed that the event type could be made variable,
appending an ID to each separately posted event so that each one could
separately be locked to the same mxid in the `state_key`. However, this is
problematic because you can't proactively refer to these event types in the
`events` field of the `m.room.power_levels` event to allow users to post
them - and they also are awkward for some client implementations to
manipulate.

## Proposal
andybalaam marked this conversation as resolved.
Show resolved Hide resolved

Therefore, we need a different way to state that a given state event may only
be written by its owner. **We propose that if a state event's `state_key` *starts with* a matrix ID (followed by an underscore), only the sender with that matrix ID (or higher PL users) can set the state event.** This is an extension of the current behaviour where state events may be overwritten only if the sender's mxid *exactly equals* the `state_key`.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it valid to have an empty string as the suffix? E.g @kegan:matrix.org_? What about having arbitrary unicode characters? Null bytes? Please let's be sensible and force sensible restrictions on this new user-defined variable, lest we end up in another hell of poor validation causing problems for clients/servers.

I would suggest a very strong restriction of [a-z0-9] unless there are good reasons to allow other characters (there almost certainly are not).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Allowing an empty suffix probably won't hurt. I can imagine cases of the suffix being a property that may take on an empty/nullish value.

For a device ID to be usable as a suffix, the suffix charset must include all characters that can be used as a device ID. Unfortunately, there's no specced device ID charset that I can find, and in practice it is quite broad:

  • Device IDs generated by QR code logins (at least on Element X Android) can contain / and +.
  • /_matrix/client/v3/login allows setting a custom device ID of any string, and the Synapse implementation allows all sorts of characters (and lets me set a device ID of a b ?!@#$%^&*()-=_+/).

So if we want device ID suffixes now, maybe defining a suffix charset is premature, unless this MSC also defines a stricter device ID charset. But that raises the issue of what to with existing device IDs that don't follow the charset...

Copy link
Member

@kegsay kegsay Sep 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is surely too literal though. You can map device IDs deterministically to a valid character set, at its most basic SHA256(device_id).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good idea. I was worried that the input size of device IDs would be too large to safely hash, but the point is to avoid hash collisions, the chance of which should still be small.

Then, this MSC can do away with trying to define a suffix length / charset that can fit a raw device ID.

I will still propose a few non-word characters to be in the suffix charset, because it may be handy to have a suffix containing multiple properties that are easy to separate with a non-word character (eg. @user:hs_prop1_prop2_prop3). The spec already uses a charset of [0-9a-zA-Z.=_-] for email login secrets/tokens, and would be a decent charset to use here as well.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On second thought, a handy consequence of not restricting the suffix charset is the ability to prefix any existing state key with a user's MXID to scope its ownership to that user. With a restricted suffix charset, there may be some state keys that would be invalid as a suffix.

IMO as long as the charset of state keys in general is left unrestricted, there's little benefit in restricting the suffix charset, unless doing so is a step towards restricting the general state key charset.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that there's no point in restricting these specific state keys when all other state keys are still arbitrary strings. To take advantage of this MSC, you must give users permission to send state events of certain types. That means users will be able to send unprefixed state keys too, which will not have any character set restrictions.

Restricting state keys in general might be a good idea to do in combination with MSC2828

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't agree with the attitude of "we don't have good validation, so let's not add validation". There's zero reason to punt this down the line to another MSC. It's absolutely trivial to expand the character set or length limits in another MSC. It's very difficult to restrict it once there are client implementations in the wild relying on there being no restrictions. Validation is important to reduce the attack surface of any newly added features. The suffix string will be stored in new places where the state key is not (e.g DBs will likely either have this as a column, or indexed in such a way to allow efficient ordered lookups), which means there will be new code written to read this input. Validate the input, please.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the intent of a restricted suffix charset is to ease parsing out the user ID prefix, then the usefulness of the restriction might be limited by how the prefix must be able to contain any printable ASCII character, due to having to support historical user IDs. Also, since the parsing is concerned more with the user ID prefix than with the suffix, locking down the suffix charset might not impact much.

Besides, the "suffix string" isn't really meant to be a new kind of syntax for state keys, but is just a result of the semantic of user-scoped state keys via string packing. It's perhaps better to think of a scoped state key not as a user ID + a suffix, but as an ordinary state key prefixed by a user ID; or rather, that this MSC proposes allowing state keys to optionally be prefixed by a user ID. Being prefixed doesn't change the nature of the content of the rest of the state key, but applying different restrictions to prefixed & unprefixed state keys would imply that it does.

Though if restricting the charset is non-negotiable, then maybe a compromise is to apply a broad charset, like all printable ASCII characters.


We also allow users with higher PL than the original sender to overwrite state
events even if their mxid doesn't match the event's `state_key`. This fixes an abuse
vector where a user can immutably graffiti the state within a room
by sending state events whose `state_key` is their matrix ID.

Practically speaking, this means modifying the [authorization rules](https://spec.matrix.org/v1.2/rooms/v9/#authorization-rules) such that rule 8:

> 8. If the event has a `state_key` that starts with an `@` and does not match the `sender`, reject.
becomes:

> 8. If the event has a `state_key` that starts with an `@`:
> 1. If the prefix of the `state_key` before the first `_` that follows the first `:` (or end of string) is a valid user ID:
> 1. If that user ID does not match the `sender`, and the `sender`'s power level is not greater than that of the user denoted by the ID, reject.
> 2. Otherwise, reject.
No additional restrictions are made about the content of the `state_key`, so any characters that follow the `sender` + `_` part are only required to be valid for use in a `state_key`.

For example, to post a live location sharing beacon from [MSC3672](https://github.com/matrix-org/matrix-spec-proposals/pull/3672):

```json=
{
"type": "m.beacon_info",
"state_key": "@stefan:matrix.org_assetid1", // Ensures only the sender or higher PL users can update
kegsay marked this conversation as resolved.
Show resolved Hide resolved
"content": {
"m.beacon_info": {
"description": "Stefan's live location",
"timeout": 600000,
"live": true
},
"m.ts": 1436829458432,
"m.asset": {
"type": "m.self"
}
}
}
```

Since `:` is not permitted in the localpart and `_` is not permitted in the domain part of an mxid (see [Historical User IDs](https://spec.matrix.org/v1.2/appendices/#historical-user-ids)), it is not possible to craft an mxid that matches the beginning of a state key constructed for another user's mxid, so state keys restricted to one owner can never be overwritten by another user.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While the spec specifies a grammar for server names, are we SURE that servers verify that grammar? While officially underscores are invalid in domain names and you can't order a TLS certificate for it nowadays, such domains still exist in the wild and wildcard certificates even allow using TLS with them. For example: https://my_sarisari_store.typepad.com/

I think relying on underscores as a separator is a rather scary trap and I wouldn't bet on all currently developed Matrix servers rejecting such server names.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ac24510 adds this to the "Potential issues" section.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about using @ as the separator? Since the separator will never be the first character in the state key, it shouldn't be possible to confuse it with the user ID sigil.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

| could also be a good separator.


## Potential issues

### Incompatibility with long user IDs

As the spec enforces [a size limit of 255 bytes for both MXIDs and state keys](https://spec.matrix.org/unstable/client-server-api/#size-limits),
the set of available MXID-prefixed state keys is smaller for long MXIDs than for short ones,
with the worst case of none being available for MXIDs equal to the size limit.
Thus, long MXIDs are restricted from being used as state key prefixes to designate state ownership.
This issue could be solved by increasing the size limit for state keys.
AndrewFerr marked this conversation as resolved.
Show resolved Hide resolved

### Incompatibility with domain names containing underscores

Although both [the spec](https://spec.matrix.org/unstable/appendices/#server-name)
and [RFC 1035 §2.3.1](https://www.rfc-editor.org/rfc/rfc1035#section-2.3.1)
forbid the presence of underscores in domain names,
there noneless exist resolvable domain names that contain underscores.
The proposed auth rule for parsing an MXID prefix from an underscore-separated state key would fail
on such domain names.

Possible solutions include:
- using a different character to terminate an MXID prefix in state keys. The character must be one
that's known to be absent from domain names in practice, and must also not be any character that
the spec allows to appear in a server name.
- refining the proposed auth rule for parsing an MXID prefix such that it does not fail on domain
names that contain an underscore. One way to achieve this is to leverage the absence of
underscores from top-level domains.

## Alternatives
AndrewFerr marked this conversation as resolved.
Show resolved Hide resolved

As originally proposed in [MSC3489](https://github.com/matrix-org/matrix-spec-proposals/pull/3489) and [MSC3672](https://github.com/matrix-org/matrix-spec-proposals/pull/3672), we can require
the use of a state key equal to the sender's mxid, but this means we can only
have one such event of each type, so those MSCs proposed using different types
for each unique event.

An earlier draft of this MSC proposed putting a flag on the contents of the
event (outside of the E2EE payload) called `m.peer_unwritable: true` to indicate
if other users were prohibited from overwriting the event or not. However, this
unravelled when it became clear that there wasn't a good value for the `state_key`,
which needs to be unique and not subject to races from other malicious users.
By scoping who can set the `state_key` to be the mxid of the sender, this problem
goes away.

[MSC3760](https://github.com/matrix-org/matrix-spec-proposals/pull/3760)
proposes to include a dedicated `state_subkey` as the third component of what
makes a state event unique.

## Security considerations

This change requires a new room version, so will not affect old events.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The mention of requiring a new room version should be at the top of the Proposal section. How I've done this in the past is literally:

In a future room version...

[actual proposal]


As this changes auth rules, we should think carefully about whether could
introduce an attack on state resolution. For instance: if a user had higher
PL at some point in the past, will they be able to abuse somehow this to
overwrite the state event, despite not being its owner?

When using a `state_key` prefix to restrict who can write the event, we have
deliberately chosen an underscore to terminate the mxid prefix, as underscores
are not allowed in [any form of server name](https://spec.matrix.org/v1.11/appendices/#server-name)
(either a DNS name or IPv4/6 address, with or without a numeric port specifier).
A pure prefix match will **not** be sufficient,
as `@matthew:matrix.org` will match a `state_key` of form `@matthew:matrix.org.evil.com:id1`.

This changes auth rules in a backwards incompatible way, which will break any
use cases which assume that higher PL users cannot overwrite state events whose
`state_key` is a different mxid. This is considered a feature rather than a bug,
fixing an abuse vector where users could send arbitrary state events
which could never be overwritten.

andybalaam marked this conversation as resolved.
Show resolved Hide resolved
## Unstable prefix

While this MSC is not considered stable, implementations should apply the behaviours of this MSC on top of room version 10 or higher as `org.matrix.msc3757`.

## Dependencies

None