Join room from invitation client-server API fails for Synapse 1.71.0

[manning-ncsa]

Description

I am unable to join a room using an access token on v1.71.0 using the same code that worked on v1.69.0. The Python function I am using is

def accept_invitation(self, room_id=None, access_token=None):
    try:
        assert room_id
        if not access_token:
            access_token = self.conf['auth_token']
        response = requests.post(
            f'''{self.conf['server_url']}/_matrix/client/v3/rooms/{room_id}/join''',
            params = {
                "access_token": access_token,
            }
        )
        return True
    except Exception as e:
        log.error(f'''ERROR accepting invitation: {e}''')
        return False

When I provide the access token for the invited Matrix account and the room to be joined, I get a warning in the Synapse server logs as shown below, where no response is ever sent and the account is not allowed to join the room:

2022-11-17 19:01:28,329 - synapse.http.servlet - 668 - WARNING - POST-48927- Unable to parse JSON from POST /_matrix/client/v3/join/!roomid:matrix.example.com?access_token=<redacted> response: Expecting value: line 1 column 1 (char 0) (b'')

However, a subsequent identical API call, but using a blank or invalid access token, actually allows the account to join the room!

2022-11-17 19:14:55,712 - synapse.api.auth - 455 - WARNING - POST-49861- Invalid access token in auth: <class 'pymacaroons.exceptions.MacaroonInitException'> Must supply serialized macaroon..

2022-11-17 19:14:55,713 - synapse.http.server - 108 - INFO - POST-49861- <XForwardedForRequest at 0x7f056054aa90 method='POST' uri='/_matrix/client/v3/rooms/!roomid:matrix.example.com/join?access_token=<redacted>' clientproto='HTTP/1.1' site='8008'> SynapseError: 401 - Invalid access token passed.

Steps to reproduce

• Invite an account to a private, unencrypted room.
• Attempt to join the room using the account's access token and the _matrix/client/v3/rooms/{room_id}/join API function. The request will stall and the account will not join the room.
• Repeat the request with a blank or invalid access token. The request will fail due to invalid credentials, but the account will now join the room.

Homeserver

matrix.musesframework.io

Synapse Version

1.71.0

Installation Method

Docker (matrixdotorg/synapse)

Platform

Kubernetes, using this Helm chart

Relevant log output

See description above

Anything else that would be useful to know?

No response

[DMRobertson]

First POST request

I get a warning in the Synapse server logs as shown below,

2022-11-17 19:01:28,329 - synapse.http.servlet - 668 - WARNING - POST-48927- Unable to parse JSON from POST /_matrix/client/v3/join/!roomid:matrix.example.com?access_token=<redacted> response: Expecting value: line 1 column 1 (char 0) (b'')

The warnings comes from

synapse/synapse/http/servlet.py

Lines 665 to 677 in 1bf2832

	try:
	content = json_decoder.decode(content_bytes.decode("utf-8"))
	except Exception as e:
	logger.warning(
	"Unable to parse JSON from %s %s response: %s (%s)",
	request.method.decode("ascii", errors="replace"),
	redact_uri(request.uri.decode("ascii", errors="replace")),
	e,
	content_bytes,
	)
	raise SynapseError(
	HTTPStatus.BAD_REQUEST, "Content not JSON.", errcode=Codes.NOT_JSON
	)

This is caught here and dealt with, but the warning is not suppressed:

synapse/synapse/rest/client/room.py

Lines 955 to 960 in 86c5a71

	try:
	content = parse_json_object_from_request(request)
	except Exception:
	# Turns out we used to ignore the body entirely, and some clients
	# cheekily send invalid bodies.
	content = {}

To get rid of the warning, you should send an empty JSON object {} as the body of your POST request.

where no response is ever sent and the account is not allowed to join the room:

Your logs aren't sufficient to demonstrate this.

WARNING - POST-48927- Unable to parse JSON from [...]

Can you please grep for all lines in your Synapse logs matching POST-48927 and provide them here?

Second POST request

2022-11-17 19:14:55,712 - synapse.api.auth - 455 - WARNING - POST-49861- Invalid access token in auth: <class 'pymacaroons.exceptions.MacaroonInitException'> Must supply serialized macaroon..

This comes from

synapse/synapse/api/auth.py

Lines 419 to 460 in eaed4e6

	# If the token isn't found in the database, then it could still be a
	# macaroon for a guest, so we check that here.
	try:
	user_id = self._macaroon_generator.verify_guest_token(token)
	# Guest access tokens are not stored in the database (there can
	# only be one access token per guest, anyway).
	#
	# In order to prevent guest access tokens being used as regular
	# user access tokens (and hence getting around the invalidation
	# process), we look up the user id and check that it is indeed
	# a guest user.
	#
	# It would of course be much easier to store guest access
	# tokens in the database as well, but that would break existing
	# guest tokens.
	stored_user = await self.store.get_user_by_id(user_id)
	if not stored_user:
	raise InvalidClientTokenError("Unknown user_id %s" % user_id)
	if not stored_user["is_guest"]:
	raise InvalidClientTokenError(
	"Guest access token used for regular user"
	)
	return create_requester(
	user_id=user_id,
	is_guest=True,
	# all guests get the same device id
	device_id=GUEST_DEVICE_ID,
	authenticated_entity=user_id,
	)
	except (
	pymacaroons.exceptions.MacaroonException,
	TypeError,
	ValueError,
	) as e:
	logger.warning(
	"Invalid access token in auth: %s %s.",
	type(e),
	e,
	)
	raise InvalidClientTokenError("Invalid access token passed.")

The warning is Synapse's way of saying it doesn't recognise your token.

However, a subsequent identical API call, but using a blank or invalid access token, actually allows the account to join the room!

Your logs aren't consistent with this.

2022-11-17 19:14:55,713 - synapse.http.server - 108 - INFO - POST-49861- <XForwardedForRequest at 0x7f056054aa90 method='POST' uri='/_matrix/client/v3/rooms/!roomid:matrix.example.com/join?access_token=' clientproto='HTTP/1.1' site='8008'> SynapseError: 401 - Invalid access token passed.

This suggests we returned HTTP 401 Unauthorized; I'd expect 200 Ok for a successful join.

Can you please also provide all logs containing POST-49861?

[clokep]

As an aside -- it is recommended that you send an access token via headers, not a query parameter. Your above code could be corrected for this (and the empty body) with:

def accept_invitation(self, room_id=None, access_token=None):
    try:
        assert room_id
        if not access_token:
            access_token = self.conf['auth_token']
        response = requests.post(
            f'''{self.conf['server_url']}/_matrix/client/v3/rooms/{room_id}/join''',
            headers = {
                "Authorization": f"Bearer {access_token}",
            },
			json={},
        )
        return True
    except Exception as e:
        log.error(f'''ERROR accepting invitation: {e}''')
        return False

[manning-ncsa]

Thank you both for your responses. While I still do not understand why the original Python code did not work as it did before, it is possible that a problem with my Redis server that I discovered after creating this issue was actually to blame for the odd behavior. The improved code shared by @clokep worked perfectly to join the account to a new room I created to test it. We can close this.