Kinto does not start when auth policy is not available

[SimonKlausLudwig]

I create a docker-compose file with Keycloak as the Identity Provider and Keycloak. As Keycloak takes some time to start, Kinto does not start and exists with this message:

<class 'json.decoder.JSONDecodeError'>: Expecting value: line 1 column 1 (char 0)

Im pretty sure this is as the issuer does not provide a valid json file. In Previous versions of docker-compose it was possible to make the kinto container depended. But this is no longer recommended.

"There's been a move away from specifying container dependencies in compose. They're only valid at startup time and don't work when dependent containers are restarted at run time. Instead, each container should include mechanism to retry to reconnect to dependent services when the connection is dropped. Many libraries to connect to databases or REST API services have configurable built-in retries. I'd look into that. It is needed for production code anyway."

What is the best way to handle this ?

[leplatrem]

Could you please share the stacktrace? I'd like to understand what code path leads to a call to authentication on startup!

[SimonKlausLudwig]

kinto_1 | Traceback (most recent call last):
kinto_1 | File "/usr/local/lib/python3.7/site-packages/pyramid/config/actions.py", line 307, in execute_actions
kinto_1 | callable(*args, **kw)
kinto_1 | File "/usr/local/lib/python3.7/site-packages/pyramid_multiauth/init.py", line 280, in grab_policies
kinto_1 | policy = factory(**kwds)
kinto_1 | File "/app/kinto/plugins/openid/init.py", line 26, in init
kinto_1 | self.oid_config = fetch_openid_config(issuer)
kinto_1 | File "/app/kinto/plugins/openid/utils.py", line 11, in fetch_openid_config
kinto_1 | _configs[issuer] = resp.json()
kinto_1 | File "/usr/local/lib/python3.7/site-packages/requests/models.py", line 900, in json
kinto_1 | return complexjson.loads(self.text, **kwargs)
kinto_1 | File "/usr/local/lib/python3.7/json/init.py", line 348, in loads
kinto_1 | return _default_decoder.decode(s)
kinto_1 | File "/usr/local/lib/python3.7/json/decoder.py", line 337, in decode
kinto_1 | obj, end = self.raw_decode(s, idx=_w(s, 0).end())
kinto_1 | File "/usr/local/lib/python3.7/json/decoder.py", line 355, in raw_decode
kinto_1 | raise JSONDecodeError("Expecting value", s, err.value) from None
kinto_1 | json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
kinto_1 |
kinto_1 | During handling of the above exception, another exception occurred:
kinto_1 |
kinto_1 | Traceback (most recent call last):
kinto_1 | File "/usr/local/bin/kinto", line 33, in
kinto_1 | sys.exit(load_entry_point('kinto', 'console_scripts', 'kinto')())
kinto_1 | File "/app/kinto/main.py", line 216, in main
kinto_1 | env = bootstrap(config_file, options={"command": "migrate"})
kinto_1 | File "/usr/local/lib/python3.7/site-packages/pyramid/paster.py", line 117, in bootstrap
kinto_1 | app = get_app(config_uri, options=options)
kinto_1 | File "/usr/local/lib/python3.7/site-packages/pyramid/paster.py", line 30, in get_app
kinto_1 | return loader.get_wsgi_app(name, options)
kinto_1 | File "/usr/local/lib/python3.7/site-packages/plaster_pastedeploy/init.py", line 129, in get_wsgi_app
kinto_1 | global_conf=defaults,
kinto_1 | File "/usr/local/lib/python3.7/site-packages/paste/deploy/loadwsgi.py", line 253, in loadapp
kinto_1 | return loadobj(APP, uri, name=name, **kw)
kinto_1 | File "/usr/local/lib/python3.7/site-packages/paste/deploy/loadwsgi.py", line 278, in loadobj
kinto_1 | return context.create()
kinto_1 | File "/usr/local/lib/python3.7/site-packages/paste/deploy/loadwsgi.py", line 715, in create
kinto_1 | return self.object_type.invoke(self)
kinto_1 | File "/usr/local/lib/python3.7/site-packages/paste/deploy/loadwsgi.py", line 152, in invoke
kinto_1 | return fix_call(context.object, context.global_conf, **context.local_conf)
kinto_1 | File "/usr/local/lib/python3.7/site-packages/paste/deploy/util.py", line 55, in fix_call
kinto_1 | val = callable(*args, **kw)
kinto_1 | File "/app/kinto/init.py", line 51, in main
kinto_1 | kinto.core.initialize(config, version=version, default_settings=DEFAULT_SETTINGS)
kinto_1 | File "/app/kinto/core/initialization.py", line 581, in initialize
kinto_1 | config.include("kinto.core", route_prefix=api_version)
kinto_1 | File "/usr/local/lib/python3.7/site-packages/pyramid/config/init.py", line 666, in include
kinto_1 | c(configurator)
kinto_1 | File "/app/kinto/core/init.py", line 207, in includeme
kinto_1 | config.commit()
kinto_1 | File "/usr/local/lib/python3.7/site-packages/pyramid/config/actions.py", line 151, in commit
kinto_1 | self.action_state.execute_actions(introspector=self.introspector)
kinto_1 | File "/usr/local/lib/python3.7/site-packages/pyramid/config/actions.py", line 314, in execute_actions
kinto_1 | tb,
kinto_1 | File "/usr/local/lib/python3.7/site-packages/pyramid/util.py", line 732, in reraise
kinto_1 | raise value.with_traceback(tb)
kinto_1 | File "/usr/local/lib/python3.7/site-packages/pyramid/config/actions.py", line 307, in execute_actions
kinto_1 | callable(*args, **kw)
kinto_1 | File "/usr/local/lib/python3.7/site-packages/pyramid_multiauth/init.py", line 280, in grab_policies
kinto_1 | policy = factory(**kwds)
kinto_1 | File "/app/kinto/plugins/openid/init.py", line 26, in init
kinto_1 | self.oid_config = fetch_openid_config(issuer)
kinto_1 | File "/app/kinto/plugins/openid/utils.py", line 11, in fetch_openid_config
kinto_1 | _configs[issuer] = resp.json()
kinto_1 | File "/usr/local/lib/python3.7/site-packages/requests/models.py", line 900, in json
kinto_1 | return complexjson.loads(self.text, **kwargs)
kinto_1 | File "/usr/local/lib/python3.7/json/init.py", line 348, in loads
kinto_1 | return _default_decoder.decode(s)
kinto_1 | File "/usr/local/lib/python3.7/json/decoder.py", line 337, in decode
kinto_1 | obj, end = self.raw_decode(s, idx=_w(s, 0).end())
kinto_1 | File "/usr/local/lib/python3.7/json/decoder.py", line 355, in raw_decode
kinto_1 | raise JSONDecodeError("Expecting value", s, err.value) from None
kinto_1 | pyramid.exceptions.ConfigurationExecutionError: <class 'json.decoder.JSONDecodeError'>: Expecting value: line 1 column 1 (char 0)
kinto_1 | in:
kinto_1 | Line 0 of file None:

[leplatrem]

The reason why we fetch issuer information during init is because we expose its information (endpoints etc.) in the root URL response.

A possible fix to that would consist in offering the possibility for plugins (like openid here) to register their metadata lazily.

Instead of calling

   config.add_api_capability(
        "openid",
        description="OpenID connect support.",
        url="http://kinto.readthedocs.io/en/stable/api/1.x/authentication.html",
        providers=providers_infos,
    )

we could have:

def get_infos(config):
     # fetch issuer infos
     return infos

config.add_lazy_api_capability("openid", get_infos)

And in the root URL code, when the content of a capability is a function we just call it before serving the response.