Commits on Jan 28, 2020

1. XP-456: replace the CLI arguments with a config file …

   Cc: finiteprods <kwok.doc@gmail.com>
   Cc: Robert Steiner <robertt.debug@gmail.com>
   Cc: janpetschexain <58227040+janpetschexain@users.noreply.github.com>
   
   References
   ==========
   
   https://xainag.atlassian.net/browse/XP-456
   https://xainag.atlassian.net/browse/DO-58
   
   Rationale
   =========
   
   The CLI is getting complex so it is worth loading the configuration
   from a file instead.
   
   Implementation details
   ======================
   
   TOML
   ----
   
   We decided to use TOML for the following reasons:
   
     - it is human friendly, ie easy to read and write
     - our configuration has a pretty flat structure which makes TOML
       quite adapted
     - it is well specified and has lots of implementation
     - it is well known
   
   The other options we considered:
   
     - INI: it is quite frequent in the Python ecosystem to use INI for
       config files, and the standard library even provides support for
       this. However, INI is not as powerful as TOML and does not have a
       specification
     - JSON: it is very popular but is not human friendly. For instance,
       it does not support comments, is very verbose, and breaks
       easily (if a trailing comma is forgotten at the end of a list for
       instance)
     - YAML: another popular choice, but is in my opinion more complex
       than TOML.
   
   Validation
   ----------
   
   We use the third-party `schema` library to validate the
   configuration. It provides a convenient way to:
   
   - declare a schema to validate our config
   - leverage third-party libraries to validate some inputs (we use the
     `idna` library to validate hostnames)
   - define our own validators
   - transform data after it has been validated: this can be useful to
     turn a relative path into an absolute one for example
   - provide user friendly error message when the configuration is
     invalid
   
   The `Config` class
   ------------------
   
   By default, the `schema` library returns a dictionary containing a
   valid configuration, but that is not convenient to manipulate in
   Python. Therefore, we dynamically create a `Config` class from the
   configuration schema, and instantiate a `Config` object from the data
   returned by the `schema` validator.
   
   Package re-organization
   -----------------------
   
   We moved the command line and config file logic into its own `config`
   sub-package, and moved the former `xain_fl.cli.main` entrypoint into
   the `xain_fl.__main__` module.
   
   Docker infrastructure
   ---------------------
   
   - Cache the xain_fl dependencies. This considerably reduces
     "edit->build-> debug" cycle, since installing the dependencies takes
     about 30 minutes.
   - Move all the docker related files into the `docker/` directory
   
   Current limitations and future work
   -----------------------------------
   
   1. The documentation generated for the `ServerConfig`,
      `AiConfig` and `StorageConfig`  classes is wrong. Each attribute is
      documented as "Alias for field number X". This can be fixed by
      having `create_class_from_schema()` setting the `__doc__` attribute
      for each attribute. However, we won't be able to automatically
      document the type of each attribute.
   
   2. When the configuration contains an invalid value, the error message
      we generate does not contain the invalid value in question. I think
      it is possible to enable this in the future but haven't really
      looked into it.

   

   little-dude committed Jan 28, 2020
   Configuration menu

   • View commit details
   • Copy full SHA for 06ba121
   • Browse repository at this point

   Copy the full SHA

   06ba121 View commit details

   Browse the repository at this point in the history