RxPY 3.0: Documentation

[MainRo]

With v3, an official documentation should be provided. This issue can be used to follow the progress on this topic. The following things must be done

• create outline
• migrate part of readme in doc
• reference doc from readme
• add a migration section
• document APIs : autodoc for all operators and classes
• choose a theme : Guzzle.
• publish the doc somewhere: https://rxpy.readthedocs.io
• add marble diagrams for operators
• add additional readings section
• license section
• contributing section
• Fix docstring errors in init.py
• spellcheck
• final pass on each section

[dbrattli]

There are some theme samples are http://www.writethedocs.org/guide/tools/sphinx-themes/ . Read-the-docs is nice. Also the Guzzle theme looks quite nice and clean.

As for hosting we should be able to get permission to deploy at e.g. http://reactivex.io/rxpy

[MainRo]

I have several commits ready. Should I push them on feature/rxpy-3.0 or wait until the merge on master is done ?

[MainRo]

Concerning the marble diagrams, initially I wanted to reuse the ones from reactivex.io. However they are not all consistent (there are several graphic styles, depending on the operators). Moreover linking to picture does not allow to see the marble diagram when reading the source code.

So I wonder if the following could be possible: Create a sphinx addon that would use a tool to convert text based marble diagram to pictures. This would have several advantages compared to static pictures that are hard to update:

• The diagrams are readable in the code documentation.
• They can be easily updated.
• diagrams are automatically available when adding new operators
• diagrams style is consistent.

The main issue here is to get this marble renderer. Surprisingly I did not find such a tool. If anybody is aware of an existing tool, I am interested! Otherwise maybe it could be relatively easy to implement one based on matplotlib. I will try some mocks if we want to document marble diagrams this way.

[dbrattli]

We might want to merge with master first. It's now being added new features on the feature branch, so it's time to merge. We can still have a period of rapid development before locking down and starting doing PRs for the master branch.

It would be interesting to see if it would be possible to use Mermaid flowcharts to generate marble diagrams. They are text based and can be used with Markdown and Sphinx. I haven't tried making marble diagrams, and they won't (currently) render directly on GitHub, but they render e.g on HackMD. But I cannot find a way to render to ascii, so it would not be possible to use inside the code doc. Perhaps plain ascii diagrams are simpler and better.

[MainRo]

Status update: I enabled the Guzzle theme, the outline should be complete. Now I will work on the content of the missing parts.
Once this is done I will look at the marble diagrams.

[jcafhe]

I have some questions regarding the convention we should use(or not) for docstrings, especially in examples. It is not that serious but there are actually multiple ways for naming input/output observables:
Concerning rx

>>> res =  ...
>>> obs = ...
>>> merged = ...
>>> rx.merge(obs1, obs2)
>>> rx.merge(xs, ys)

Concerning rx.operators

>>> res = ...
>>> op = ...

When I have the opportunity to update a docstring, I use res = because it seems to be the most common and rx.merge(obs1, obs2). For rx.operators I use op = ... for denoting that the return value is not an observable but a function.
What do you think ?

[MainRo]

I played with a marble diagram generator from text and have a first implementation running. It is available here:
https://github.com/MainRo/dooble

it is also published on pypi. there is a map example in "examples/map.txt"

Do not focus on the visual aspects yet. I am confident that we can easily generate diagrams this way. I think I also found a way to describe higher order observable in an easy way. The tool is based on matplotlib, which allows to generate better marble diagrams than what could be done with mermaid diagrams (in my opinion)

generate a png with:

dooble --input examples/map.txt --output /tmp/map.png

what are the opinions on that ?

[jcafhe]

This is great! Now I'm curious to see how a higher order observable would look like.

A few comments:

• in your example, you should append an 's' to example in path examples/map.txt:
  dooble --input examples/map.txt --output /tmp/map.png
• have you considered to make an entry point instead of a shell script. In fact, in windows I didn't manage to make it work by calling dooble or python dooble in cli. However, it's a bit more tedious to set up.

[MainRo]

I changed the script with an entry point. Tell me if installation works better on windows.
I also added a support for higher order observables. I use the '+' character as an anchor point between the parent and the child: A parent emits a '+' item to indicate that it emits an observable. A child starts with a '+' to indicate that it is a continuation of the parent. The parent and the child must have the anchor at the same position to be associated.
I added an example with the window operator which is:

--a-b-c---d-e-f-->
[     window     ]
--+-------+------>
          +d-e-f-|
  +a-b-c-|

We do not draw the vertical links on the text because this is tedious and would break the ebnf based grammar, but they are generated on the diagram. Keeping in mind that this is for documentation with simple diagrams, the text representation seems still readable to me.

[jcafhe]

I changed the script with an entry point. Tell me if installation works better on windows.

Yes, it's working perfectly, thank you 👍 .
I played with window example and this is fine. Great work!

[MainRo]

Now that the content of the readme is - almost - completely in the documentation, I will strip it. I have two questions on that:

• I would like to change the format from md to rst so that it can be displayed on pypi.
• Should I reference the doc with relative paths to github, absolute paths to github or a placeholder where the doc will be published ? For now I would use absolute paths to github, and later reference the published doc.

[MainRo]

I still have some paragraphs to complete, but the overall documentation is in good shape. @dbrattli do you know if there is a way to automatically publish it somewhere on reactivex.io ? With travis we can encrypt credentials so that the CI automatically update the doc and nobody needs access to the credentials. I made something like this on a private gitlab-ci that could be relevant here:

• All commits on master trigger a new doc build being published on site-url/master.
• All tags creation trigger a new doc build being published on site-url/[tag].
• site-url/lastest is linked to the most recent tag url (site-url/latest -> site-url/[tag]).

I can work on the travis part if needed, provided that there is a way to publish via tooling.

[erikkemperman]

Does anyone know why sphinx doesn't crosslink properly? For example:
https://rxpy.readthedocs.io/en/latest/reference_observable_factory.html

These methods have return type Observable, and I would expect to be able to click on that to go the Observable class.

[erikkemperman]

About those failing crosslinks -- I just quickly tried what happened if I use the bundled sphinx-apidoc script to generate rst files, and it turns out that the crosslinks then just work...

Please note, I'm not proposing to do this necessarily, because it will just create a "dry" package/subpackage index at the upper levels, not the nicely decorated "topical" structure there is now under Reference.

Still, perhaps it might help in tracking down why the crosslinks don't work currently? To me it seems like kind of a big deal to get those working!

erikkemperman@4c6c8bd

[MainRo]

strange. sphinx-apidoc seems to manually create rst files. So we should have the same results than with sphinx-build.

There are old references to this issue here:
sphinx-doc/sphinx#1059
https://github.com/agronholm/sphinx-autodoc-typehints/issues/15

So it's supposed to work here. I tried adding the parameters set by sphinx-apidoc in the rst, and setting the napoleon "use_param" option without success.

[erikkemperman]

Hm, never noticed this before: if there are docstrings for both class and __init__ then sphinx only uses the first. So for example, AsyncSubject has both,

RxPY/rx/subjects/asyncsubject.py

Lines 13 to 15 in e5fb1dc

	"""Represents the result of an asynchronous operation. The last value
	before the close notification, or the error received through
	on_error, is sent to all subscribed observers."""

RxPY/rx/subjects/asyncsubject.py

Lines 18 to 19 in e5fb1dc

	"""Creates a subject that can only receive one value and that value is
	cached for all future observations."""

But the generated docs only have the first section:
https://rxpy.readthedocs.io/en/latest/reference_subject.html#rx.subjects.AsyncSubject

So the init function is not actually documented! This behaviour can change this with a single sphinx setting, but I just wanted to check if this might be deliberate (i.e. that we were only supposed to put docstrings in either class or __init__(), not both).

[MainRo]

Wow nice catch! I think that documenting in both class and constructor makes sense: The class documentation contains information on the class itself, while init is specifically for the constructor.

[MainRo]

closing this issue now that v3 is released. We will iterate on the doc with dedicated issues and PR.