Add documentation about how horovod works

[pescap]

Here.

It could be interesting to add bibliography (from academia, and blogs (Medium, etc...).

Try to find the best articles to understand Horovod, how it works (technically), its structure.

I suggest to list the references, for example:

Documentation

Meet Horovod: Uber’s Open Source Distributed Deep Learning Framework for TensorFlow
Add a summary of what matters/you learnt or understood in this article

Distributed Deep Learning with Horovod
summary

and so on.

You can start with these references (and add your notes gradually):

https://towardsdatascience.com/distributed-deep-learning-with-horovod-2d1eea004cb2
https://arxiv.org/pdf/1706.02677.pdf
Horovod Article: https://arxiv.org/pdf/1802.05799.pdf
Uber presentation: https://developer.download.nvidia.com/video/gputechconf/gtc/2019/presentation/s9321-distributed-deep-learning-with-horovod.pdf

Add more references if you find more.

Important. Read: lululxvi/deepxde#39

[pescap]

Horovod accelerated PINNs: NeuralSolvers.
Read the paper, comment it, and understand the code. We aim to implement the same ideas with DeepXDE.

Also has MirroredStrategy: https://github.com/tensordiffeq/TensorDiffEq

Read: https://eng.uber.com/michelangelo-machine-learning-platform/

[Ehaw04]

[Uber Open Summit 2018] Horovod: Deep Learning distribuido en 5 líneas de Python:

https://www.youtube.com/watch?v=4y0TDK3KoCA&t=900s&ab_channel=UberEngineering

[github-actions]

This issue is stale because it has been open for 7 days with no activity.

[github-actions]

This issue was closed because it has been inactive for 7 days since being marked as stale.

[pescap]

#58

[github-actions]

This issue was closed because it has been inactive for 7 days since being marked as stale.