Improve documentation and add distributed.rst #84
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jytug
requested changes
May 31, 2017
| Modes | ||
| ===== | ||
|
|
||
| There are two modes to control the way calculations are distributed: master-worker mode and process-group mode. |
There was a problem hiding this comment.
the m-w mode and the p-g mode
| There are two modes to control the way calculations are distributed: master-worker mode and process-group mode. | ||
| Each of them is designed to resemble interfaces and APIs familiar to PyTorch users. | ||
|
|
||
| The process-group mode API is made to look like the MPI API. |
There was a problem hiding this comment.
I don't think the MPI API is a great reference in a tutorial - maybe a little eplanation of the characteristics instead?
|
|
||
| .. TODO: code examples | ||
|
|
||
| The master-worker mode on the other hand is dedicated to the users familiar with Nvidia CUDA. |
There was a problem hiding this comment.
I think it's dedicated to users familiar with PyTorch's CUDA API
|
|
||
| The master-worker mode on the other hand is dedicated to the users familiar with Nvidia CUDA. | ||
| The API is simple and makes it easy to start using it right away. | ||
| This model does not scale well due to the bottleneck in the master node which is responsible for all the planning job queuing in all the worker nodes. |
There was a problem hiding this comment.
does not scale well -> is not designed for great scalability
?
| Distributed computing | ||
| ===================== | ||
|
|
||
| In order to configure connections for a program using THD it is required to set a couple of environmental variables. |
There was a problem hiding this comment.
I think THD is the C++ library, which doesn't concern the users
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes rebase madness (#76).
TODO: