Integrate incremental push #669
Conversation
…ion (`IFilesystem`).
…o handle the no-snapshots case.
Change push to first retrieve the snapshots that exist remotely. Supply this information to the local filesystem reader so it can construct an incremental stream. Update the filesystem writer so it is capable of receiving incremental streams into the filesystem. Push has become asynchronous - so propagate Deferred-handling code up the stack from it.
|
Note that as buildbot points out, this is not really ready to be merged. Review comments apart from "it crashes the buildslaves" nevertheless welcome. 😉 |
|
I skimmed through the branch. The code and docstrings make sense to me. I guess the |
|
|
||
| A blocking API, for now. | ||
|
|
||
| The returned file-like object will be closed by this object. | ||
|
|
||
| :param list remote_snapshots: The snapshots which are available on the |
There was a problem hiding this comment.
Doesn't mention None as default argument. Might be simpler to just have input be an ordered iterable and then you can have tuple as default and remove a code branch.
|
All the comments above are pretty minimal, so I'd be happy with it to be merged if buildbot were happy. |
|
Once the comments are addressed, that is. |
|
in some manual testing just now, I had this error when moving an application: Error: Could not create local directory '/flocker/f28b1950-7783-4121-aeaf-55fa6bf54b2b.mongodb-volume-example' for bind mount: mkdir /flocker/f28b1950-7783-4121-aeaf-55fa6bf54b2b.mongodb-volume-example: read-only file system! |
adamtheturtle
force-pushed
the
master
branch
from
892883b
to
7fa3ba7
Compare
|
Actually... is there documentation of incremental push in the volume manager docs? it should be. |
I think this was a consequence of creating the flocker pool and volume with an older version of the code and then upgrading to the read-only mounts version of the code and trying to re-use the same setup. After I started with a fresh pool everything worked fine for a large number of pushes. |
|
The build is failing but only because of a (pre-existing) problem with the way the spellchecker is run. Adam is looking into that separately (#724) so I'm merging this now. |
This is the last part of #46. Thus, it actually fixes #46.
It depends on #658. Review that first.
This change takes the two pieces built previously, ability to discover what snapshots exist remotely and ability to generate incremental data streams given information about remote snapshots and ties them together with the push implementation. It makes push retrieve the snapshots and feed them into the data stream generator.