datalad-annex:: Test for a repository deposit before establishing mirrorrepo
#638
Conversation
|
The current implementation is not yet optimal: Initial push to a not yet existing repo deposit Initial datalad push The latter fails in the git-push, even though that git-push succeeds when ran on its own. |
As of now, the local side seems OK (DataLad produces an understandable error message): But on the remote side (in Sciebo), So I'd say it is not fixed, or partially fixed. |
|
I found one of the issues that are a problem here. In except mod_push.CommandError as e:
# it is OK if the remote doesn't have a git-annex branch yet
# (e.g. fresh repo)
# TODO is this possible? we just copied? Maybe check if anything
# was actually copied?
if "fatal: couldn't find remote ref git-annex" not in e.stderr.lower():
raise
lgr.debug('Remote does not have a git-annex branch: %s', e)The messaging from git that it expects is not happening, because the actual message is coming from I believe that this check needs to be replaced. Not sure how to handle that yet. It would be useful to stop there in cases where something other than a missing ref causes an error exit. Maybe the test is shortened to |
Previously, one had to call `git-clone/fetch` directly with some `-v` options in order to trigger verbose output of the remote helper. With this change a DataLad log level of 'debug' (or more verbose), also triggers verbose output in the helper. This helps with accessibility of this feature for internal Git calls. Note that due to a limitation of the `datalad -l` CLI option handling the log-level is not propagated to subprocesses. This only works for env specification ala DATALAD_LOG_LEVEL=debug
Codecov ReportAttention:
Additional details and impacted files@@ Coverage Diff @@
## main #638 +/- ##
==========================================
+ Coverage 92.67% 93.05% +0.38%
==========================================
Files 171 171
Lines 11564 11945 +381
Branches 1781 1806 +25
==========================================
+ Hits 10717 11116 +399
+ Misses 657 642 -15
+ Partials 190 187 -3 ☔ View full report in Codecov by Sentry. |
There was a problem hiding this comment.
Thank you for investigating this. I tried clone, and I like the improved error message and the fact that an empty repo is not created.
The commit message for e415687 says that it would also close #373 but as far as I understand the webdav case still warrants investigation (which I cannot commit to at the moment), as clone would still create an empty directory on the webdav end.
|
Thanks for catching that! |
The `mirrorrepo` property of the datalad-annex Git remote helper implementation creates an empty repository, if the remote has no refs (or does not exist). This is sensible behavior for enabling initial pushes. However, it leads to successful but empty clones/fetches for any misspecified URL or connectivity failure. This is undesirable. This change adds an explicit test for an available/accessible repository deposit, whenever a clone/fetch is performed. Wrong URL specifications or connectivity issues will now lead to an explicit error exit. This fact is communicated via a non-verbose message, and the full error is available at verbosity level 2, i.e. with a `git clone -v`. Closes: dataladgh-636
The
mirrorrepoproperty of the datalad-annex Git remote helper implementation creates an empty repository, if the remote has no refs (or does not exist). This is sensible behavior for enabling initial pushes.However, it leads to successful but empty clones/fetches for any misspecified URL or connectivity failure. This is undesirable.
This change adds an explicit test for an available/accessible repository deposit, whenever a clone/fetch is performed. Wrong URL specifications or connectivity issues will now lead to an explicit error exit. This fact is communicated via a non-verbose message, and the full error is available at verbosity level 2, i.e. with a
git clone -v.datalad-annex::git remote helper #636TODO