Approach with store
on shared folder
#16
Replies: 1 comment
-
I created What I think may help more is some sort of content-addressable storage (CAS) as discussed at ropensci/targets#1232 (comment). As described in the thread, this may be developed at some point by the folks at rOpenSci.
You can skip (1). I am less sure about
Yes, that is the recommended workflow. |
Beta Was this translation helpful? Give feedback.
-
Hi, my teams has been using the
targetopia
for a variety of projects. Currently we usetargets
,tarchetypes
,crew
,crew.cluster
andgeotargets
. We utilize an internally maintained data storage and slurm managed cluster. For projects with multiple members contributing to the code base for a large pipeline, we thinkgittargets
could be an important piece. I have a few questions and comments about the recommended practices here.We are utilizing a shared file path set in the
_targets.yaml
such asmain: store: /common/path/for/us/all
So, if I understand correctly, a good use case of
gittargets
would be for the team members to work on their local versions of the pipeline that has the_targets/objects
to the common store, and then to usegittargets
to work on separate branches, so that themain
branch is not changed everytime someone makes a change. Can you confirm if that is a good use case?If that is the case, then I have a question about setting it up for what is currently an active
targets
pipeline. Given the recommended workflow, it seems we can skip (1) since the pipeline has run and the data store exists. We also have a git repo going already, so does that mean we can skip2. tar_git_init()
or is it doing something to connecttargets
togit
that is required for proper functionality? If not, perhaps we should runtar_git_init(git_lfs = FALSE)
since we don't want to commit the target objects.Lastly, as a point of clarification, it looks like we are intended to still use the standard git procedures for managing the code base and then just call
tar_git_snapshot()
occasionally to add a group of commits to thegittargets
snapshot, correct?Beta Was this translation helpful? Give feedback.
All reactions