Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FileDFG should just use tar files (deprecate folders) #306

Closed
GearsAD opened this issue Feb 16, 2020 · 3 comments
Closed

FileDFG should just use tar files (deprecate folders) #306

GearsAD opened this issue Feb 16, 2020 · 3 comments
Labels
Milestone

Comments

@GearsAD
Copy link
Collaborator

GearsAD commented Feb 16, 2020

This would simplify the code and would lead to less user issues.

@GearsAD GearsAD added this to the v0.6.x milestone Feb 16, 2020
@dehann
Copy link
Member

dehann commented Feb 16, 2020

another thing I ran into is when the tar gets unpacked, it drops the folder in common location like /tmp/caesar/random, which is a problem if two processes are untarring files with the same name. Maybe just untar to unique folder in common location instead?

There should be code in there to delete the untarred folder after loading the fg object, otherwise the number of files just becomes crazy high in a short amount of time.

@GearsAD
Copy link
Collaborator Author

GearsAD commented Feb 16, 2020

Yep, we should also consider Tar.jl for tarring/untarring.

@dehann
Copy link
Member

dehann commented Apr 27, 2020

I fixed the common location thing at some point with temporary uuid folder names.

@dehann dehann modified the milestones: v0.7.x, v0.7.0 Apr 27, 2020
@dehann dehann closed this as completed Apr 27, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants