Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add S3 support to all input/output files #23

Open
junkert opened this issue Mar 19, 2019 · 1 comment
Open

Add S3 support to all input/output files #23

junkert opened this issue Mar 19, 2019 · 1 comment
Assignees
Labels
enhancement New feature or request

Comments

@junkert
Copy link
Collaborator

junkert commented Mar 19, 2019

We need to support AWS-Lambda which will allow Gonymizer to run as a serverless application. To do this we need to be able to store all files in S3 (not just the processed file from the dump command and row_counts file from the dump and load comma). This should be pretty easy to implement since we already have the S3 functions in s3.go

@junkert junkert added the enhancement New feature or request label Mar 19, 2019
@junkert junkert self-assigned this Mar 19, 2019
@junkert junkert changed the title A Add S3 support to all input/output files Mar 19, 2019
@junkert
Copy link
Collaborator Author

junkert commented Jun 20, 2019

We can use tmp space in lambda jobs. See: https://forums.aws.amazon.com/thread.jspa?threadID=174119

Instead, I found that file storage should be generalized to use either local or s3 storage depending on the path the user supplies on the CLI interface.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant