Skip to content

Latest commit

 

History

History
16 lines (12 loc) · 697 Bytes

writedataframetos3.md

File metadata and controls

16 lines (12 loc) · 697 Bytes
Raw

Documentation Home | AWS Home

Write DataFrame to S3 Parquet

This step group pipeline will write a DataFrame an S3 location. This step group is designed to work with the LoadToParquet pipeline.

General Information

Id: a8f1acd0-c39b-11eb-b944-4f8822efc9f5

Name: WriteDataFrameToS3

Required Parameters

Required parameters indicated with a *:

  • bronzeZonePath * - The HDFS path for the root bronze zone folder.
  • fileId * - The unique id for the file that is processed.
  • credentialName * - The name of the Secrets Manager key containing the key/secret to use when writing data.