Skip to content

Commit

Permalink
Python: Clarify docs on environment variables (#165)
Browse files Browse the repository at this point in the history
  • Loading branch information
Fokko committed Nov 29, 2023
1 parent a22cba5 commit cbba413
Showing 1 changed file with 8 additions and 4 deletions.
12 changes: 8 additions & 4 deletions mkdocs/docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,13 +32,19 @@ There are three ways to pass in configuration:
- Through environment variables
- By passing in credentials through the CLI or the Python API

The configuration file is recommended since that's the most transparent way. If you prefer environment configuration:
The configuration file is recommended since that's the easiest way to manage the credentials.

Another option is through environment variables:

```sh
export PYICEBERG_CATALOG__DEFAULT__URI=thrift://localhost:9083
export PYICEBERG_CATALOG__DEFAULT__S3__ACCESS_KEY_ID=username
export PYICEBERG_CATALOG__DEFAULT__S3__SECRET_ACCESS_KEY=password
```

The environment variable picked up by Iceberg starts with `PYICEBERG_` and then follows the yaml structure below, where a double underscore `__` represents a nested field.
The environment variable picked up by Iceberg starts with `PYICEBERG_` and then follows the yaml structure below, where a double underscore `__` represents a nested field, and the underscore `_` is converted into a dash `-`.

For example, `PYICEBERG_CATALOG__DEFAULT__S3__ACCESS_KEY_ID`, sets `s3.access-key-id` on the `default` catalog.

## FileIO

Expand Down Expand Up @@ -80,8 +86,6 @@ For the FileIO there are several configuration options available:

### Azure Data lake

### Azure Data lake

| Key | Example | Description |
| ----------------------- | ----------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| adlfs.connection-string | AccountName=devstoreaccount1;AccountKey=Eby8vdM02xNOcqF...;BlobEndpoint=http://localhost/ | A [connection string](https://learn.microsoft.com/en-us/azure/storage/common/storage-configure-connection-string). This could be used to use FileIO with any adlfs-compatible object storage service that has a different endpoint (like [azurite](https://github.com/azure/azurite)). |
Expand Down

0 comments on commit cbba413

Please sign in to comment.