Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can we manage multiple catalogs ? #40

Open
quydx opened this issue Aug 16, 2023 · 1 comment
Open

Can we manage multiple catalogs ? #40

quydx opened this issue Aug 16, 2023 · 1 comment

Comments

@quydx
Copy link

quydx commented Aug 16, 2023

When using Jdbc Catalog, I can have many catalogs as I want. Each catalog contains its own namespaces and tables.

image

But with REST catalog, I wonder if I can do the same thing like JDBC Catalog. I can't find any API to create catalog and show all catalogs.
Can someone give me an ideas?
Thank!

@dupen01
Copy link

dupen01 commented Oct 29, 2023

To achieve this goal, here's what I did: I changed the name of the rest catalog to be obtained from the environment variable, and then built the rest server as a docker image. If I need to manage multiple different catalogs, I can start multiple rest servers containers using different environment variables, and then my client can connect to anyone of them, thereby achieving the purpose of managing multiple catalogs. I hope this will help you.

...
private static final String ENV_REST_CATALOG_NAME = "REST_CATALOG_NAME";
String catalogName = System.getenv().get(ENV_REST_CATALOG_NAME);
        LOG.info("REST Catalog Name: {}", catalogName);
        return CatalogUtil.buildIcebergCatalog(catalogName, catalogProperties, new Configuration());
docker run -id \
-p 8181:8181 \
--name ice-rest \
--net lakenet \
--ip 192.168.10.16 \
-e REST_CATALOG_NAME=rest \
-e CATALOG_IO__IMPL=org.apache.iceberg.aws.s3.S3FileIO \
-e CATALOG_WAREHOUSE=s3://lakehouse/iceberg/warehouse \
-e CATALOG_S3_ENDPOINT=http://192.168.10.12:9000 \
-e CATALOG_URI=jdbc:mysql://192.168.10.11:3306/iceberg \
...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants