Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Throw error when building Iceberg tables without behavior flag set #1184

Merged
merged 5 commits into from
Sep 18, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .changes/unreleased/Under the Hood-20240917-181147.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
kind: Under the Hood
body: Change behavior flag semantics to log iceberg flag warnings..
time: 2024-09-17T18:11:47.525026-07:00
custom:
Author: versusfacit
Issue: "321"
13 changes: 12 additions & 1 deletion dbt/adapters/snowflake/impl.py
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,18 @@ class SnowflakeAdapter(SQLAdapter):

@property
def _behavior_flags(self) -> List[BehaviorFlag]:
return [{"name": "enable_iceberg_materializations", "default": False}]
return [
{
"name": "enable_iceberg_materializations",
"default": False,
"description": (
"Enabling Iceberg materializations introduces latency to metadata queries, "
"specifically within the list_relations_without_caching macro. Since Iceberg "
"benefits only those actively using it, we've made this behavior opt-in to "
"prevent unnecessary latency for other users."
),
}
]

@classmethod
def date_function(cls):
Expand Down
5 changes: 5 additions & 0 deletions dbt/include/snowflake/macros/relations/table/create.sql
Original file line number Diff line number Diff line change
@@ -1,4 +1,9 @@
{% macro snowflake__create_table_as(temporary, relation, compiled_code, language='sql') -%}

{%- if relation.is_iceberg_format and not adapter.behavior.enable_iceberg_materializations.no_warn %}
{% do exceptions.raise_compiler_error('Was unable to create model as Iceberg Table Format. Please set the `enable_iceberg_materializations` behavior flag to True in your dbt_project.yml. For more information, go to <url pending>.') %}
{%- endif %}

{%- set materialization_prefix = relation.get_ddl_prefix_for_create(config.model.config, temporary) -%}
{%- set alter_prefix = relation.get_ddl_prefix_for_alter() -%}

Expand Down