Skip to content

Latest commit

 

History

History
executable file
·
324 lines (253 loc) · 32.9 KB

CHANGELOG.md

File metadata and controls

executable file
·
324 lines (253 loc) · 32.9 KB
Raw

dbt Core Changelog

  • This file provides a full account of all changes to dbt-core and dbt-postgres
  • Changes are listed under the (pre)release in which they first appear. Subsequent releases include changes from previous releases.
  • "Breaking changes" listed under a version may require action from end users or external maintainers when upgrading to that version.
  • Do not edit this file directly. This file is auto-generated using changie. For details on how to document a change, see the contributing guide

dbt-core 1.0.8 - June 15, 2022

Under the Hood

  • Update context readme + clean up context code" (#4796, #5334)

dbt-core 1.0.7 - May 20, 2022

Dependencies

  • Lowering networkx dependency range due to new version's breaking change (#5254, #5280)

dbt-core 1.0.6 - April 27, 2022

Fixes

  • Use yaml renderer (with target context) for rendering selectors (#5131, #5136)
  • Fix retry logic to return values after initial try (#5023, #5137)
  • Scrub secret env vars from CommandError in exception stacktrace (#5151, #5152)

Under the Hood

  • Move package deprecation check outside of package cache (#5068, #5069)

dbt-core 1.0.5 - April 20, 2022

Fixes

  • Fix bug causing empty node level meta, snapshot config errors (#4459, #4726)
  • Support click versions in the v7.x series (#4566, #4681)
  • Fixed a bug where nodes that depend on multiple macros couldn't be selected using -s state:modified (#4678, #4820)
  • Catch all Requests Exceptions on deps install to attempt retries. Also log the exceptions hit. (#4849, #4865)
  • Fix partial parsing bug with multiple snapshot blocks (#4771, #4773)
  • Use cli_vars instead of context to create package and selector renderers (#4876, #4878)
  • Catch more cases to retry package retrieval for deps pointing to the hub. Also start to cache the package requests. (#4849, #4982)
  • Relax minimum supported version of MarkupSafe (#4745, #5039)

Under the Hood

  • Automate changelog generation with changie (#4652, #4743)
  • Fix broken links for changelog generation and tweak GHA to only post a comment once when changelog entry is missing (#4848, #4857)

Docs

  • Resolve errors related to operations preventing DAG from generating in the docs. Also patch a spark issue to allow search to filter accurately past the missing columns. (#4578, #4763)
  • backporting performance regression testing readme (#4904, #5042)

Contributors

dbt-core 1.0.4 - March 18, 2022

Fixes

  • Depend on new dbt-extractor version with fixed GitHub links to resolve Homebrew installation issues (#4891, #4890)

dbt-core 1.0.3 (February 21, 2022)

Fixes

  • Fix bug accessing target fields in deps and clean commands (#4752, #4758)

dbt-core 1.0.2 (February 18, 2022)

Dependencies

  • Pin MarkupSafe==2.0.1. Deprecation of soft_unicode in MarkupSafe==2.1.0 is not supported by Jinja2==2.11

dbt-core 1.0.2rc1 (February 4, 2022)

Fixes

  • Projects created using dbt init now have the correct seeds directory created (instead of data) (#4588, #4599)
  • Don't require a profile for dbt deps and clean commands (#4554, #4610)
  • Select modified.body works correctly when new model added(#4570, #4631)
  • Fix bug in retry logic for bad response from hub and when there is a bad git tarball download. (#4577, #4579, #4609)
  • Restore previous log level (DEBUG) when a test depends on a disabled resource. Still WARN if the resource is missing (#4594, #4647)
  • User wasn't asked for permission to overwite a profile entry when running init inside an existing project (#4375, #4447)
  • A change in secret environment variables won't trigger a full reparse #4650 4665
  • adapter compability messaging added(#4438 #4565)
  • Add project name validation to dbt init (#4490,#4536)

Contributors:

dbt-core 1.0.1 (January 03, 2022)

dbt-core 1.0.1rc1 (December 20, 2021)

Fixes

  • Fix wrong url in the dbt docs overview homepage (#4442)
  • Fix redefined status param of SQLQueryStatus to typecheck the string which passes on ._message value of AdapterResponse or the str value sent by adapter plugin. (#4463)
  • Fix DepsStartPackageInstall event to use package name instead of version number. (#4482)
  • Reimplement log message to use adapter name instead of the object method. (#4501)
  • Issue better error message for incompatible schemas (#4470, #4497)
  • Remove secrets from error related to packages. (#4507)
  • Prevent coercion of boolean values (True, False) to numeric values (0, 1) in query results (#4511, #4512)
  • Fix error with an env_var in a project hook (#4523, #4524)

Docs

  • Fix missing data on exposures in docs (#4467)

Contributors:

dbt-core 1.0.0 (December 3, 2021)

Fixes

  • Configure the CLI logger destination to use stdout instead of stderr (#4368)
  • Make the size of EVENT_HISTORY configurable, via EVENT_BUFFER_SIZE global config (#4411, #4416)
  • Change type of log_format in profiles.yml user config to be string, not boolean (#4394)

Under the hood

  • Only log cache events if LOG_CACHE_EVENTS is enabled, and disable by default. This restores previous behavior (#4369)
  • Move event codes to be a top-level attribute of JSON-formatted logs, rather than nested in data (#4381)
  • Fix failing integration test on Windows (#4380)
  • Clean up warning messages for clean + deps (#4366)
  • Use RFC3339 timestamps for log messages (#4384)
  • Different text output for console (info) and file (debug) logs (#4379, #4418)
  • Remove unused events. More structured ConcurrencyLine. Replace \n message starts/ends with EmptyLine events, and exclude EmptyLine from JSON-formatted output (#4388)
  • Update events module README (#4395)
  • Rework approach to JSON serialization for events with non-standard properties (#4396)
  • Update legacy logger file name to dbt.log.legacy (#4402)
  • Rollover dbt.log at 10 MB, and keep up to 5 backups, restoring previous behavior (#4405)
  • Use reference keys instead of full relation objects in cache events (#4410)
  • Add node_type contextual info to more events (#4378)
  • Make materialized config optional in node_type (#4417)
  • Stringify exception in GenericExceptionOnRun to support JSON serialization (#4424)
  • Add "interop" tests for machine consumption of structured log output (#4327)
  • Relax version specifier for dbt-extractor to ~=0.4.0, to support compiled wheels for additional architectures when available (#4427)

dbt-core 1.0.0rc3 (November 30, 2021)

Fixes

  • Support partial parsing of env_vars in metrics (#4253, #4322)
  • Fix typo in UnparsedSourceDefinition.__post_serialize__ (#3545, #4349)

Under the hood

  • Change some CompilationExceptions to ParsingExceptions (#4254, #4328)
  • Reorder logic for static parser sampling to speed up model parsing (#4332)
  • Use more augmented assignment statements (#4315), (#4311)
  • Adjust logic when finding approximate matches for models and tests (#3835), #4076)
  • Restore small previous behaviors for logging: JSON formatting for first few events; WARN-level stdout for list task; include tracking events in dbt.log (#4341)

Contributors:

dbt-core 1.0.0rc2 (November 22, 2021)

Breaking changes

  • Restrict secret env vars (prefixed DBT_ENV_SECRET_) to profiles.yml + packages.yml only. Raise an exception if a secret env var is used elsewhere (#4310, #4311)
  • Reorder arguments to config.get() so that default is second (#4273, #4297)

Features

  • Avoid error when missing column in YAML description (#4151, #4285)
  • Allow --defer flag to dbt snapshot (#4110, #4296)
  • Install prerelease packages when version explicitly references a prerelease version, regardless of install-prerelease status (#4243, #4295)
  • Add data attributes to json log messages (#4301)
  • Add event codes to all log events (#4319)

Fixes

  • Fix serialization error with missing quotes in metrics model ref (#4252, #4287)
  • Correct definition of 'created_at' in ParsedMetric nodes (#4298, #4299)

Fixes

  • Allow specifying default in Jinja config.get with default keyword (#4273, #4297)
  • Fix serialization error with missing quotes in metrics model ref (#4252, #4287)
  • Correct definition of 'created_at' in ParsedMetric nodes (#4298, #4299)

Under the hood

  • Add --indirect-selection parameter to profiles.yml and builtin DBT_ env vars; stringified parameter to enable multi-modal use (#3997, #4270)
  • Fix filesystem searcher test failure on Python 3.9 (#3689, #4271)
  • Clean up deprecation warnings shown for dbt_project.yml config renames (#4276, #4291)
  • Fix metrics count in compiled project stats (#4290, #4292)
  • First pass at supporting more dbt tasks via python lib (#4200)

Contributors:

dbt-core 1.0.0rc1 (November 10, 2021)

Breaking changes

  • Replace greedy flag/property for test selection with indirect_selection: eager/cautious flag/property. Set to eager by default. Note: This reverts test selection to its pre-v0.20 behavior by default. dbt test -s my_model will select multi-parent tests, such as relationships, that depend on unselected resources. To achieve the behavior change in v0.20 + v0.21, set --indirect-selection=cautious on the CLI or indirect_selection: cautious in yaml selectors. (#4082, #4104)
  • In v1.0.0, pip install dbt will raise an explicit error. Instead, please use pip install dbt-<adapter> (to use dbt with that database adapter), or pip install dbt-core (for core functionality). For parity with the previous behavior of pip install dbt, you can use: pip install dbt-core dbt-postgres dbt-redshift dbt-snowflake dbt-bigquery (#4100, #4133)
  • Reorganize the global_project (macros) into smaller files with clearer names. Remove unused global macros: column_list, column_list_for_create_table, incremental_upsert (#4154)
  • Introduce structured event interface, and begin conversion of all legacy logging (#3359, #4055)
    • This is a breaking change for adapter plugins, requiring a very simple migration. See events module README for details.
    • If you maintain another kind of dbt-core plugin that makes heavy use of legacy logging, and you need time to cut over to the new event interface, you can re-enable the legacy logger via an environment variable shim, DBT_ENABLE_LEGACY_LOGGER=True. Be advised that we will remove this capability in a future version of dbt-core.

Features

  • Allow nullable error_after in source freshness (#3874, #3955)
  • Add metrics nodes (#4071, #4235)
  • Add support for dbt init <project_name>, and support for skip_profile_setup argument (dbt init -s) (#4156, #4249)

Fixes

  • Changes unit tests using assertRaisesRegexp to assertRaisesRegex (#4136, #4136)
  • Allow retries when the answer from a dbt deps is None (#4178, #4225)

Docs

  • Fix non-alphabetical sort of Source Tables in source overview page (docs#81, docs#218)
  • Add title tag to node elements in tree (docs#202, docs#203)
  • Account for test rename: schema → generic, data → singular. Use test_metadata instead of schema/data tags to differentiate (docs#216, docs#222)
  • Add metrics (core#216, docs#223)

Under the hood

  • Bump artifact schema versions for 1.0.0: manifest v4, run results v4, sources v3. Notable changes: added metrics nodes; schema test + data test nodes are renamed to generic test + singular test nodes; freshness threshold default values (#4191)
  • Speed up node selection by skipping incorporate_indirect_nodes if not needed (#4213, #4214)
  • When on_schema_change is set, pass common columns as dest_columns in incremental merge macros (#4144, #4170)
  • Clear adapters before registering in lib module config generation (#4218)
  • Remove official support for python 3.6, which is reaching end of life on December 23, 2021 (#4134, #4223)

Contributors:

dbt-core 1.0.0b2 (October 25, 2021)

Breaking changes

  • Enable on-run-start and on-run-end hooks for dbt test. Add flags.WHICH to execution context, representing current task (#3463, #4004)

Features

  • Normalize global CLI arguments/flags (#2990, #3839)
  • Turns on the static parser by default and adds the flag --no-static-parser to disable it. (#3377, #3939)
  • Generic test FQNs have changed to include the relative path, resource, and column (if applicable) where they are defined. This makes it easier to configure them from the tests block in dbt_project.yml (#3259, #3880
  • Turn on partial parsing by default (#3867, #3989)
  • Add result:<status> selectors to automatically rerun failed tests and erroneous models. This makes it easier to rerun failed dbt jobs with a simple selector flag instead of restarting from the beginning or manually running the dbt models in scope. (#3859, #4017)
  • dbt init is now interactive, generating profiles.yml when run inside existing project (#3625)

Under the hood

  • Fix intermittent errors in partial parsing tests (#4060, #4068)
  • Make finding disabled nodes more consistent (#4069, #4073)
  • Remove connection from render_with_context during parsing, thereby removing misleading log message (#3137, #4062)
  • Wait for postgres docker container to be ready in setup_db.sh. (#3876, #3908)
  • Prefer macros defined in the project over the ones in a package by default (#4106, #4114)
  • Dependency updates (#4079), (#3532
  • Schedule partial parsing for SQL files with env_var changes (#3885, #4101)
  • Schedule partial parsing for schema files with env_var changes (#3885, #4162)
  • Skip partial parsing when env_vars change in dbt_project or profile (#3885, #4212)

Contributors:

dbt-core 1.0.0b1 (October 11, 2021)

Breaking changes

  • The two type of test definitions are now "singular" and "generic" (instead of "data" and "schema", respectively). The test_type: selection method accepts test_type:singular and test_type:generic. (It will also accept test_type:schema and test_type:data for backwards compatibility) (#3234, #3880). Not backwards compatible: The --data and --schema flags to dbt test are no longer supported, and tests no longer have the tags 'data' and 'schema' automatically applied.
  • Deprecated the use of the packages arg adapter.dispatch in favor of the macro_namespace arg. (#3895)

Features

  • Normalize global CLI arguments/flags (#2990, #3839)
  • Turns on the static parser by default and adds the flag --no-static-parser to disable it. (#3377, #3939)
  • Generic test FQNs have changed to include the relative path, resource, and column (if applicable) where they are defined. This makes it easier to configure them from the tests block in dbt_project.yml (#3259, #3880
  • Turn on partial parsing by default (#3867, #3989)
  • Generic test can now be added under a generic subfolder in the test-paths directory. (#4052)

Fixes

  • Add generic tests defined on sources to the manifest once, not twice (#3347, #3880)
  • Skip partial parsing if certain macros have changed (#3810, #3982)
  • Enable cataloging of unlogged Postgres tables (3961, #3993)
  • Fix multiple disabled nodes (#4013, #4018)
  • Fix multiple partial parsing errors (#3996, #4020)
  • Return an error instead of a warning when runing with --warn-error and no models are selected (#4006, #4019)
  • Fixed bug with error_if test option (#4070)

Under the hood

  • Enact deprecation for materialization-return and replace deprecation warning with an exception. (#3896)
  • Build catalog for only relational, non-ephemeral nodes in the graph (#3920)
  • Enact deprecation to remove the release arg from the execute_macro method. (#3900)
  • Enact deprecation for default quoting to be True. Override for the dbt-snowflake adapter so it stays False. (#3898)
  • Enact deprecation for object used as dictionaries when they should be dataclasses. Replace deprecation warning with an exception for the dunder methods of __iter__ and __len__ for all superclasses of FakeAPIObject. (#3897)
  • Enact deprecation for adapter-macro and replace deprecation warning with an exception. (#3901)
  • Add warning when trying to put a node under the wrong key. ie. A seed under models in a schema.yml file. (#3899)
  • Plugins for redshift, snowflake, and bigquery have moved to separate repos: dbt-redshift, dbt-snowflake, dbt-bigquery
  • Change the default dbt packages installation directory to dbt_packages from dbt_modules. Also rename module-path to packages-install-path to allow default overrides of package install directory. Deprecation warning added for projects using the old dbt_modules name without specifying a packages-install-path. (#3523)
  • Update the default project paths to be analysis-paths = ['analyses'] and test-paths = ['tests]. Also have starter project set analysis-paths: ['analyses'] from now on. (#2659)
  • Define the data type of sources as an array of arrays of string in the manifest artifacts. (#3966, #3967)
  • Marked source-paths and data-paths as deprecated keys in dbt_project.yml in favor of model-paths and seed-paths respectively.(#1607)
  • Surface git errors to stdout when cloning dbt packages from Github. (#3167)

Contributors:

Previous Releases

For information on prior major and minor releases, see their changelogs: