dbt-core v1.0.0

[ciklista]

This PR should make dbt-invoke compatible with dbt-core v1.0.0.

The main issue was the changed format of the dbt run-operation logs (#13).

What has changed:

• dbt run-operation is now being called with the --log-format=json global cli flag. This allows for easier parsing of the log output. (source)
• Making use of newly introduced log codes of dbt-core v1.0.0 in order to parse logs. This should also make it easier to solve Bug: properties command fails when dbt issues warning #12. I haven't included it in this PR, to keep things clean. (source)

Backward compatibility
The code should be backward compatible with the two other dbt-core versions (0.18.x and 0.19.x) of this project. I've run the tests on all three dbt-core versions and they passed. This included parsing the dbt run-opration json log output accordingly as well as setting up two different dbt_project.yml for testing. I haven't done any additional testing as of now.
I am still undecided whether that is the best solution or if a new release / tag of dbt-invoke should require dbt-core>=1.0.0 and users should opt for older releases for older dbt-core versions. Any opinions on that?

[robastel]

@ciklista Thank you for opening this PR (along with issues #12 and #13). Also, thank you for your detailed comment.

At a quick first glance, I like the idea of making dbt-invoke backwards compatible and not having to require dbt-core>=1.0.0.

Unfortunately, I am unavailable for the remainder of this week (I know, bad timing with the release of dbt 1.0.0) to actually complete the PR review, but I'll be back in action on Monday and plan on taking an in-depth look then. I hope that delay isn't too troublesome. Thanks again!

[ciklista]

@robastel No worries. I can run it with my local version for now. Looking forward to your review!

[robastel]

I think there should be a quick addition on line 194 of the README to mention that dbt-invoke is now tested against dbt version 1.0:

dbt-invoke/README.md

Line 194 in 8e54944

- dbt 0.18 and 0.19

And update the version number here to '0.1.3' so that we can create a new release with your changes:

dbt-invoke/dbt_invoke/internal/_version.py

Line 1 in 8e54944

__version__ = '0.1.2'

[robastel]

Approving! If this fails the linter then just pip install black and run black dbt_invoke tests setup.py -S -l 79 from within the top level of the repo.

[ciklista]

Looks like we have a dependency issue for the agate package and python 3.10.

When I do from collections import Sequence from python 3.8.10 I get

 DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working

should we fix the python version for the tests as well?

[ciklista]

From a quick first investigation, the failure of the second test also seems to be related to python 3.10.x.

The test was failing for dbt 0.19.2 (release), which imports a package called mashumaro in core/dbt/dataclass_schema.py.
https://github.com/dbt-labs/dbt-core/blob/745ae3e64ac03fedb19228b31252fea05eb0ed37/core/dbt/dataclass_schema.py#L13

The version of mashumaro for dbt 0.19.2 is fixed at 2.0 (see setup.py).
The test raises a NotImplementedError which is being raised based on the python version. However, for the 2.0 release, this only accounts for pythons up to 3.9.

Here is the function that raises the error in mashumaro v2.0: https://github.com/Fatal1ty/mashumaro/blob/v2.0/mashumaro/meta/helpers.py#L45-#L54

[robastel]

Yeah I was looking into it too. dbt doesn't support Python 3.10 yet (https://docs.getdbt.com/faqs/all), so let's just test 3.9 instead of 3.x here: https://github.com/ciklista/dbt-invoke/blob/317cdece28d7eb120a307b0d9db77f2fc3d872d6/.github/workflows/test.yml#L33