Bug: changed log output of dbt-core v1.0.0 causes log parsing to fail.

[ciklista]

Problem:
The v1.0.0 release introduced structured logging across all of dbt logs (also those printed to stdout) (release notes). See also dbt-labs/dbt-core#3359 (comment) and dbt-labs/dbt-core#4055.

Unfortunately (or thankfully), this means that logs for dbt run-operation now look like

dbt run-operation --project-dir /pat/to/dir/ --args '{"sql": "SELECT 1 AS __dbt_invoke_check_macro__log_columns_list LIMIT 0"}'

08:58:14  Running with dbt=1.0.0
08:58:15  ['__dbt_invoke_check_macro__log_columns_list']

instead of

dbt run-operation --project-dir /pat/to/dir/ --args '{"sql": "SELECT 1 AS __dbt_invoke_check_macro__log_columns_list LIMIT 0"}'

Running with dbt=0.21.0
['__dbt_invoke_check_macro__log_columns_list']

In the case that I am currently looking at (generating the column names), logs are parsed based on line start and ends

dbt-invoke/dbt_invoke/properties.py

Lines 505 to 507 in 8e54944

	result_list_strings = [
	s for s in result_lines if s.startswith('[') and s.endswith(']')
	]

Possible solution
One possibility might be to switch to json logs to facilitate log parsing (using the global cli flag log-format, e.g. dbt --log-format=json run-operation)

dbt --log-format=json run-operation --project-dir /pat/to/dir/ _log_columns_list --args '{"sql": "SELECT 1 AS __dbt_invoke
_check_macro__log_columns_list LIMIT 0"}'

{"timestamp": "2021-12-08T09:13:56.977677Z", "message": "Running with dbt=0.21.0", "channel": "dbt", "level": 11, "levelname": "INFO", "thread_name": "MainThread", "process": 2960718, "extra": {"run_state": "internal"}}
{"timestamp": "2021-12-08T09:13:59.541933Z", "message": "['__dbt_invoke_check_macro__log_columns_list']", "channel": "dbt", "level": 11, "levelname": "INFO", "thread_name": "MainThread", "process": 2960718, "extra": {"run_state": "internal"}}

This is just a first investigation of the issue (or potential breaking changes of V1.0.0) and I will continue to look into it as I would really enjoy keeping dbt-invoke for subsequent dbt releases

[robastel]

Closed by merge of PR #14. Thank you, @ciklista, for the solution!