You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I have searched the existing issues, and I could not find an existing issue for this bug
Current Behavior
Job runs sporadically hitting "conflict with concurrent transaction error." The failure is inconsistent as the customer cannot reproduce it on every execution. On one of their dbt Cloud environment (5 threads) it happened twice but on our their local DBT core instances(20+ threads) they were able to reproduce it every third execution. Their local execution has another model victim.
They checked their logs and were not able to locate any other thread running parallel execution that would cause conflict.
Expected Behavior
Job run does not fail on concurrent transactions.
Steps To Reproduce
Not yet, but customer mentioned:
"Neither of the models are using incremental materialization, only thing that might be related to this failure is that these models being used in the materialized view downstream."
I had ran this past our team internally and it was mentioned:
I suspect this could be the culprit. I remember that querying the object dependency tree in Redshift is not easy when using MVs. So I wonder if dbt is dropping the MV in parallel to the refresh of this table and that's attempting to drop this table at the same time.
Relevant log output
2024-09-03 00:40:17.026263 (Thread-3 (worker)): 00:40:17 On model.A_dbt.modelA: /* {"app": "dbt", "dbt_version": "1.7.17", "profile_name": "user", "target_name": "default", "node_id": "model.A_dbt.modelA"} */ drop table if exists "abcd"."dbt_1234"."modelA__dbt_backup" cascade
2024-09-03 00:40:22.922586 (Thread-3 (worker)): 00:40:22 Redshift adapter: Redshift error: could not complete because of conflict with concurrent transaction 2024-09-03 00:40:22.923500 (Thread-3 (worker)): 00:40:22 On model.A_dbt.modelA: ROLLBACK 2024-09-03 00:40:23.147961 (Thread-3 (worker)): 00:40:23 Redshift adapter: Error running SQL: macro drop_relation
Please feel free to reach out to me and we can further discuss logs. I did not add them as they are customer logs and not dbt sandbox or test logs from dbt.
The text was updated successfully, but these errors were encountered:
Is this a new bug in dbt-redshift?
Current Behavior
Job runs sporadically hitting "conflict with concurrent transaction error." The failure is inconsistent as the customer cannot reproduce it on every execution. On one of their dbt Cloud environment (5 threads) it happened twice but on our their local DBT core instances(20+ threads) they were able to reproduce it every third execution. Their local execution has another model victim.
They checked their logs and were not able to locate any other thread running parallel execution that would cause conflict.
Expected Behavior
Job run does not fail on concurrent transactions.
Steps To Reproduce
Not yet, but customer mentioned:
I had ran this past our team internally and it was mentioned:
Relevant log output
Environment
Additional Context
Please feel free to reach out to me and we can further discuss logs. I did not add them as they are customer logs and not dbt sandbox or test logs from dbt.
The text was updated successfully, but these errors were encountered: