-
Notifications
You must be signed in to change notification settings - Fork 476
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Airflow 2.3.0 Support #572
Comments
When updating the image to 2.3.0 I get the following Error during CheckDB:
Actually, acording to https://airflow.apache.org/docs/apache-airflow/stable/release_notes.html#database-configuration-moved-to-new-section-22284, it should just throw a deprecation warning and not fail. Not sure what I am missing. |
I traced the broken backward compatibility over to Airflow and created an Issue: apache/airflow#23408 Nevertheless, we should use AIRFLOW__DATABASE__SQL_ALCHEMY_CONN_CMD instead of AIRFLOW__CORE__SQL_ALCHEMY_CONN_CMD. |
Until this is fixed, you can use the following workaround: airflow:
config:
AIRFLOW__DATABASE__SQL_ALCHEMY_CONN_CMD: "bash -c 'eval \"$DATABASE_SQLALCHEMY_CMD\"'" |
Thanks for this @c-thiel - your suggestion helped. It appears to be trying to run this script: and eventually fails with this error: So maybe there is something more that also needs to be changed? |
Could you also add support for the separate DagProcessorManager deployment that was introduced in 2.3.0? |
Really need this one! |
Looking forward to this feature |
It will be great |
stuck with this too, hoping for this feature to be released soon |
Waiting for this feature |
1 similar comment
Waiting for this feature |
i hope this will be released soon |
Hoping for this feature |
Waiting for this feature |
This will be really helpful. |
Looking forward to this feature |
Will be great to have this feature |
Waiting for this feature too |
Waiting for this feature! |
Waiting to start to use this feature |
Waiting for the release of this feature 🙏 |
Really need this one |
Hey all, I plan to release chart version However, I think we should wait until chart version
PS: If anyone knows of any other Airflow 2.3 features, we currently don't support, please comment them here. |
@thesuperzapper That's great, thank you for your work! Do you have any time estimates in mind for the 2.3.0 support release date? |
@Valenzione I expect to release Track progress on the Regarding the Right now, my hope is to finish the new task-aware celery autoscaler which allows safe and configurable up/downscaling of CeleryExecutor workers (including selectively removing the inactive worker Pods when downscaling). The prototype is coming along, but I haven't had much time to put the final polish on yet. PS: the reason But I also made large updates to |
Hey All, I am very sorry about the delay, I have been quite bogged down this week and last (Airflow Summit and the like). However, we now have resolutions for the main problems caused by 2.3.0, so should be ready to cut 8.6.1:
If anyone knows of other issues from 2.3+ not listed above please comment ASAP, so we can get them fixed in 8.6.1! |
@thesuperzapper I think we should give it a go, let's test that in our environments and provide a feedback if necessary. Disclaimer: We struggle a little bit with one of the bugs in Airflow 2.2.* which should be fixed in 2.3.1 hopefully, so no pressure! 😄😄 |
@jurovee not sure if it helps, but I have managed to upgrade from v2.2.5 to v2.3.2 with the chart version v8.6.0 without any changes in my values file. The breaking changes that were introduced in v2.3.0 seem to be fixed in the subsequent versions, so the upgrade itself works fine from this chart's perspective. |
@karakanb good to know! 👏 Thanks for the info. I think that 8.6.1 is almost ready and it should contain a fix for one of my issues with logs and external volumeMounts, so I think I'll just wait to have everything together + 2.3.2 in a single package 👍 |
Hey all, the long-awaited I would love it if you tested it out, and shared feedback! |
Just gave it a test... |
Checks
User-Community Airflow Helm Chart
.Motivation
The Chart is currently not compatible with Airflow 2.3.0 due to changes in the Database Configuration:
https://airflow.apache.org/docs/apache-airflow/stable/release_notes.html#database-configuration-moved-to-new-section-22284
Implementation
No response
Are you willing & able to help?
The text was updated successfully, but these errors were encountered: