General discussion for Qt6 migration

[hmaarrfk]

Comment:

There is alot of worry that Qt6 will cause a slow rollout for many libraries.

Already we can see that many of the packages have not yet completed the qt -> qt-main migration in

https://conda-forge.org/status/#qt515

Many of the packages seem abandoned, so it might be best to:

1. Archive those packages.
2. Assess the situation with a few key packages.
3. Compare how other linux distros are doing it.

It seems at least that ubuntu is at least providing both:

$ sudo apt search libqt[56]net
Sorting... Done
Full Text Search... Done
libqt5network5/jammy-updates,now 5.15.3+dfsg-2ubuntu0.1 amd64 [installed,automatic]
  Qt 5 network module

libqt5networkauth5/jammy 5.15.3-1 amd64
  online account access for Qt apps - Library

libqt5networkauth5-dev/jammy 5.15.3-1 amd64
  online account access for Qt apps - Development Files

libqt6network6/jammy 6.2.4+dfsg-2ubuntu1 amd64
  Qt 6 network module

libqt6networkauth6/jammy 6.2.4-1 amd64
  Qt 6 QtNetworkAuth library

libqt6networkauth6-dev/jammy 6.2.4-1 amd64
  Qt 6 QtNetworkAuth - development files

Other relevant discussion:

• Update to 6.x #44 (comment)
• Update to 6.x #14
• Update to 6.x #14 (comment)

Updating Qt/PyQt to 5.15 was a huge deal of work, so on the Spyder team we won't be looking at Qt6 for at least a year, perhaps more. And I don't know who else would be willing to do this as a paid effort, sorry.

[Tobias-Fischer]

@scopatz - tagging you here as you seem to be the sole maintainer of quite a few packages that have not yet been migrated to qt 5.15.

[ccordoba12]

Pasting here @Tobias-Fischer #44 (comment):

So if we don't allow installation of qt5 and qt6 side-by-side, and are not careful in downstream feedstocks (where some maintainers might opt to manually migrate to qt6 without also still building for qt5), won't we fairly soon run into the situation where packages become incompatible with each other? We would need to maintain two branches of qt5/6-webengine etc. anyhow, regardless of the renaming.

Note that I don't have particularly strong feelings about this, I am just curious how we envision the transition to qt6 :).

This is a good point, one that I haven't considered before. Feedstocks would need to explicitly declare if they support only qt5 or qt5 and 6, in case we decide not to version Qt6. I don't know if this could be handled by the migration bot, i.e. by adding a preventive constraint on qt/pyqt/pyside < 6 for all feedstocks that currently depend on those packages.

[ccordoba12]

Compare how other linux distros are doing it.
It seems at least that ubuntu is at least providing both

Yeah, they do that because it can take maintainers a couple of years to move their packages from qt5 to 6, especially for C++ apps such as Paraview, as @jschueller mentioned.

[hmaarrfk]

My main question is:

• Does upstream support having qt5 and qt6 libraries installed at the same time? Doe the symbols clash? Do the headers clash?

If upstream supports it, then I am in favor of supporting it in conda-forge too.

If upstream does not support it, I dont think we should work too hard to get both installed in the same environment. It could take longer to do that than "a few years"

[Tobias-Fischer]

Yes, upstream supports installation of qt5 and qt6 at the same time. This happens in e.g. Ubuntu, too. See https://doc.qt.io/qt-6/packaging-recommendations.html

In #44 @jschueller already follows these packaging recommendations. I am not sure if we need to update our qt5 builds in some way, too, though?

[jschueller]

there should be no conflicts with the qt5 version, we install in qt6 subfolders, and for binaries only versioned symlinks are available, which means qmake would point to the qt5 version and qmake6 to the qt6 version
and although conda wont allow to install different versions of the same package, it means it will be ready when the time comes

[Tobias-Fischer]

So if we keep qt-main as qt5, and create a new package qt6-main that contains qt6 (which is very easy), we should be good to go? I suggest an easy way to test things would be to then install qt-main and qt6-main in the same environment, and check whether both the tests for qt-main and qt6-main still work.

We also might want to try whether https://manpages.ubuntu.com/manpages/jammy/man1/qtchooser.1.html (https://code.qt.io/cgit/qtsdk/qtchooser.git/) works with conda packaging. If qtchooser works, that would be amazing!

[jschueller]

what do you think @ccordoba12 ? should I close #44 and request it as a new qt6-main feedstock ?

[ccordoba12]

what do you think @ccordoba12 ? should I close #44 and request it as a new qt6-main feedstock ?

We have mentioned pros and cons but I think we haven't made a decision on that yet. I'm also uncertain on how to proceed. Should we call a vote on this? @hmaarrfk, what do you think?

We also might want to try whether https://manpages.ubuntu.com/manpages/jammy/man1/qtchooser.1.html (https://code.qt.io/cgit/qtsdk/qtchooser.git/) works with conda packaging. If qtchooser works, that would be amazing!

For that we're using qt.conf at the moment. So we'd probably need to patch Qt6 to point to a differently named qt.conf, so that Qt5 and 6 can coexist in the same environment.

[hmaarrfk]

I agree with the coinstallation attempt.

[hmaarrfk]

I mostly wanted to have the Convo here so as not to loose it and to clutter jschulers technical work

[hmaarrfk]

We can likely create a qt5-main package, and do the switchover in 1 year or two from qt5-main to qt6-main. An other mini migrator!

[Tobias-Fischer]

For that we're using qt.conf at the moment. So we'd probably need to patch Qt6 to point to a differently named qt.conf, so that Qt5 and 6 can coexist in the same environment.

It seems like this is simply a matter of renaming qt.conf to qt6.conf: "For this purpose, a file named qt6.conf can be used instead of the qt.conf file. If both files exist in the directory described above, qt6.conf is used." (https://doc-snapshots.qt.io/qt6-6.4/qt-conf.html)

what do you think @ccordoba12 ? should I close #44 and request it as a new qt6-main feedstock ?

There is no need to create a new feedstock - we can simply change the name of the package that is created. See e.g. https://github.com/conda-forge/libignition-gazebo-feedstock/blob/main/recipe/meta.yaml where the name consists of the base name and the major version.

[jschueller]

I renamed the package to qt6-main in #44 and checked it can be co-installed with qt-main and still pass its tests.
For win32 unfortunately some plugins and exes still clobber with qt-main, I'll try to ask upstream.
Would this be considered good enough for merging though ?

[hmaarrfk]

if you add an exclusive constraint on qt6-main and qt-main for now, that might be ok.

[jschueller]

ok, I just did that

[jschueller]

opened upstream issue for the coinstallation on windows: https://bugreports.qt.io/browse/QTBUG-107009
what do you think now ?

[hmaarrfk]

Green from me. I'm not the biggest expert in QT so I defer to all of you!

[Tobias-Fischer]

Hiya @traversaro - sorry to bother you, you are just too much of a Windows expert. Is there anything suspicious you can see in https://bugreports.qt.io/browse/QTBUG-107009 regarding the location of the *.dll / *.lib / *.exe files?

[traversaro]

Hiya @traversaro - sorry to bother you, you are just too much of a Windows expert. Is there anything suspicious you can see in https://bugreports.qt.io/browse/QTBUG-107009 regarding the location of the *.dll / *.lib / *.exe files?

I am relatively new to this, but reading the discussion in https://bugreports.qt.io/browse/QTBUG-107009 probably a workaround that we could use in the conda build script is to add build_dir/$INSTALL_ARCHDATADIR/bin to the PATH before invocking cmake/cmake --build .?

[jschueller]

update: qt-main and qt6-main can now be co-installed even on win32 so we would at least avoid breaking qt5-only packages
(and packages wanting to go to qt6 may choose to require qt6-main instead of qt-main)

[jschueller]

There is also the new PySide6 package to think about: conda-forge/staged-recipes#20673
@ccordoba12 seems to think we need some migration path before it can be made available
I'm rather in favor in making it available asap (either in a new feedstock or in a branch of the pyside2 recipe like we did for qt6-main)
Any thoughts on this ?

[Tobias-Fischer]

I’m in favour of a new branch in the existing feedstock to mirror qt-main and qtwebengine.

[hoechenberger]

I'm rather in favor in making it available asap

I'd love to see that happen :)

[jschueller]

what do you think @hmaarrfk (you also maintain pyside2) ?

[hmaarrfk]

Branch is fine. I don't think the bot is working too well for pyside2

[jschueller]

branch it is then: conda-forge/pyside2-feedstock#136

[jschueller]

we cannot split completely easily all modules because host tools need to be natively compiled when cross-compiling for at least qtbase+qtdeclarative+qtshadertools+qttools, so we would need at least one package for these 4, this can be circumvented possibly with multiple outputs, but that makes packaging a bit harder also.

I understand your concern that the new qt6-main does not provide qt3d anymore and that may confuse maintainers though,
but I also prefer to keep things simple to maintain.

[hmaarrfk]

has anybody noticed that qt3d was missing? Maybe the few visualizations softwares that still aren't migrated.

[jschueller]

I re-thought about it, and imho qt-3d does not seem to be used that much, I only found qgis, maybe you're mistaken with the qopengl classes which are part of qtbase.

[hmaarrfk]

What should we do about things like opencv, a while back, i had something for 5.9, 5.12, and none. Should I revive this effort? conda-forge/opencv-feedstock#206

[h-vetinari]

So the qt 5.15 migration got closed, and so did the migration for qt6 6.5.

Now that we have different & co-installable outputs qt & qt6, I'm not sure how the longterm migration plan looks like? Add a piggyback migrator that rewrites "- qt" in meta.yaml to "- qt6"?

[hmaarrfk]

I don't want to push things in one way or an other yet. I've been testing applications and many have growing pains with QT6. I would like to wait before migrating. OpenCV for one already has enough "double builds".

[h-vetinari]

Coming back ~6 months later, a lot of qt5-things are starting to fall off the wagon. For example, pyside2 isn't really working anymore (missing migrations), whereas pyside6 is fine. This is not a criticism at all, just an indicator of the state of things, so I'm wondering if perhaps the time to migrate to qt6 has come (closer).

[hmaarrfk]

I use pyside2 on a daily basis. not sure what "doesn't work anymore" means.

we are missing qtwebengine for qt6 so we can't just switch over. Spyder uses it.

I could use some help finding a way to build out opencv.

[h-vetinari]

Well, the branch for 5.x on the pyside2-feedstock is not particularly alive (whereas the pyside6 branch gets regular updates). I looked at it because of a user-report that

conda install pyside2 ant python-ldap "openjdk>=17"

was running into conflichts. Might be related to missing migrations for libxml or alsa or whatnot.

[hmaarrfk]

we haven't been updating the version of at (and I don't want to use unsynchronized pyside2 and at versions) but the migrations have been happening.

the problem is that the windows builds of qtwebengine time out, and as such, cannot be built. this will very likely be true for qt6 as well, we just don't build qtwebengine so we haven't run into this yet.

[h-vetinari]

In conda-forge/opencv-feedstock#417, @hmaarrfk asked whether we want to drop qt5 builds (Context: OpenCV has a massive build matrix), leading to the following conversation:

@h-vetinari: Fine by me per se. What's the status on the overall qt6 migration? How many feedstocks (if any) are still qt5-only? I'd kinda like to drop qt5 in general...

@hmaarrfk: Spyder is in general PyQt (not sure if they handle 6) and require webengine. Which we don't have for Qt6.

However. All platforms have a "headless" option. So it should be co-installable, just without the GUI parts on opencv.

I wanted to bring this back into a bit of a wider forum. I'm a bit hesitant to drop qt5 variants if there are important use-cases that cannot yet be supported with qt6 (but at the same time the OpenCV CI situation really asks for a reduction).

I fixed some conflicts in conda-forge/qt-webengine-feedstock#46 for the webengine side, but don't know the status for spyder etc. @ccordoba12?

[hmaarrfk]

See: spyder-ide/spyder#20201

[ccordoba12]

I fixed some conflicts in conda-forge/qt-webengine-feedstock#46 for the webengine side, but don't know the status for spyder

Spyder should be working with PyQt6 because a volunteer send us several PRs to support it. But we haven't tested it, so I'm not really sure.

The problem for us in Conda-forge is Webengine because we depend on it in several places.

[traversaro]

For pcl we are migrating to qt6 in conda-forge/pcl-feedstock#65 as there is the constraint of using the same qt version used by vtk.

[Krakonos]

Hi! When considering different modules, I suggest each of them gets a its own package based on theqt structure. This has a huge advantage for developers, since they already know the package name and don't have to search. The dependency list already is written in the CMakeLists.txt for the project (well, mostly) .

I'm currently missing the NetworkAuth package and had to spend considerable time just to find out it's not available in the repository (I think, I don't have a good way of verifying).

I'd be happy to help out, but TBH I'm not familiar with conda-forge package creation and the process, so I'm kind of lost in the many closed PRs about the qt packages. Is there an active effort to get other packages into conda?

[hmaarrfk]

I'd be happy to help out, but TBH I'm not familiar with conda-forge package creation and the process, so I'm kind of lost in the many closed PRs about the qt packages. Is there an active effort to get other packages into conda?

A PR similar to conda-forge/staged-recipes#25992 is likely the best "documentation" we have.

Thanks for your willingness to help!

[traversaro]

If you know the name of a file/library that is contained in the library you are looking for, a possible strategy is to use https://conda-metadata-app.streamlit.app/Search_by_file_path?q=conda-forge to search which package (if any) in conda-forge contains that file.

[Krakonos]

@hmaarrfk Thanks. There seems to be a lot of boilerplate, like the x11 deps. Is there any chance to share those between the recipes? I could hack at it and see if I can build the libs I need, but I don't think it is feasible to expand this to all the qt libs (well, it might be, but it will become a huge maintenance burden).

@traversaro Thanks, this is awesome and should be part of the main package repository :-)

[ccordoba12]

Hi! When considering different modules, I suggest each of them gets a its own package based on theqt structure. This has a huge advantage for developers, since they already know the package name and don't have to search. The dependency list already is written in the CMakeLists.txt for the project (well, mostly) .

I said it before and I'll reiterate again: that would be too much work for Conda-forge maintainers. @Krakonos, all you need to know is that mostly everything required for development is in the qt-main package. The one Qt module that is not part of it is QtWebEngine, which has its own separate package.

[hmaarrfk]

boilerplate, like the x11 deps

There are efforts to make alot of the boiler plate easier, see #287 but...

Don't get loose track of your goal, which is to get your app up and running ^_^.
Copy paste is a very well established technology.

@ccordoba12

that mostly everything required for development is in the qt-main package.

this doesn't seem to be true for the package of interest of his. Ultimately, maintaining the megapackage is really difficult..... I think beyond what we have today for the "qt6", we should be trying to split things off. the build time is MUCH faster, and allows more contributors to get involved.

[Krakonos]

I said it before and I'll reiterate again: that would be too much work for Conda-forge maintainers. @Krakonos, all you need to know is that mostly everything required for development is in the qt-main package. The one Qt module that is not part of it is QtWebEngine, which has its own separate package.

Noted. And yeah, that might be a bit excessive. But I guess matching the source tarballs makes sense though, right? For example, Core, widgets etc. is in a single base package, but extras like networkauth, svg, 5compat, etc. should be separate package. Otherwise I don't know how to handle single package from multiple tarballs.

(list of packages: https://download.qt.io/official_releases/qt/6.7/6.7.2/submodules/ )

@hmaarrfk My app is running. QNetworkAuth is small enough so I can just build it as part of the job, not a huge deal now, but I don't like the concept of mixing source & conda packages together (and building whole qt is out of the question, qt official unattended installer seems pretty hard to run).

But I'm a bit confused, the package in qt-main-feedstock seems to be fixed to a single 5.x version, is there a separate qt6 package?

[hmaarrfk]

We build it as part of a separate branch
https://github.com/conda-forge/qt-main-feedstock/tree/qt6

[hmaarrfk]

An other reason to separate it it out is that it is a GPL-3.0-only package. My attempt is here: conda-forge/staged-recipes#27591