Use ogre version 1.12 on focal

[simonschmeisser]

I finally managed to get Ogre in Ubuntu updated today and since @rhaschke already ported rviz to support it, it should be used in noetic

[clalancette]

I finally managed to get Ogre in Ubuntu updated today and since @rhaschke already ported rviz to support it, it should be used in noetic

This is just a question. According to apt-cache, it looks like both libogre1.9-dev and libogre-1.12-dev are currently available in Focal. Is that expected to be the case in the final release of 20.04?

[simonschmeisser]

yes, that is expected. They conflict however, so by installing either you decide against which version to link.

[clalancette]

All right. Looking at the rdepends, it looks like gazebo9 in Ubuntu 20.04 depends on libogre-1.9.0v5. @sloretz If we make the libogre key point to 1.12, is that going to cause problems for Noetic?

[rhaschke]

It would be great if Gazebo could switch to Ogre 1.12 as well. The 1.9 version is from 2013 and meanwhile several bugs have been resolved. To get an idea, which API changes are required, it might be useful to look at ros-visualization/rviz#1434.

[sloretz]

@sloretz If we make the libogre key point to 1.12, is that going to cause problems for Noetic?

AFAIK the plan is to release Gazebo 11 and Ignition Citadel into packages.ros.org. @chapulina do you know if those are compatible with ogre 1.12? EDIT answered my own question, Gazebo 11 uses 1.9, and ignition rendering 3 uses either 1.9 or 2.1

Looking at this from the other direction, @simonschmeisser do RViz and Gazebo need to use the same ogre version? What if we made separate rosdep keys fro 1.9 and 1.12?

[rhaschke]

As mentioned here, Ogre 1.9 and 1.12 will be available in Focal, but they conflict with each other. Hence, we need to decide on one or the other.

[sloretz]

As mentioned here, Ogre 1.9 and 1.12 will be available in Focal, but they conflict with each other. Hence, we need to decide on one or the other.

Darn, sure enough

Package: libogre-1.12-dev
Source: ogre-1.12
Version: 1.12.4+dfsg1-4
Architecture: amd64
Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>
Original-Maintainer: Debian Games Team <pkg-games-devel@lists.alioth.debian.org>
Installed-Size: 8102
Depends: libogre-1.12 (= 1.12.4+dfsg1-4), libboost-dev, libboost-thread-dev
Suggests: ogre-1.12-doc
Conflicts: libogre-1.8-dev, libogre-1.9-dev, libogre-dev
Section: libdevel
Priority: optional
Homepage: https://ogre3d.org/
Description: 3D Object-Oriented Graphics Rendering Engine (development files)
 OGRE (Object-Oriented Graphics Rendering Engine) is a scene-oriented, flexible
 3D engine written in C++ designed to make it easier and more intuitive for
 developers to produce applications utilising hardware-accelerated 3D
 graphics. The class library abstracts all the details of using the underlying
 system libraries like Direct3D and OpenGL and provides an interface based on
 world objects and other intuitive classes.
 .
 This package contains the headers needed to develop with OGRE.

Ouch, and there are API breaks between 1.9 and 1.12. What new features since 1.9 make the 1.12 upgrade worth it? Are Gazebo and Ignition Rendering the only other packages in the ROS ecosystem that use Ogre?

I think the Gazebo/Ignition team is currently occupied with the Bitbucket/Mercurial apocalypse migration.

@chapulina Do Gazebo 11 and Ignition Rendering build with ogre 1.12? If not, is there someone on the Ignition team or community that could work on making them build with that version? With the bitbucket/github transition going on, if someone from the community was going to take it up, where would they open PRs?

[simonschmeisser]

Risking to state the obvious, but only the -dev packages conflict (and decide which version to build against).

Also gazebo is released as a Ubuntu package and it's not possible or necessary to change anything there due to freezes. It can however easily have a different version of ogre than rviz. Finally it has it's dependencies done right, so it doesn't depend on the -dev package.

Ignition is a different story, there seem to be some packages distributed via Ubuntu but not the rendering parts. Is the rest actually going to be distributed via the ros packages archive at osrf?

[sloretz]

Also gazebo is released as a Ubuntu package and it's not possible or necessary to change anything there due to freezes. It can however easily have a different version of ogre than rviz. Finally it has it's dependencies done right, so it doesn't depend on the -dev package.

That's good news. Looking at packages.osrfoundation.org, it looks like there is a separation between libgazebo11-dev and libgazebo11 where the former depends on the ogre -dev package and the latter does not. gazebo_ros_pkgs/gazebo_dev does depend on the libgazeboN-dev package, but as long as they're not building RViz from source they shouldn't have to have the conflicting ogre -dev package installed.

Ignition is a different story, there seem to be some packages distributed via Ubuntu but not the rendering parts. Is the rest actually going to be distributed via the ros packages archive at osrf?

Yeah, both Gazebo 11 and Ignition Citadel will be released via packages.ros.org. It looks like ignition is the same where there are separate -dev packages, where libignition-rendering3-ogre1-dev is the one that depends on the ogre -dev package.

I still have a concern that RViz has a build_export_depend on the libogre-dev rosdep key. Does that become a debian package dependency?

It seems like it does based on ros-melodic-rviz depending on libogre-1.9-dev

Package: ros-melodic-rviz
Version: 1.13.9-2bionic.20200406.134254
Architecture: amd64
Maintainer: William Woodall <william@osrfoundation.org>
Installed-Size: 10923
Depends: libassimp4, libboost-filesystem1.65.1, libboost-program-options1.65.1, libboost-system1.65.1, libboost-thread1.65.1, libc6 (>= 2.14), libconsole-bridge0.4, libgcc1 (>= 1:3.0), libgl1, libogre-1.9.0v5 (>= 1.9.0+dfsg1-9~), libqt5core5a (>= 5.9.0~beta), libqt5gui5 (>= 5.7.0), libqt5widgets5 (>= 5.2.0~alpha1), libstdc++6 (>= 5.2), libtinyxml2-6 (>= 5.0.0), liburdfdom-world, libx11-6, libyaml-cpp0.5v5, libassimp-dev, libgl1-mesa-dev, libglu1-mesa-dev, libogre-1.9-dev, libqt5opengl5, libtinyxml2-dev, libyaml-cpp-dev, ros-melodic-geometry-msgs, ros-melodic-image-transport, ros-melodic-interactive-markers, ros-melodic-laser-geometry, ros-melodic-map-msgs, ros-melodic-media-export, ros-melodic-message-filters, ros-melodic-nav-msgs, ros-melodic-pluginlib, ros-melodic-python-qt-binding, ros-melodic-resource-retriever, ros-melodic-rosbag, ros-melodic-rosconsole, ros-melodic-roscpp, ros-melodic-roslib, ros-melodic-rospy, ros-melodic-sensor-msgs, ros-melodic-std-msgs, ros-melodic-std-srvs, ros-melodic-tf, ros-melodic-urdf, ros-melodic-visualization-msgs
Homepage: http://wiki.ros.org/rviz
Priority: optional
Section: misc
Filename: pool/main/r/ros-melodic-rviz/ros-melodic-rviz_1.13.9-2bionic.20200406.134254_amd64.deb
Size: 2200960
SHA256: 12f076ce17b473f4250c0d0194a9bb645faed264b43b5ceefccae7ce113ecb61
SHA1: 2fbc6756870b4c921e793f947443463115b832c1
MD5sum: 2af93636affa06e6f6399208cddc6f25
Description: 3D visualization tool for ROS.

I think that means anyone that wants to build a gazebo plugin would have to uninstall RViz first.

[rhaschke]

I still have a concern that RViz has a build_export_depend on the libogre-dev rosdep key. Does that become a debian package dependency?

No, according to package.xml documentation, packages listed as build_export_depend are required if you want to build downstream packages that build-depend on rviz. Thus, it defines transitive build-only dependencies.

Thus, people run into trouble if they want to compile both a gazebo plugin and a rviz plugin.
To avoid this, I suggest to stick with ogre 1.9 for now, and simultaneously switch to Ogre 1.12.2 if both Gazebo and Ignition are ready to do so as well - which hopefully is soon.
This way, we don't further block the release process.

[chapulina]

Sorry @sloretz , I just saw these questions.

Do Gazebo 11 and Ignition Rendering build with ogre 1.12?

There have been community contributions adding Ogre 1.12 support. I haven't tried it yet, but I guess they either work already, or there's little left to do.

With the bitbucket/github transition going on, if someone from the community was going to take it up, where would they open PRs?

Gazebo and Ignition's GitHub migration is complete. Here's where the PRs would go:

• https://github.com/osrf/gazebo
• https://github.com/ignitionrobotics/ign-rendering/

Yeah, both Gazebo 11 and Ignition Citadel will be released via packages.ros.org.

Correct. But beware that both of them have already been released into http://packages.osrfoundation.org/ using Ogre 1.9 / 2.1, so I don't think we should release a different version into http://packages.ros.org/.

---

CC @j-rivero

[simonschmeisser]

Correct. But beware that both of them have already been released into http://packages.osrfoundation.org/ using Ogre 1.9 / 2.1, so I don't think we should release a different version into http://packages.ros.org/.

Given that focal was only released a few days ago I think that doing a new release to http://packages.osrfoundation.org/ for focal should still be fine?

[simonschmeisser]

It appears that while gazebo11 might already work with ogre 1.12, they do not intend to make the update at all due to risk of regressions. Is there a way to find out how many gazebo plugin packages exist and if those actually build-depend on ogre itself? If this number is low we could add an additional rosdep-entry for ogre-1.12 (and ogre-1.12-dev) instead of modifying the current one.

I would not expect the conflict between libogre-1.9-dev and libogre-1.12-dev to cause any problems on the build farm but just for users compiling plugins for both rviz and gazebo from code (but those could still go one more step and compile gazebo themselves as well?)

[chapulina]

risk of regressions

There are known regressions, like heightmap support, see gazebosim/gazebo-classic#2686. But the main reason is that Gazebo 11 has already been released, and we can't break existing users (within the ROS ecosystem or outside of it).

we could add an additional rosdep-entry for ogre-1.12 (and ogre-1.12-dev) instead of modifying the current one

As long as they're co-installable and can be found independently, I think that could be ok. We just need to be sure that when users are compiling Gazebo plugins, they link against the correct version. I'm not sure if Gazebo is exporting these dependencies correctly for plugins to pick them up.

users compiling plugins for both rviz and gazebo from code

Is that an existing use-case? For a user to be affected, they'd need to be compiling the same library against both RViz and gazebo::rendering. It's risky to guess what downstream users are doing, but I think it could be reasonable to say we won't support such combinations on Noetic. Compile your RViz plugins against Ogre 1.12 and your Gazebo plugins against Ogre 1.9 - am I missing something or is that ok?

[clalancette]

Compile your RViz plugins against Ogre 1.12 and your Gazebo plugins against Ogre 1.9 - am I missing something or is that ok?

Unfortunately, that is going to be tricky (though not impossible). The thing is that that libogre-1.9.0v5 and libogre-1.12 are side-by-side installable, but libogre-1.12-dev and libogre-1.9-dev are not. And you need libogre-*-dev to compile plugins.

[wjwwood]

Just getting to this issue, I didn't see if before.

@simonschmeisser Sorry, I know this wasn't the solution you wanted (Gazebo on ogre 1.9), which is really our faults for not coordinating on that earlier. Assuming that's not going to change (based on gazebosim/gazebo-classic#2726 (comment)), I now wonder if it makes sense to put rviz on ogre 1.12 and have to deal with any of the downstream development issues caused by the ogre 1.9-dev and ogre 1.12-dev packages being conflicting. Is there an important reason for pushing rviz in Noetic to ogre 1.12 rather than sticking with ogre 1.9 (like technical advantages for instance)?

[wjwwood]

Actually, it was pointed out to me that since rviz and gazebo-ros-pkgs neither have been split into dev/runtime packages, we have to use ogre 1.9 with rviz due to gazebo11's decision to stick with it.

Shall I update this pr or open a new one?

[simonschmeisser]

rviz only depends on libogre-* but has a build_export_depend on libogre-*-dev so its perfectly fine to co-install rviz with ogre 1.12 and libogre-1.9-dev for gazebo. Unfortunately I was not able to find where the dependencies of gazebo are defined, then I could check if there are any problems there (it should only build-depend on dev as well)

As to benefits of using ogre-1.12, well, 7 years of bugfixes and speed improvements. @rhaschke might be able to point to specific issues. Plus options to enable more modern rendering backends and modern graphics.

Not sure what you mean by updating this PR?

[rhaschke]

@wjwwood, as far as I understand, it should be possible to install the binary versions of rviz and Gazebo11 in parallel - together with their corresponding libogre dependencies. If configured correctly, they won't need libogre-*-dev. rviz was recently configured correctly (having a build_export_depend on libogre-*-dev only). The open question is, whether Gazebo (and gazebo-ros-pkgs) is configured in the same fashion. If not (yet), this needs to be done.

As pointed out in #24448 (comment), only users trying to compile both rviz and gazebo plugins (and thus requiring both -dev versions), will be affected by the conflict. Of course, it's also an open question, whether we should expose this conflict at all.

There are indeed some rviz bugs (#1192,#1082), which are resolved with a newer Ogre version.
Hence, a transition would be beneficial!

[simonschmeisser]

It boils down to this change gazebosim/gazebo-classic#2727 in order to remove the exec_depend here https://github.com/ros-simulation/gazebo_ros_pkgs/blob/00a0064a1477667dfed75c3d22d91f14224db3a1/gazebo_ros/package.xml#L30 fixing this issue ros-simulation/gazebo_ros_pkgs#323

Short summary: gazebo_ros uses pck-config to find the gazebo binaries but the pck-config file is included in gazebo_dev which is therefore currently an exec_depend of gazebo_ros

[j-rivero]

Sorry, I know this wasn't the solution you wanted (Gazebo on ogre 1.9), which is really our faults for not coordinating on that earlier.

Full context about this: our main mechanism to coordinate base dependencies during the development cycle of ROS is the REP3. ogre-1.9 was the only version of Ogre available in Ubuntu Focal until 11th of April, it required Freeze Exception to go into Focal and landed after the last Beta, 12 days before the final release if I'm not wrong. The only reason for Ubuntu to accept it at that point was that no other Ubuntu package depends on it because it's complicated to do transitions properly. My feeling is not that our sync mechanism has failed but more than we are trying to force policies and procedures to get benefits from latest releases which could be acceptable but is risky in my opinion.

[simonschmeisser]

I'm not trying to blame anybody (or process) for anything and I'm really sorry if it appeared that way!

I started updating ogre in October and expected this to be a straightforward process. However Debian has really high quality gates and incremental improvements seem difficult. The package was then waiting in the Debian NEW queue for two months till end of March. Only once it passed this bottleneck could I ask Ubuntu to still pull it into Focal. Being new and not part of any installation image it did not require a freeze exception btw but just a manual sync (which is also not over-documented and as such took some more days). This is my first Debian/Ubuntu upload after all.

I will open a PR on gazebo_ros to fix the scripts to work without the pkg-config file.

[simonschmeisser]

I added a simple fallback in the start scripts of gazebo_ros which makes it no longer depend on libogre-1.9-dev. So now it is perfectly possible to build plugins for rviz when you have libogre-1.12-dev installed and to build plugins for gazebo11 when you have libogre-1.9-dev installed

For more details please see ros-simulation/gazebo_ros_pkgs#1100

[wjwwood]

So, maybe I'm wrong, but just having a build_export on ogre-dev will not avoid it being installed when you install rviz... build_export is a run depend, as in, after I install ros-noetic-rviz I need to be able to build a package against rviz's headers, and if any of those headers include ogre (which they do I believe) then ogre-dev would need to already be installed.

The only way to avoid this issue would be to remove all use of ogre headers from rviz headers and only build depend on ogre-dev.

[simonschmeisser]

that's the difference between build_depend and build_export_depend, yes. But both do not translate to a dependency of the debian/installing it.

[wjwwood]

I don't think that's true @simonschmeisser, see this line where the depends are gathered in bloom:

https://github.com/ros-infrastructure/bloom/blob/dffd212a7e11a0d1d504ced705b2643c8a1c9b5a/bloom/generators/debian/generator.py#L318

They are later used in this template:

https://github.com/ros-infrastructure/bloom/blob/dffd212a7e11a0d1d504ced705b2643c8a1c9b5a/bloom/generators/debian/templates/cmake/control.em#L11

Which would indicate to me that build_export_depend's will be installed when you install the deb... Also with the situation I explained above, I believe this is required for things to work as expected.

[wjwwood]

Note run_depends there includes build export:

https://github.com/ros-infrastructure/catkin_pkg/blob/7b5e269bfed9702767015064bb1d4b6b5ff87476/src/catkin_pkg/package.py#L120

[sloretz]

It would be unfortunate that Ogre 1.9 is the version being used in Focal since this is an LTS ROS release. Since you've done so much work @simonschmeisser towards Ogre 1.12 already, here is a possible map of what it would take to use Ogre 1.12 in Focal. Given the Noetic release is barely over 2 weeks away, it would have to be done (meaning merged) very quickly.

• remove <build_export_depend>libogre-dev</build_export_depend> from RViz
• remove <build_export_depend>libgazebo11-dev</build_export_depend> from gazebo_dev
• Add separate rosdep keys for libogre1.9-dev and libogre1.12-dev
• Add note to the Noetic Migration page that any ROS package that builds gazebo plugins needs to add an explicit <build_depend>libogre-1.9-dev</build_depend> instead of relying on it comeing in transitively
• Add note to the Noetic Migration page that any ROS package who builds RViz plugins needs to add an explicit <build_depend>libogre-1.12-dev</build_depend> instead of relying on it comeing in transitively

Benefits

• Removal of libogre-dev from RViz makes the Noetic desktop_full docker images smaller
• Can use a much more recent version of Ogre

Drawbacks

• No single package could build both a Gazebo plugin and an RViz plugin
• No single workspace could be built if it had both a package that builds an RViz plugin and a package that builds a Gazebo plugin
• packages building RViz or gazebo plugins would need to declare dependency on appropriate libogre*-dev rosdep key instead of getting it transitively

It would need buy-in from these people

• RViz maintainers @wjwwood @rhaschke
• gazebo_ros_pkgs maintainers @j-rivero (also looks like Dave Coleman and John Hsu are maintainers?)

And maybe also

• @jacobperron who is handling the Foxy release
• me who is handling the Noetic release

Personally, I'm a bit nervous that this increases the risk of delaying the Noetic release.
I could be convinced, but Ogre-1.9 looks very attractive to me. Whatever happens, I intend to get a solution to libogre-dev in by the end tomorrow so that RViz can be released.

[wjwwood]

I think this approach is unorthodox, but I am willing to pursue it to try and get the benefits.

Instead of having people depend on libogre directly, maybe we could introduce a rviz_dev package that provides the transitive dependencies needed to build against rviz, and we'd explicitly move the build_export depends from rviz to this new package, at least the ogre one. I think this is slightly nicer because it removes some details from the user.

[rhaschke]

I think it is a too strong restriction to forbid building rviz and gazebo plugins within the same workspace. Of course, people could split their workspaces, but nevertheless this requires switching the installed libogre-dev debian, which - for example - will render our local CI system infeasible.
I was actually hoping that Gazebo folks would be willing to switch to Ogre 1.12 as well. And yes, it's indeed my and @simonschmeisser's fault not having coordinated/announced this transition early enough. Hence, I'm afraid we are stuck to Ogre 1.9 for another 4 years.

[simonschmeisser]

Thanks for your full picture summary @sloretz !

* remove [`<build_export_depend>libogre-dev</build_export_depend>`](https://github.com/ros-visualization/rviz/blob/13d061d973e96c6315c7768a7f316d2cc501299d/package.xml#L30) from RViz

* remove [`<build_export_depend>libgazebo11-dev</build_export_depend>`](https://github.com/ros-simulation/gazebo_ros_pkgs/blob/bf6858214507787aaaf4d1311e6d1a9a896f6960/gazebo_dev/package.xml#L22) from gazebo_dev

* Add separate rosdep keys for `libogre1.9-dev` and `libogre1.12-dev`

* Add note to [the Noetic Migration page](http://wiki.ros.org/noetic/Migration) that any ROS package that builds gazebo plugins needs to add an explicit `<build_depend>libogre-1.9-dev</build_depend>` instead of relying on it comeing in transitively

* Add note to [the Noetic Migration page](http://wiki.ros.org/noetic/Migration) that any ROS package who builds RViz plugins needs to add an explicit `<build_depend>libogre-1.12-dev</build_depend>` instead of relying on it comeing in transitively

I would prefer the approach suggested by @wjwwood which would have the following action points remaining:

• add a rviz_dev package to rviz with a build_export_depend on libogre-dev
• Add note to the Noetic Migration page that any ROS package who builds RViz plugins needs to add an explicit <build_depend>rviz_dev</build_depend> instead of relying on it comeing in transitively
• release rviz_dev to melodic as well to accomodate people working on a common branch for both
• merge Use fallback instead of pkg-config for gazebo discovery ros-simulation/gazebo_ros_pkgs#1100
• possibly fix packages depending on gazebo_ros to also build depend on gazebo_dev

Note that there is no need for an explicit versioned rosdep libogre-dev key as gazebo is not using rosdep to my understanding and the dependency is then implicitly brought in by libgazebo11-dev

If at any later point gazebo decides to release a hypothetical 11.1, this could all be updated transparently. Such a release would also possibly allow to remove ogre-1.9 from the next debian stable bullseye next year.

(If I would have been certain to get the update included in time for focal I would of course have announced it earlier, but this way all I could do is announce it one month ago which was still before both the focal release and obviously the noetic release)

[simonschmeisser]

@wjwwood regarding "build_export_depend": you're right and that's unfortunate ;) (but expected ...) It renders build_export_dependsomewhat meaningless. But we should discuss in another issue how that could be improved with future generations/iterations of tooling

[simonschmeisser]

@rhaschke wouldn't it just require that you compile gazebo11 against ogre-1.12 locally on your CI as well? (assuming you have a somewhat messy bundle-based and not bloom/package based CI just like we do?)

[rhaschke]

Yes, that would be an option. I think we need to be prepared for a lot of issues from users regarding this problem.

[simonschmeisser]

I really hope that there can still be a gazebo11.1 or gazebo12 release based on ogre-1.12 before the majority of users switches to noetic. Keep in mind that adoption of new releases is really slow in ROS(-1) (http://download.ros.org/downloads/metrics/metrics-report-2019-07.pdf shows that 14 months after melodic release only 27% had migrated yet). This will however neither possible nor motivated if we stick to ogre-1.9 now (and for remaining life-time of ROS-1)

(but I should stop flooding this issue)

[wjwwood]

Well, it's a tough decision, but given the time we have left and comments like @rhaschke's which I tend to agree with:

I think it is a too strong restriction to forbid building rviz and gazebo plugins within the same workspace.

I think I will have to decline this pr in favor of #24767.
I know that's not the resolution we wanted, but at least users could build with 1.12 if they wanted and possibly in the future we could have an ogre 1.12 variant of rviz and gazebo debs in parallel, sort of how we allow for newer versions of gazebo with older ros distributions.

[rhaschke]

Thanks for taking a decision on this @wjwwood. I will proceed with the Noetic release of rviz.