Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

or-tools v9.7 #47

Open
wants to merge 9 commits into
base: main
Choose a base branch
from

Conversation

regro-cf-autotick-bot
Copy link
Contributor

It is very likely that the current package version for this feedstock is out of date.

Checklist before merging this PR:

  • Dependencies have been updated if changed: see upstream
  • Tests have passed
  • Updated license if changed and license_file is packaged

Information about this PR:

  1. Feel free to push to the bot's branch to update this PR if needed.
  2. The bot will almost always only open one PR per version.
  3. The bot will stop issuing PRs if more than 3 version bump PRs generated by the bot are open. If you don't want to package a particular version please close the PR.
  4. If you want these PRs to be merged automatically, make an issue with @conda-forge-admin,please add bot automerge in the title and merge the resulting PR. This command will add our bot automerge feature to your feedstock.
  5. If this PR was opened in error or needs to be updated please add the bot-rerun label to this PR. The bot will close this PR and schedule another one. If you do not have permissions to add this label, you can use the phrase @conda-forge-admin, please rerun bot in a PR comment to have the conda-forge-admin add it for you.

Pending Dependency Version Updates

Here is a list of all the pending dependency version updates for this repo. Please double check all dependencies before merging.

Name Upstream Version Current Version
or-tools 9.7 Anaconda-Server Badge
protobuf 23.4 Anaconda-Server Badge

Dependency Analysis

We couldn't run dependency analysis due to an internal error in the bot, depfinder, or grayskull. :/ Help is very welcome!

This PR was created by the regro-cf-autotick-bot. The regro-cf-autotick-bot is a service to automatically track the dependency graph, migrate packages, and propose package version updates for conda-forge. Feel free to drop us a line if there are any issues! This PR was generated by https://github.com/regro/cf-scripts/actions/runs/5799934866, please use this URL for debugging.

@conda-forge-webservices
Copy link

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

@h-vetinari
Copy link
Member

@Mizux, there's a problem with CpSolverResponse it seems:

[119/504] Building CXX object ortools/constraint_solver/CMakeFiles/ortools_constraint_solver.dir/routing_breaks.cc.o
FAILED: ortools/constraint_solver/CMakeFiles/ortools_constraint_solver.dir/routing_breaks.cc.o 
$BUILD_PREFIX/bin/x86_64-conda-linux-gnu-c++ -DOR_TOOLS_AS_DYNAMIC_LIB -DUSE_BOP -DUSE_CBC -DUSE_CLP -DUSE_GLOP -DUSE_LP_PARSER -DUSE_PDLP -DUSE_SCIP -I$SRC_DIR -I$SRC_DIR/build-cpp -fvisibility-inlines-hidden -fmessage-length=0 -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem $PREFIX/include -fdebug-prefix-map=$SRC_DIR=/usr/local/src/conda/or-tools-package-9.7 -fdebug-prefix-map=$PREFIX=/usr/local/src/conda-prefix -O3 -DNDEBUG -std=c++17 -fPIC -fwrapv -MD -MT ortools/constraint_solver/CMakeFiles/ortools_constraint_solver.dir/routing_breaks.cc.o -MF ortools/constraint_solver/CMakeFiles/ortools_constraint_solver.dir/routing_breaks.cc.o.d -o ortools/constraint_solver/CMakeFiles/ortools_constraint_solver.dir/routing_breaks.cc.o -c $SRC_DIR/ortools/constraint_solver/routing_breaks.cc
In file included from $SRC_DIR/ortools/graph/graph.h:174,
                 from $SRC_DIR/ortools/constraint_solver/routing.h:188,
                 from $SRC_DIR/ortools/constraint_solver/routing_breaks.cc:28:
$SRC_DIR/ortools/graph/iterators.h:106:19: warning: 'template<class _Category, class _Tp, class _Distance, class _Pointer, class _Reference> struct std::iterator' is deprecated [-Wdeprecated-declarations]
  106 |     : public std::iterator<std::input_iterator_tag, IntegerType> {
      |                   ^~~~~~~~
In file included from $BUILD_PREFIX/x86_64-conda-linux-gnu/include/c++/12.3.0/bits/stl_algobase.h:65,
                 from $BUILD_PREFIX/x86_64-conda-linux-gnu/include/c++/12.3.0/algorithm:60,
                 from $SRC_DIR/ortools/constraint_solver/routing_breaks.cc:14:
$BUILD_PREFIX/x86_64-conda-linux-gnu/include/c++/12.3.0/bits/stl_iterator_base_types.h:127:34: note: declared here
  127 |     struct _GLIBCXX17_DEPRECATED iterator
      |                                  ^~~~~~~~
In file included from $PREFIX/include/absl/log/internal/strip.h:24,
                 from $PREFIX/include/absl/log/internal/check_op.h:37,
                 from $PREFIX/include/absl/log/internal/check_impl.h:19,
                 from $PREFIX/include/absl/log/check.h:37,
                 from $SRC_DIR/ortools/base/logging.h:23,
                 from $SRC_DIR/ortools/constraint_solver/routing_breaks.cc:25:
$PREFIX/include/absl/log/internal/log_message.h: In instantiation of 'absl::lts_20230125::log_internal::LogMessage& absl::lts_20230125::log_internal::LogMessage::operator<<(const T&) [with T = operations_research::sat::CpSolverResponse; typename std::enable_if<(! absl::lts_20230125::strings_internal::HasAbslStringify<T>::value), int>::type <anonymous> = 0]':
$SRC_DIR/ortools/constraint_solver/routing_lp_scheduling.h:592:16:   required from here
$PREFIX/include/absl/log/internal/log_message.h:289:17: error: no match for 'operator<<' (operand types are 'std::ostream' {aka 'std::basic_ostream<char>'} and 'const operations_research::sat::CpSolverResponse')
  289 |   view.stream() << log_internal::NullGuard<T>().Guard(v);
      |   ~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

In particular, it seems there are type conversions missing.

In file included from $PREFIX/include/absl/base/log_severity.h:19,
                 from $SRC_DIR/ortools/base/logging.h:21:
$BUILD_PREFIX/x86_64-conda-linux-gnu/include/c++/12.3.0/ostream:108:7: note: candidate: 'std::basic_ostream<_CharT, _Traits>::__ostream_type& std::basic_ostream<_CharT, _Traits>::operator<<(__ostream_type& (*)(__ostream_type&)) [with _CharT = char; _Traits = std::char_traits<char>; __ostream_type = std::basic_ostream<char>]'
  108 |       operator<<(__ostream_type& (*__pf)(__ostream_type&))
      |       ^~~~~~~~
$BUILD_PREFIX/x86_64-conda-linux-gnu/include/c++/12.3.0/ostream:108:36: note:   no known conversion for argument 1 from 'const operations_research::sat::CpSolverResponse' to 'std::basic_ostream<char>::__ostream_type& (*)(std::basic_ostream<char>::__ostream_type&)' {aka 'std::basic_ostream<char>& (*)(std::basic_ostream<char>&)'}
  108 |       operator<<(__ostream_type& (*__pf)(__ostream_type&))
      |                  ~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~
$BUILD_PREFIX/x86_64-conda-linux-gnu/include/c++/12.3.0/ostream:117:7: note: candidate: 'std::basic_ostream<_CharT, _Traits>::__ostream_type& std::basic_ostream<_CharT, _Traits>::operator<<(__ios_type& (*)(__ios_type&)) [with _CharT = char; _Traits = std::char_traits<char>; __ostream_type = std::basic_ostream<char>; __ios_type = std::basic_ios<char>]'
  117 |       operator<<(__ios_type& (*__pf)(__ios_type&))
      |       ^~~~~~~~
$BUILD_PREFIX/x86_64-conda-linux-gnu/include/c++/12.3.0/ostream:117:32: note:   no known conversion for argument 1 from 'const operations_research::sat::CpSolverResponse' to 'std::basic_ostream<char>::__ios_type& (*)(std::basic_ostream<char>::__ios_type&)' {aka 'std::basic_ios<char>& (*)(std::basic_ios<char>&)'}
  117 |       operator<<(__ios_type& (*__pf)(__ios_type&))
      |                  ~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~
$BUILD_PREFIX/x86_64-conda-linux-gnu/include/c++/12.3.0/ostream:127:7: note: candidate: 'std::basic_ostream<_CharT, _Traits>::__ostream_type& std::basic_ostream<_CharT, _Traits>::operator<<(std::ios_base& (*)(std::ios_base&)) [with _CharT = char; _Traits = std::char_traits<char>; __ostream_type = std::basic_ostream<char>]'
  127 |       operator<<(ios_base& (*__pf) (ios_base&))
      |       ^~~~~~~~
$BUILD_PREFIX/x86_64-conda-linux-gnu/include/c++/12.3.0/ostream:127:30: note:   no known conversion for argument 1 from 'const operations_research::sat::CpSolverResponse' to 'std::ios_base& (*)(std::ios_base&)'
  127 |       operator<<(ios_base& (*__pf) (ios_base&))
      |                  ~~~~~~~~~~~~^~~~~~~~~~~~~~~~~
$BUILD_PREFIX/x86_64-conda-linux-gnu/include/c++/12.3.0/ostream:166:7: note: candidate: 'std::basic_ostream<_CharT, _Traits>::__ostream_type& std::basic_ostream<_CharT, _Traits>::operator<<(long int) [with _CharT = char; _Traits = std::char_traits<char>; __ostream_type = std::basic_ostream<char>]'
  166 |       operator<<(long __n)
      |       ^~~~~~~~
$BUILD_PREFIX/x86_64-conda-linux-gnu/include/c++/12.3.0/ostream:166:23: note:   no known conversion for argument 1 from 'const operations_research::sat::CpSolverResponse' to 'long int'
  166 |       operator<<(long __n)
      |                  ~~~~~^~~
[...]

Also, does 9.7 only support protobuf >=4.23.3 or should I still expect it to be buildable (even if non-standard) with 3.21? We still haven't flipped the switch to stop building 3.21, so if possible I'd like to build for both (but not extremely important).

@Mizux
Copy link

Mizux commented Aug 9, 2023

IIRC this may due to:

  1. the update of re2 which change its type from re2::StringPiece to absl::string_view

  2. The bump of protobuf using the last abseil-cpp

Seems likely the 2, started from v9.7 we are using Protobuf v23.3 (i.e. abseil-cpp based version)...

@h-vetinari
Copy link
Member

the update of re2 which change its type from re2::StringPiece to absl::string_view

Conda-forge's re2 is using absl::string_view since conda-forge/re2-feedstock#62.

The bump of protobuf using the last abseil-cpp

Yeah, I copied the wrong error (was too busy this morning 🙈). It's fine to drop support for 3.21 then. Guess I'll have to dig into the errors on protobuf 4, which indeed look related to re2::StringPiece. Are you using a vendored re2 and/or abseil by any chance?

@Mizux
Copy link

Mizux commented Aug 9, 2023

for re2 not anymore, for abseil-cpp small patch https://github.com/google/or-tools/blob/stable/patches/abseil-cpp-20230125.3.patch (IIRC mostly add log because since cmake 3.19 )
check_cxx_source_compiles() do not take into account CMAKE_CXX_STANDARD and old debian gcc -> abseil-cpp was fallback to c++14 instead of c++17 (even if set CMAKE_STANDARD_CXX etc is set...) causing the build to fail.

Still need to investigate and send an error to kitware/abseil-cpp to confirm the behaviour and find a clean fix.

lopsided98 added a commit to lopsided98/nixpkgs that referenced this pull request Aug 23, 2023
Needs:
  * re2 update (in staging currently)
  * pybind11_protobuf (for Python bindings)
    - CMake build system is kind of broken:
      pybind/pybind11_protobuf#128
    - Needs protobuf proto_api.h, which isn't installed and upstream says
      wontfix: protocolbuffers/protobuf#9464

See also:
  * conda-forge/or-tools-feedstock#47
  * pybind/pybind11_protobuf#60
lopsided98 added a commit to lopsided98/nixpkgs that referenced this pull request Sep 2, 2023
Needs:
  * re2 update (in staging currently)
  * pybind11_protobuf (for Python bindings)
    - CMake build system is kind of broken:
      pybind/pybind11_protobuf#128
    - Needs protobuf proto_api.h, which isn't installed and upstream says
      wontfix: protocolbuffers/protobuf#9464

See also:
  * conda-forge/or-tools-feedstock#47
  * pybind/pybind11_protobuf#60
@h-vetinari
Copy link
Member

@Mizux this still fails with an incompatibility between absl::string_view and re2::StringPiece, which is strange to me, because we have an explicit test for that interoperability and it's passing.

$SRC_DIR/ortools/lp_data/lp_parser.cc: In function 'absl::lts_20230802::StatusOr<operations_research::glop::ParsedConstraint> operations_research::glop::ParseConstraint(absl::lts_20230802::string_view)':
$SRC_DIR/ortools/lp_data/lp_parser.cc:365:20: error: cannot convert 'absl::lts_20230802::string_view*' {aka 'std::basic_string_view<char>*'} to 'operations_research::glop::{anonymous}::StringPiece*' {aka 're2::StringPiece*'}
  365 |       ConsumeToken(&constraint, &consumed_name, &consumed_coeff);
      |                    ^~~~~~~~~~~
      |                    |
      |                    absl::lts_20230802::string_view* {aka std::basic_string_view<char>*}

check_cxx_source_compiles() do not take into account CMAKE_CXX_STANDARD and old debian gcc -> abseil-cpp was fallback to c++14 instead of c++17 (even if set CMAKE_STANDARD_CXX etc is set...) causing the build to fail.

Still need to investigate and send an error to kitware/abseil-cpp to confirm the behaviour and find a clean fix.

I'm not 100% I understand what you mean here - is there still something in or-tools that could force compilation with C++14? That would be a possible explanation for the ABI-mismatch.

@Mizux
Copy link

Mizux commented Sep 27, 2023

don't know if it help but on main we have:

%git grep StringPiece
ortools/lp_data/lp_parser.cc:using StringPiece = ::re2::StringPiece;

and we are using [0]─[~/work/main/build/_deps/re2-src]-[tags/2023-08-01]

cd build/_deps/re2-src
cat re2/stringpiece.h
...
namespace re2 {
// Until RE2 requires C++17 and uses std::string_view, allow users to
// continue to #include "re2/stringpiece.h" and use re2::StringPiece.
using StringPiece = absl::string_view;
}  // namespace re2

so ortools seems to use an re2 version which rely on absl::string_view (when using -DBUILD_DEPS=ON)
However, in your case i guess you are compiling ortools against the re2 version provided by conda -> may I have the version ?

note: https://github.com/google/re2/releases seems there is a 2023-09-01,
note2: we should release a v9.8 around october so if we can fix the re2 type mismatch before so conda could integrate it
and get rid of this v9.7 borken build ^^;

@h-vetinari
Copy link
Member

Thanks for the response!

so ortools seems to use an re2 version which rely on absl::string_view

It's my understanding that these should be compatible (when both compiled with C++17), see the comment above. So I'm surprised why this shouldn't work.

However, in your case i guess you are compiling ortools against the re2 version provided by conda -> may I have the version ?

Yes. It's currently 2023.03.01. Normally we're much faster with upgrading, but we have a discussion that got stuck in conda-forge/re2-feedstock#68, and I haven't had time/energy to get it unstuck again.

@Mizux
Copy link

Mizux commented Sep 27, 2023

Also or-tools need C++17 to compile (actually C++20 for msvc due to usage of "agregate initialization" (which is backported to -std=c++17 on clang/gcc since it comes from c99...)). Which means we are testing with abseil-cpp using c++17/c++20 so absl::string_view is an alias to std::string_view .

ref:

@h-vetinari
Copy link
Member

h-vetinari commented Sep 27, 2023

I'm aware of the ABI-implications of compiling abseil with C++17. ;-)

So since both of those types (re2::StringPiece and absl::string_view) should alias to std::string_view, I don't understand the incompatibility.

@Mizux
Copy link

Mizux commented Sep 27, 2023

https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=788588&view=logs&j=656edd35-690f-5c53-9ba3-09c10d0bea97&t=986b1512-c876-5f92-0d81-ba851554a0a3&l=1678 correctly see a -std=c++17
so or-tools seems compiled in C++17 which leads me to challenge the version of re2 headers used 🤔

@Mizux
Copy link

Mizux commented Sep 27, 2023

@Mizux
Copy link

Mizux commented Sep 27, 2023

FYI re2 perform the transition to absl::string_view alias in tag 2023-06-01 (which is the next release after re2 2023-03-01 so any release bump should fix this or-tools bug)
src: https://github.com/google/re2/blob/2023-06-01/re2/stringpiece.h

@h-vetinari
Copy link
Member

FYI re2 perform the transition to absl::string_view alias in tag 2023-06-01 (which is the next release after re2 2023-03-01 so any release bump should fix this or-tools bug)

Thanks. AFAIR the use of abseil types was already supported before that with an option, but it seems the first step should just be to unblock that re2 upgrade. :)

@h-vetinari
Copy link
Member

@Mizux, with the newer re2, this gets further, but now fails at

$SRC_DIR/ortools/sat/cuts.cc:79:36: error: no matching function for call to 'StrCat(const char [13], const absl::lts_20230802::int128&, const char [2])'
   79 |   std::string result = absl::StrCat("CutData rhs=", rhs, "\n");
      |                        ~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~

should the rhs variable be (cast to) a string type here, rather than being an integer reference?

@h-vetinari
Copy link
Member

Any ideas what's going on here @Mizux? :)

@Mizux
Copy link

Mizux commented Oct 30, 2023

@h-vetinari seems to comes from google/or-tools@734df45

https://github.com/google/or-tools/commits/main/ortools/sat/cuts.cc see around August 10th seems we had fought against abseil-cpp/re2 version bump...

Also you are using re2 2023-06-02 while v9.7 has been released with 2023-07-01
ref: https://github.com/google/or-tools/blob/v9.7/Dependencies.txt#L9

ps: For incoming v9.8 we will use re2 2023-09-01 (patch on main comming soon)...

@Mizux
Copy link

Mizux commented Nov 8, 2023

@h-vetinari
Concerning the std::string result = absl::StrCat("CutData rhs=", rhs, "\n");
due to abseil/abseil-cpp@34eb767 (20230802.1)
We had to remove our custom AbslStringify() overload for absl::int128
see google/or-tools@734df45 (main/v9.8)

So:

  • to build or-tools v9.7 you need abseil-cpp < 20230802.1,
  • to build or-tools main/v9.8+ you need abseil-cpp >= 20230802.1 and the commit above which is already in the branch.

note: in your Pending Dependency Version Updates please add the abseil-cpp version and re2 version used...

@h-vetinari
Copy link
Member

Sorry for the very long delay here. I finally unblocked newer re2, but by now, abseil has moved on. The lowest we can go is 20230802, though the migration to 20240116 is already in progress. In any case, I'm surprised by the following error.

$PREFIX/include/absl/strings/str_cat.h:468:8: error: invalid 'static_cast' from type 'const absl::lts_20230802::int128' to type 'const absl::lts_20230802::AlphaNum&'
  468 |        static_cast<const AlphaNum&>(args).Piece()...});
      |        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

AFAICT, this should be explicitly covered by abseil/abseil-cpp@34eb767, which is part of 20230802, but it seems that either abseil (or the compiler) don't think that the respective template applies, presumably because std::enable_if<strings_internal::HasAbslStringify<T>::value>::type> somehow wrongly evaluates to False?

PS. The errors for abseil 20240116 looked even worse (at first glance), so I made it possible to use newer re2 with the previous abseil as well. That said, if things don't work for 9.7, we can try 9.8 directly as well of course.

@Mizux
Copy link

Mizux commented Feb 27, 2024

@h-vetinari FYI we are stabilising or-tools for v9.9 release (hopefully this week) so you may try to bump directly to it.
As any release, we try to be up to date with last release of our dependencies:

C++

  • Protobuf=v25.3
  • abseil-cpp=20240116.1
  • re2=2024-02-01

Python

  • pybind11=v2.11.1
  • pybind11_abseil=52f2739
  • pybind11_protobuf=3b11990

Conda-forge ref

https://github.com/conda-forge/abseil-cpp-feedstock 20240116.1 up to date
https://github.com/conda-forge/protobuf-feedstock (note: current release v21.2 in README but you've merged conda-forge/protobuf-feedstock#212) up to date ?
conda-forge/re2-feedstock#75 (pending 2024-02-01 update)

@regro-cf-autotick-bot regro-cf-autotick-bot mentioned this pull request Mar 7, 2024
3 tasks
@regro-cf-autotick-bot regro-cf-autotick-bot mentioned this pull request May 8, 2024
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants