Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: Multi-arch Beam SDK Python 3.11 container missing libgeos-dev for ARM #28035

Closed
1 of 15 tasks
celeste-zeng opened this issue Aug 16, 2023 · 0 comments · Fixed by #28036
Closed
1 of 15 tasks

[Bug]: Multi-arch Beam SDK Python 3.11 container missing libgeos-dev for ARM #28035

celeste-zeng opened this issue Aug 16, 2023 · 0 comments · Fixed by #28036
Labels
bug done & done Issue has been reviewed after it was closed for verification, followups, etc. P2 python

Comments

@celeste-zeng
Copy link
Contributor

What happened?

Used a multi-arch Beam SDK Python 3.11 container to run a Dataflow pipeline on ARM and the container failed to start due to the error OSError: Could not find lib geos_c or load any of its variants ['libgeos_c.so.1', 'libgeos_c.so'].

Reproduce steps:

  1. Create an ARM VM with docker installed;
  2. Pull and run the multi-arch Beam SDK Python 3.11 container: docker run -it --entrypoint=/bin/bash gcr.io/dataflow-build/siyuez/mock-release/beam_python3.11_sdk:release-test;
  3. Use the python interpreter, run: python;
  4. then run: from apache_beam.io.gcp import bigquery_schema_tools.
    The error will appear: File "/usr/local/lib/python3.11/site-packages/shapely/geos.py", line 54, in load_dll raise OSError( OSError: Could not find lib geos_c or load any of its variants ['libgeos_c.so.1', 'libgeos_c.so'].

On the VM, the error can be fixed by installing the libgeos-dev package: run apt update and thenapt install libgeos-dev. The package takes 4327 kB. We should install it in the Dockerfile for Python 3.11 when the target architecture is ARM.

Issue Priority

Priority: 2 (default / most bugs should be filed as P2)

Issue Components

  • Component: Python SDK
  • Component: Java SDK
  • Component: Go SDK
  • Component: Typescript SDK
  • Component: IO connector
  • Component: Beam examples
  • Component: Beam playground
  • Component: Beam katas
  • Component: Website
  • Component: Spark Runner
  • Component: Flink Runner
  • Component: Samza Runner
  • Component: Twister2 Runner
  • Component: Hazelcast Jet Runner
  • Component: Google Cloud Dataflow Runner
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug done & done Issue has been reviewed after it was closed for verification, followups, etc. P2 python
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants