Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

new package: spleeter-proot #1037

Draft
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

knyipab
Copy link
Contributor

@knyipab knyipab commented May 22, 2024

spleeter won't be available any time soon due to the lack of tensorflow (depending on bazel to build), so I want to package this spleeter-proot relying on proot-distro, perhaps resembling containerized apps in Linux distro.

May I know the answers of few questions:

  • Will TUR accept a proot app package like this?
  • Does the build system offer "preinstall" script to check if app_spleeter prootf-distro rootfs exists during installation and if so then stop installation?
  • Does the build system offer "uninstaller" script to run proot-distro remove app_spleeter to clean the rootfs thoroughly?

@knyipab
Copy link
Contributor Author

knyipab commented May 26, 2024

@licy183 this PR build by action can never be run by native devices, prompting error something like conda python module not found. I suspect that is related to Linux user id stuff.

I really appreciate that you added bazel. I tried but tensorflow requires specific version of bazel (https://www.tensorflow.org/install/source#linux). For spleeter, it works the best with python3.10, tensorflow==2.9.0 which requires Bazel 5.0.0. I don't have much expertise, even looking at your build script, I don't know the why it needs two packages bazel and bazel7.

@licy183
Copy link
Collaborator

licy183 commented May 26, 2024

I'll try to compile bazel5 and add it.

I don't know the why it needs two packages bazel and bazel7.

bazel uses a script to launch. It reads from .bazelversion and then tries to find the specified version of bazel from the system. If you want to use another version, you could use export BAZEL_VERSION=xxx to bypass. bazel7 is packaged for further usage.

@knyipab
Copy link
Contributor Author

knyipab commented Jun 2, 2024

Thanks, I tried to build tensorflow v2.16.1 using bazel6 cuz the version matches. The tensorflow build script fetches the Bazel rules_python module in the WORKSPACE file (github link) and setup its own python3 (which seems to be glibc, output from file command: ELF 64-bit LSB pie executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, for GNU/Linux 3.7.0, BuildID[sha1]=2cc3e5b2773dc66c398da3cd9062100b4b61b734, with debug_info, not stripped). After that, it seems to fetch packages such as numpy from pypi. The error prompt encountered:

ERROR: /data/data/com.termux/files/home/projects/tensorflow/tensorflow/python/util/BUILD:737:18: //tensorflow/python/util:nest_util depends on @pypi_wrapt//:pkg in repository @pypi_wrapt which failed to fetch. no such package '@pypi_wrapt//': whl_library pypi_wrapt failed:  (src/main/tools/process-wrapper-legacy.cc:80: "execvp(/data/data/com.termux/files/home/.cache/bazel/_bazel_u0_a401/0f272fed90f6720052f9bef9084b447b/external/python_aarch64-unknown-linux-gnu/bin/python3, ...)": No such file or directory

It seems to me that to solve the problem, we will need to host our own rules_python with repositories.bzl (github link) patched to use TUR python virtual environment and TUR pypl, and handle the case when whl is not available. However, it doesn't look easy.

Also, FYR, these are commands I ran:

pkg install -y bazel bazel6
git clone https://github.com/tensorflow/tensorflow.git -b v2.16.1 --depth 1
cd tensorflow
./configure
### some interactive terminal configurations and then
bazel build //tensorflow/tools/pip_package:build_pip_package --repo_env=WHEEL_NAME=tensorflow_cpu

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants