Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

tensorflow==2.3.0 ? #6

Closed
PhVerkerk opened this issue Jan 13, 2022 · 13 comments · Fixed by #7
Closed

tensorflow==2.3.0 ? #6

PhVerkerk opened this issue Jan 13, 2022 · 13 comments · Fixed by #7

Comments

@PhVerkerk
Copy link

Hi,
I am trying to install Latin-BERT from scratch and it seems that your requirements (requirements.txt) are too stringent and/or obsolete.

ERROR: Could not find a version that satisfies the requirement tensorflow==2.3.0 (from versions: 2.5.0rc0, 2.5.0rc1, 2.5.0rc2, 2.5.0rc3, 2.5.0, 2.5.1, 2.6.0rc0, 2.6.0rc1, 2.6.0rc2, 2.6.0, 2.6.2, 2.7.0rc0, 2.7.0rc1, 2.7.0)
ERROR: No matching distribution found for tensorflow==2.3.0

Well, too bad ! I upgraded to 2.5.0, but obviously things turned badly again, with numpy.

INFO: pip is looking at multiple versions of tensor2tensor to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of numpy to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of <Python from Requires-Python> to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of future to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of cltk to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of beautifulsoup4 to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of pygame to determine which version is compatible with other requirements. This could take a while.
ERROR: Cannot install -r requirements.txt (line 6), -r requirements.txt (line 7) and numpy==1.18.5 because these package versions have conflicting dependencies.

The conflict is caused by:
    The user requested numpy==1.18.5
    tensor2tensor 1.15.7 depends on numpy
    tensorflow 2.5.0 depends on numpy~=1.19.2

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/user_guide/#fixing-conflicting-dependencies

As I am a new-comer in this field, I'd like to have your advice on the way to solve these conflicting dependencies.

A time interval of 16 months is larger than the life-span of a python package ?

Yours,

Philippe.
@diyclassics diyclassics mentioned this issue Jan 26, 2022
Merged
@diyclassics
Copy link
Collaborator

diyclassics commented Jan 26, 2022 •
edited
Loading

@PhVerkerk I have updated the installation instructions, dependencies (thanks @todd-cook and PR #5), and also made a note that the current installation to note that LatinBERT is tested on Python 3.7 & 3.8 (and does not work on Python 3.9+). Can you try to install again with this setup and let me know if it works? PJB

@diyclassics diyclassics reopened this Jan 26, 2022
@PhVerkerk
Copy link
Author

PhVerkerk commented Jan 27, 2022 via email

@PhVerkerk
Copy link
Author

As a matter of fact, the line "tensorflow==2.3.0" is now fine (I moved back to Python 3.8.10 which is the last 3.8.* version with binary installers). But the explosion now occurs in "tokenizers==0.8.1rc2".
Something also with "atari-py==0.2.6", but I don't know where it comes from (i.e. not in your requirements.txt)

(Latin_BERT) Philippes-MacBook-Pro:latin-bert-master Philippe$ pip install tokenizers==0.8.1rc2
Collecting tokenizers==0.8.1rc2
  Using cached tokenizers-0.8.1rc2.tar.gz (97 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: tokenizers
  Building wheel for tokenizers (pyproject.toml) ... error
  ERROR: Command errored out with exit status 1:
   command: /Users/Philippe/opt/anaconda3/envs/Latin_BERT/bin/python /Users/Philippe/opt/anaconda3/envs/Latin_BERT/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py build_wheel /var/folders/y_/skp1d_jx6m1cq6twqfllxc080000gn/T/tmpcc69nq1m
[...]
  error: cargo failed with code: 101
 
  ----------------------------------------
  ERROR: Failed building wheel for tokenizers
Failed to build tokenizers
ERROR: Could not build wheels for tokenizers, which is required to install pyproject.toml-based projects

The "[...]" in the code above stands for pages and pages of red prints, probably intended for non-human brains.

Ph.

@PhVerkerk
Copy link
Author

I don't know if it helps, but in the insane lines shortened with "[...]", there are some more explicit comments (that I am unable to decrypt):

  error[E0658]: `if` is not allowed in a `const fn`
    --> /Users/Philippe/.cargo/registry/src/gitpro.ttaallkk.top-1ecc6299db9ec823/clap-2.34.0/src/app/settings.rs:7:1
     |
  7  | / bitflags! {
  8  | |     struct Flags: u64 {
  9  | |         const SC_NEGATE_REQS       = 1;
  10 | |         const SC_REQUIRED          = 1 << 1;
  ...  |
  51 | |     }
  52 | | }
     | |_^
     |
     = note: see issue #49146 <https://github.com/rust-lang/rust/issues/49146> for more information
     = help: add `#![feature(const_if_match)]` to the crate attributes to enable
     = note: this error originates in a macro (in Nightly builds, run with -Z macro-backtrace for more info)
  
  error[E0658]: `if` is not allowed in a `const fn`
    --> /Users/Philippe/.cargo/registry/src/gitpro.ttaallkk.top-1ecc6299db9ec823/clap-2.34.0/src/args/settings.rs:6:1
     |
  6  | / bitflags! {
  7  | |     struct Flags: u32 {
  8  | |         const REQUIRED         = 1;
  9  | |         const MULTIPLE         = 1 << 1;
  ...  |
  28 | |     }
  29 | | }
     | |_^
     |
     = note: see issue #49146 <https://github.com/rust-lang/rust/issues/49146> for more information
     = help: add `#![feature(const_if_match)]` to the crate attributes to enable
     = note: this error originates in a macro (in Nightly builds, run with -Z macro-backtrace for more info)

and later

  error: aborting due to 2 previous errors
  
  For more information about this error, try `rustc --explain E0658`.
  error: could not compile `clap`.

But I was not able to understand the answer to rustc --explain E0658.

Ph.

@diyclassics
Copy link
Collaborator

Thank you for trying the new setup—can you let me know what version of rustc you are running? (I have installed so far with 1.50.0 and 1.58.1.)

@PhVerkerk
Copy link
Author

PhVerkerk commented Jan 27, 2022 via email

@maxisawesome
Copy link

maxisawesome commented May 8, 2022 •
edited
Loading

Hey - I know this is old but I ran into this problem and managed to get things running!

Step 1 I needed to do conda create --name latinbert python=3.8 based on @diyclassics ' comment above (lol ur name which I just noticed).
Step 2 I did pip install torch
Step 3 failed for me. Installing all the pip packages at once with pip install -r requirements.txt seemed to send me into some sort of dependency circle. It wasn't clear to me but it failed to install some packages, continued to attempt to resolve dependency issues for a long time (15+ min), and errored on install of many packages. At one point I too was told I didn't have the proper rust install but I think that was bcus python was 3.10. unfortunately can't recall exactly. The solution for me was the pip install package==version for each package in requirements.txt. Despite some warnings, it seems to do fine
Step 4 same as readme
Step 5 I needed to run brew install wget but after that the script worked fine

For clarity's sake, and because I know some people who want to use this will not be very familiar with a lot of these concepts (like conda or packages), these were the commands I ran:

conda create --name latinbert python=3.8
conda activate latinbert
pip install torch
pip install pygame==2.0.0.dev6
pip install beautifulsoup4==4.9.1
pip install cltk==1.0.15
pip install future==0.18.2
pip install numpy==1.18.5
pip install tensor2tensor==1.15.7
pip install tensorflow==2.3.0
pip install tokenizers==0.8.1rc2
pip install transformers==3.1.0
pip install tqdm==4.48.2
pip install Unidecode==1.1.1
python3 -c "from cltk.data.fetch import FetchCorpus; corpus_downloader = FetchCorpus(language='lat');corpus_downloader.import_corpus('lat_models_cltk')"
./scripts/download.sh
python3 scripts/gen_berts.py --bertPath models/latin_bert/ --tokenizerPath models/subword_tokenizer_latin/latin.subword.encoder > berts.output.txt

@PhVerkerk
Copy link
Author

PhVerkerk commented May 8, 2022 via email

@maxisawesome
Copy link

Great :) LMK if it works. It worked for my sister on her computer, but another confirmation would be nice.
I'd open a pull request to change the README instructions, but another confirmation that this works right would be nice, and having a script that does all the pip installs instead of a requirements.txt just feels wrong. Willing to do that if the authors think its the right choice!

@PhVerkerk
Copy link
Author

PhVerkerk commented May 9, 2022 via email

@PhVerkerk
Copy link
Author

PhVerkerk commented May 9, 2022 via email

@PhVerkerk
Copy link
Author

PhVerkerk commented Oct 11, 2022 via email

@diyclassics
Copy link
Collaborator

Happy to hear that it is working—sorry it was as difficult as it was. I will review this thread and see if the installation instructions can be made clearer. I am going to close this issue—please reopen if other installation issues (esp. with the tensorflow dependency) arise.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Update install instructions; add Python version info diyclassics/latin-bert
3 participants
@diyclassics @PhVerkerk @maxisawesome