Handle undefined model max length

[thanawan-atc]

Description

For some model, model.tokenizer.model_max_length is not properly defined (i.e., set to be a very large number). This led to truncation problems and caused model deployment to fail.

• For more context, see this huggingface/transformer issue.
• For some models, the problem has already been fixed directly to the model repo (e.g. intfloat/multilingual-e5-small), but some still have that problem (e.g. intfloat/e5-small-v2, gpt]

Issues Resolved

Helped in tracing intfloat/e5-small-v2 in #218

Check List

• Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[codecov]

Codecov Report

Merging #219 (f783502) into main (bd50b8b) will increase coverage by 0.01%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main     #219      +/-   ##
==========================================
+ Coverage   91.23%   91.25%   +0.01%     
==========================================
  Files          37       37              
  Lines        4131     4138       +7     
==========================================
+ Hits         3769     3776       +7     
  Misses        362      362              

Files Changed	Coverage Δ
...search_py_ml/ml_models/sentencetransformermodel.py	74.35% <100.00%> (+0.42%)	⬆️

[dhrubo-os]

Let's assign a static variable for this magic number and also add more comments including the github issue why we need this.

In addition, I assume this large is same for every cases, but I was wondering if we should do something like:

model.tokenizer.model_max_length = min (model.tokenizer.model_max_length, model.get_max_seq_length())

Can you please check if that's applicable for every cases?

[thanawan-atc]

Okay! I can add a static variable for this magic number and more comments.

From what I've seen, the number is the same.

For model.tokenizer.model_max_length = min (model.tokenizer.model_max_length, model.get_max_seq_length()), I agree that it is more generalizable. And from what I've checked, it should be applicable and beneficial.

[thanawan-atc]

I just made some changes. I use if model.tokenizer.model_max_lengt > model.get_max_seq_length() instead of min function, so that we can add print() to let user know what's happening.

[dhrubo-os]

Thanks for raising this PR.