Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Generalize make_model_config_json function #200

Merged
merged 6 commits into from
Jul 27, 2023
Merged

Generalize make_model_config_json function #200

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
26165f6
Improve make_model_config
thanawan-atc Jul 26, 2023
f372ecb
Update sentencetransformermodel.py
thanawan-atc Jul 26, 2023
4c5bea3
Update CHANGELOG.md
thanawan-atc Jul 26, 2023
a9694b0
Update CHANGELOG.md
thanawan-atc Jul 26, 2023
1221733
Update CHANGELOG.md
thanawan-atc Jul 26, 2023
9b9adfb
Update CHANGELOG.md
thanawan-atc Jul 26, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,8 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
- Fix ModelUploader bug & Update model tracing demo notebook by @thanawan-atc in ([#185](https://github.com/opensearch-project/opensearch-py-ml/pull/185))
- Fix make_model_config_json function by @thanawan-atc in ([#188](https://github.com/opensearch-project/opensearch-py-ml/pull/188))
- Make make_model_config_json function more concise by @thanawan-atc in ([#191](https://github.com/opensearch-project/opensearch-py-ml/pull/191))
- Enabled auto-truncation for any pretrained models ([#192]https://github.com/opensearch-project/opensearch-py-ml/pull/192)
- Enabled auto-truncation for any pretrained models by @Yerzhaisang in ([#192](https://github.com/opensearch-project/opensearch-py-ml/pull/192))
- Generalize make_model_config_json function by @thanawan-atc in ([#200](https://github.com/opensearch-project/opensearch-py-ml/pull/200))

## [1.0.0]

Expand Down
30 changes: 11 additions & 19 deletions opensearch_py_ml/ml_models/sentencetransformermodel.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1068,28 +1068,20 @@ def make_model_config_json(
or normalize_result is None
):
try:
if (
model_type is None
and len(model._modules) >= 1
and isinstance(model._modules["0"], Transformer)
):
model_type = model._modules["0"].auto_model.__class__.__name__
model_type = model_type.lower().rstrip("model")
if embedding_dimension is None:
embedding_dimension = model.get_sentence_embedding_dimension()
if (
pooling_mode is None
and len(model._modules) >= 2
and isinstance(model._modules["1"], Pooling)
):
pooling_mode = model._modules["1"].get_pooling_mode_str().upper()
if normalize_result is None:
if len(model._modules) >= 3 and isinstance(
model._modules["2"], Normalize
):

for str_idx, module in model._modules.items():
if model_type is None and isinstance(module, Transformer):
model_type = module.auto_model.__class__.__name__
model_type = model_type.lower().rstrip("model")
elif pooling_mode is None and isinstance(module, Pooling):
pooling_mode = module.get_pooling_mode_str().upper()
elif normalize_result is None and isinstance(module, Normalize):
normalize_result = True
else:
normalize_result = False
# TODO: Support 'Dense' module
if normalize_result is None:
normalize_result = False
except Exception as e:
raise Exception(
f"Raised exception while getting model data from pre-trained hugging-face model object: {e}"
Expand Down
Loading