Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix VarBuilder handling for custom code GTE variants #473

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

alvarobartt
Copy link
Member

@alvarobartt alvarobartt commented Jan 16, 2025

What does this PR do?

This PR fixes how the VarBuilder loading is being handled for GTEModel since some models such as Alibaba-NLP/gte-multilingual-base require custom code (available at https://huggingface.co/Alibaba-NLP/new-impl) using the base_model_prefix="new" as per this line, meaning that the safetensors weights are not stored as e.g. embeddings.word_embeddings like in other variants of GTE as e.g. Alibaba-NLP/gte-base-en-v1.5, but as new.embeddings.word_embeddings.

Fixes #460
Fixes #471

Note

Some of the changes proposed in this PR are pretty specific to some of the Alibaba NLP model variants and should or could be fixed within the repositories, but since those are widely used e.g. Alibaba-NLP/gte-multilingual-base has over 500k monthly downloads on the Hub, being on the top 100 models on the MTEB Leaderboard, and showing as text-embeddings-inference compatible on the Hub; we could ship this as a tiny QoL improvement causing less friction when deploying TEI models from the Hub (it can now be deployed but setting refs/pr/7 as the revision as e.g. text-embeddings-router --model-id Alibaba-NLP/gte-multilingual-base --port 8081).

WDYT @OlivierDehaene? Thanks in advance!

See below the following deployments:

  • Alibaba-NLP/gte-multilingual-base: image
  • Alibaba-NLP/gte-multilingual-reranker-base: image

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you read the contributor guideline, Pull Request section?
  • Was this discussed/approved via a Github issue or the forum? Please add a link to it if that's the case.
  • Did you make sure to update the documentation with your changes? Here are the documentation guidelines, and here are tips on formatting docstrings.
  • Did you write any new necessary tests?

Who can review?

@OlivierDehaene

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
1 participant