captionbob.emoji.text_embedder

Text embedder: the meaning vector of a text in many languages.

The model intfloat/multilingual-e5-small puts a text of any of its languages in one space of meaning. Two texts with a close meaning have vectors with a high cosine, also when they are in two languages. So a keyword in Hindi can be compared with the English name of an emoji with no translation.

onnxruntime runs the ONNX file of the model. The GPU runs it when the installed onnxruntime has the CUDA provider and a GPU is present. Otherwise the CPU runs it. A loud log line names the provider that runs the model. The tokenizer file of the model gives the tokens of a text.

captionbob_env holds onnxruntime-gpu 1.26.0, which is built for CUDA 12 and cuDNN 9. The CUDA provider cannot start when the loader does not find cuDNN, and the pip package nvidia-cudnn-cu12 keeps it outside the paths of the loader. So before the GPU provider starts, onnxruntime.preload_dlls loads the CUDA and cuDNN libraries from the nvidia pip packages and then from the default paths. A library that it cannot load stops nothing: the CUDA provider then does not start, and the CPU runs the model.

onnxruntime.preload_dlls writes each library that it cannot load, and its other notes, with print to the standard output. The Python actions of the server write only JSON lines to the standard output, so the text embedder takes these lines and logs each one as a warning on the standard error.

The vector of a text is the mean of the vectors of its tokens, scaled to the length 1, as the pooling file of the model says. So the cosine of two vectors is their dot product.

Source file: captionbob/emoji/text_embedder.py

choose_providers

def choose_providers(available: Sequence[str]) -> list[str]

Choose the onnxruntime providers that run the model, best first.

Args:

• available: The providers of the installed onnxruntime.

Returns:

The GPU provider and then the CPU provider when the GPU provider is available. The CPU provider alone otherwise. When the GPU provider cannot start, for example with no GPU, onnxruntime runs the model with the CPU provider.

Raises:

• RuntimeError: When the installed onnxruntime has no CPU provider.

load_gpu_libraries

def load_gpu_libraries(providers: Sequence[str]) -> None

Load the CUDA and cuDNN libraries when the GPU provider comes first.

onnxruntime.preload_dlls prints each library that it cannot load to the standard output. Each printed line that is not blank is logged as a warning, and nothing reaches the standard output.

Args:

• providers: The providers that run the model, best first, as choose_providers gives them.

mean_pooled_vectors

def mean_pooled_vectors(
        token_vectors: np.ndarray, attention_mask: np.ndarray) -> np.ndarray

Give the vector of each text: the mean of its token vectors, scaled to the length 1.

Args:

• token_vectors: The vectors of the tokens, in the shape (texts, tokens, vector size).
• attention_mask: 1 for a token of the text and 0 for a pad token, in the shape (texts, tokens).

Returns:

One vector of length 1 for each text, in the shape (texts, vector size), as float32.

Raises:

• ValueError: When the shapes do not agree, or when a text has no token or a vector of length 0.

TextEmbedder

class TextEmbedder
    def __init__(self, model_folder: Path) -> None

Gives the meaning vectors of texts with the model of a model folder.

Args:

• model_folder: The folder that holds MODEL_FILE and TOKENIZER_FILE.

Raises:

• FileNotFoundError: When a file of the model is not there.
• ValueError: When the tokenizer has no pad token, or when the model takes an input that this class does not give.
• RuntimeError: When the installed onnxruntime has no CPU provider.

TextEmbedder.vector_size

@property
def vector_size(self) -> int

The count of the numbers in one vector.

TextEmbedder.embed

def embed(self, texts: Sequence[str]) -> np.ndarray

Give the meaning vector of each text.

Args:

• texts: The texts. The prefix QUERY_PREFIX is put in front of each one.

Returns:

One vector of length 1 for each text, in the shape (texts, vector size), as float32. No text gives the shape (0, vector size).