How to start embedding models

General
1

提个问题 vllm如何跑 embadding 用正常跑模型的参数 貌似无法跑embadding模型 求大佬解答
A question: How to run embadding with vllm? I can’t seem to run an embadding model with the normal model parameters, so I’m looking for an answer.

2

you can check Pooling Models — vLLM

3

这个没有看明白用途，有使用案例可以参考么！！

4

better to use english in this forum.

5

Didn’t see the use for this one, is there a use case to look at!

6

It is possible to calculate similarity based on the hidden state embedding for recall.

7

You can follow the example at vllm/examples/offline_inference/basic/embed.py at main · vllm-project/vllm · GitHub

1 Like
8

It is possible to calculate similarity based on the hidden state embedding for recall.

vLLM supports LLM.score, please check out this example: vllm/examples/offline_inference/basic/score.py at main · vllm-project/vllm · GitHub

Our OpenAI-compatible server also has a Score API that does the same thing.

1 Like