Rerankers

Reranker 模型通常是具有 1 个输出类别的 CrossEncoder 模型，即给定一对文本（查询，答案），模型输出一个分数。这个分数，要么是合理范围在 -10.0 到 10.0 之间的浮点分数，要么是范围在 0…1 之间的分数，表示答案在多大程度上有助于回答查询。

许多 reranker 模型都在 MS MARCO 上训练

但最有可能的是，在您自己的数据集上训练会获得最佳结果。因此，本页面包含一些您可以应用于自己数据的训练脚本示例

BinaryCrossEntropyLoss

BinaryCrossEntropyLoss 是一个非常强大而又简单的损失函数。给定文本对（例如（查询，答案）对），此损失使用 CrossEncoder 模型计算预测分数。它将这些分数与黄金（或白银，即由某些模型确定）标签进行比较，模型表现越好，计算出的损失越低。

CachedMultipleNegativesRankingLoss

CachedMultipleNegativesRankingLoss (又称带 GradCache 的 InfoNCE) 比常见的 BinaryCrossEntropyLoss 更复杂。它接受正对（即（查询，答案）对）或三元组（即（查询，正确答案，错误答案）三元组），然后通过从批次中其他问题中选择答案，随机为每个查询找到 num_negatives 个额外的错误答案。这通常被称为“批内负样本”。

然后，损失函数将计算所有（查询，答案）对的分数，包括刚刚选择的错误答案对。然后，损失函数将使用交叉熵损失来确保（查询，正确答案）的分数高于所有（随机选择的）错误答案的（查询，错误答案）的分数。

CachedMultipleNegativesRankingLoss 使用一种名为 GradCache 的方法，允许在不显着增加内存使用量的情况下在小批量中计算分数。建议使用此损失而不是“标准” MultipleNegativesRankingLoss (又称 InfoNCE) 损失，因为后者没有这种巧妙的小批量支持，因此需要大量内存。

对于此损失，有必要对 activation_fn 和 scale 进行实验。torch.nn.Sigmoid 与 scale=10.0 效果不错，torch.nn.Identity` 与 scale=1.0 也有效，而 mGTE 论文作者建议使用 torch.nn.Tanh 与 scale=10.0。

推理

tomaarsen/reranker-ModernBERT-base-gooaq-bce 模型是使用第一个脚本训练的。如果您想在自己训练之前尝试该模型，请随意使用此脚本

from sentence_transformers import CrossEncoder

# Download from the 🤗 Hub
model = CrossEncoder("tomaarsen/reranker-ModernBERT-base-gooaq-bce")

# Get scores for pairs of texts
pairs = [
    ["how to obtain a teacher's certificate in texas?", 'Some aspiring educators may be confused about the difference between teaching certification and teaching certificates. Teacher certification is another term for the licensure required to teach in public schools, while a teaching certificate is awarded upon completion of an academic program.'],
    ["how to obtain a teacher's certificate in texas?", '["Step 1: Obtain a Bachelor\'s Degree. One of the most important Texas teacher qualifications is a bachelor\'s degree. ... ", \'Step 2: Complete an Educator Preparation Program (EPP) ... \', \'Step 3: Pass Texas Teacher Certification Exams. ... \', \'Step 4: Complete a Final Application and Background Check.\']'],
    ["how to obtain a teacher's certificate in texas?", "Washington Teachers Licensing Application Process Official transcripts showing proof of bachelor's degree. Proof of teacher program completion at an approved teacher preparation school. Passing scores on the required examinations. Completed application for teacher certification in Washington."],
    ["how to obtain a teacher's certificate in texas?", 'Teacher education programs may take 4 years to complete after which certification plans are prepared for a three year period. During this plan period, the teacher must obtain a Standard Certification within 1-2 years. Learn how to get certified to teach in Texas.'],
    ["how to obtain a teacher's certificate in texas?", 'In Texas, the minimum age to work is 14. Unlike some states, Texas does not require juvenile workers to obtain a child employment certificate or an age certificate to work. A prospective employer that wants one can request a certificate of age for any minors it employs, obtainable from the Texas Workforce Commission.'],
]
scores = model.predict(pairs)
print(scores)
# [0.00121048 0.97105724 0.00536712 0.8632406  0.00168043]

# Or rank different texts based on similarity to a single text
ranks = model.rank(
    "how to obtain a teacher's certificate in texas?",
    [
        "[\"Step 1: Obtain a Bachelor's Degree. One of the most important Texas teacher qualifications is a bachelor's degree. ... \", 'Step 2: Complete an Educator Preparation Program (EPP) ... ', 'Step 3: Pass Texas Teacher Certification Exams. ... ', 'Step 4: Complete a Final Application and Background Check.']",
        "Teacher education programs may take 4 years to complete after which certification plans are prepared for a three year period. During this plan period, the teacher must obtain a Standard Certification within 1-2 years. Learn how to get certified to teach in Texas.",
        "Washington Teachers Licensing Application Process Official transcripts showing proof of bachelor's degree. Proof of teacher program completion at an approved teacher preparation school. Passing scores on the required examinations. Completed application for teacher certification in Washington.",
        "Some aspiring educators may be confused about the difference between teaching certification and teaching certificates. Teacher certification is another term for the licensure required to teach in public schools, while a teaching certificate is awarded upon completion of an academic program.",
        "In Texas, the minimum age to work is 14. Unlike some states, Texas does not require juvenile workers to obtain a child employment certificate or an age certificate to work. A prospective employer that wants one can request a certificate of age for any minors it employs, obtainable from the Texas Workforce Commission.",
    ],
)
print(ranks)
# [
#     {'corpus_id': 0, 'score': 0.97105724},
#     {'corpus_id': 1, 'score': 0.8632406},
#     {'corpus_id': 2, 'score': 0.0053671156},
#     {'corpus_id': 4, 'score': 0.0016804343},
#     {'corpus_id': 3, 'score': 0.0012104829},
# ]