全文检索相关性使用手册
更新时间：2026-09-11
本文介绍如何在 Doris 中使用全文检索相关性评分，包括创建带分词器的倒排索引、查看分词结果、执行相关性排序查询，以及查看评分明细。
概述
全文检索用于在数据集中查找包含特定词项或短语的文档，并根据匹配程度对结果排序。
| 检索方式 | 优势 | 适用场景 |
|---|---|---|
| 全文检索 | “找准”：匹配规则可控、结果可解释，能够确保关键词命中和过滤条件的确定性 | 关键词检索、短语匹配、布尔条件过滤 |
| 向量搜索 | “找全”：利用语义相似性扩展召回范围 | 语义检索、近似匹配 |
在生成式 AI 应用，尤其是检索增强生成（RAG）场景中，全文检索和向量搜索可以协同使用：
- 全文检索提供词法精度和可解释的匹配结果。
- 向量搜索提供语义相关性和更广的召回范围。
- 两者结合可以为大模型提供更准确、更相关的上下文。
相关性评分
相关性评分用于量化搜索关键词与文档之间的匹配程度，并按照得分对结果排序。得分越高，表示文档与查询条件的匹配程度越高。
百度胜算的检索分析一体引擎 Doris 在社区 Doris 倒排索引的基础上实现了 BM25 相关性评分，可用于关键词检索、短语检索和多条件组合检索。
只有在目标列上创建了带
parser参数的倒排索引，才能使用相关性评分。
创建带评分能力的倒排索引
执行以下 SQL 创建示例表：
SQL
1CREATE TABLE test_score (
2 `F_id` char(32) NULL COMMENT,
3 `F_meta_set_id` char(6) NULL,
4 `F_content` text NULL,
5 INDEX idx_content (`F_content`) USING INVERTED PROPERTIES(
6 "parser" = "chinese",
7 "support_phrase" = "true",
8 "parser_mode" = "fine_grained"
9 ) COMMENT 'content索引'
10) ENGINE=OLAP
11DUPLICATE KEY(`F_id`, `F_meta_set_id`)
12COMMENT 'OLAP'
13DISTRIBUTED BY HASH(`F_id`) BUCKETS 1;其中，倒排索引的关键配置如下：
SQL
1INDEX idx_content (`F_content`) USING INVERTED PROPERTIES(
2 "parser" = "chinese",
3 "support_phrase" = "true",
4 "parser_mode" = "fine_grained"
5) COMMENT 'content索引'| 配置项 | 说明 |
|---|---|
parser |
指定分词器。本文示例使用中文分词器 chinese。 |
support_phrase |
是否支持短语匹配。本文示例设置为 true。 |
parser_mode |
指定分词模式。本文示例使用 fine_grained。 |
注意：如果倒排索引没有配置
parser参数，则不能使用相关性评分功能。
查看分词结果
分词器会将指定文本拆分为词项。可以使用 TOKENIZE() 函数查看不同分词模式下的结果：
SQL
1SELECT TOKENIZE(
2 '武汉长江大桥',
3 '"parser"="chinese","parser_mode"="fine_grained"'
4);
5
6SELECT TOKENIZE(
7 '武汉长江大桥',
8 '"parser"="chinese","parser_mode"="coarse_grained"'
9);通过对比分词结果，可以确认分词器是否符合当前数据和查询场景。
执行相关性评分查询
使用相关性评分时，查询需要同时满足以下条件：
WHERE子句包含全文检索表达式，例如MATCH、MATCH_ALL、MATCH_ANY或MATCH_PHRASE。ORDER BY子句的第一个排序表达式是__SCORE DESC。- 最好添加
LIMIT，以减少需要参与排序和返回的结果数量。
关键词匹配
使用 MATCH 查询包含“百度”的文档，并按相关性得分从高到低返回前 10 条：
SQL
1SELECT F_id, length(F_content), __SCORE
2FROM test_score
3WHERE F_content MATCH "百度"
4ORDER BY __SCORE DESC, F_id
5LIMIT 10;
6
7+----------------------------------+-------------------+--------------------+
8| F_id | length(F_content) | __SCORE |
9+----------------------------------+-------------------+--------------------+
10| 00bc5beb7352ccb09b631d301e7f2320 | 7933 | 3.7965037822723389 |
11| 00c4f4f3207e42c51f2cb5034d0a3ff8 | 4253 | 3.7931690216064453 |
12| 01b0915180f38a9a06108a3fe179099f | 2708 | 3.7900025844573975 |
13| 01f85b44af8717620bec712cd24eea51 | 4182 | 3.7899129390716553 |
14| 00408e93140526457d4b58dc6c374294 | 651 | 3.7877652645111084 |
15| 01577d8eca555ed548c267f382360125 | 2678 | 3.7875454425811768 |
16| 00e5b7b31382881d6ece211c2dab6918 | 4074 | 3.7870957851409912 |
17| 005018c3e1a7118ff5cc25a59d865236 | 5209 | 3.7866377830505371 |
18| 00c392c5a51532f2c6713783c76b5c9a | 6030 | 3.784865140914917 |
19| 00cf4c41f1061e0e1c593cc732b0af8a | 3002 | 3.784651517868042 |
20+----------------------------------+-------------------+--------------------+短语匹配
使用 MATCH_PHRASE 查询包含完整短语“百度知道”的文档：
SQL
1SELECT F_id, length(F_content), __SCORE
2FROM test_score
3WHERE F_content MATCH_PHRASE "百度知道"
4ORDER BY __SCORE DESC, F_id
5LIMIT 10;
6
7+----------------------------------+-------------------+--------------------+
8| F_id | length(F_content) | __SCORE |
9+----------------------------------+-------------------+--------------------+
10| 0055446903c7512318533bbd1287ce44 | 1864 | 6.6292381286621094 |
11| 00a34300c094e3552c8f0749ea712d03 | 3264 | 6.6196389198303223 |
12| 00e74c5df3d6f7d37ec1cf856e6397ad | 3301 | 6.577507495880127 |
13| 00e74c024f7268c96f614976447a22fa | 1316 | 6.5747184753417969 |
14| 016b83861b8cd9fbda5b3223fe6122f1 | 4302 | 6.5640044212341309 |
15| 010c765834acc2c09712157ec822aecc | 4652 | 6.5589470863342285 |
16| 01d90479e6c4bb16c02e6750acbc984a | 2050 | 6.5452632904052734 |
17| 01c490109f2646e835e925c6229ac2ed | 1396 | 6.5404181480407715 |
18| 014dd9559dcb02c565d918467566a9be | 1739 | 6.539583683013916 |
19| 00a5fabdd53f9d36853601c673f2512d | 2021 | 6.5395698547363281 |
20+----------------------------------+-------------------+--------------------+
2110 rows in set (0.80 sec)AND 条件匹配
使用多个全文检索条件，查询同时包含“百度”和“知道”的文档：
SQL
1SELECT F_id, length(F_content), __SCORE
2FROM test_score
3WHERE F_content MATCH "百度"
4 AND F_content MATCH "知道"
5ORDER BY __SCORE DESC, F_id
6LIMIT 10;
7
8+----------------------------------+-------------------+--------------------+
9| F_id | length(F_content) | __SCORE |
10+----------------------------------+-------------------+--------------------+
11| 0055446903c7512318533bbd1287ce44 | 1864 | 6.661651611328125 |
12| 00a34300c094e3552c8f0749ea712d03 | 3264 | 6.6296517848968506 |
13| 00a5fabdd53f9d36853601c673f2512d | 2021 | 6.62890362739563 |
14| 003bb89b5ea8a4d2e4d98813526a3124 | 3595 | 6.6162307262420654 |
15| 01a876ea622e5db23ad612a843ab720c | 2872 | 6.6099758148193359 |
16| 00e74c024f7268c96f614976447a22fa | 1316 | 6.6003093719482422 |
17| 01eb394371bb834132a3da01543965e3 | 2924 | 6.5982217788696289 |
18| 010c765834acc2c09712157ec822aecc | 4652 | 6.596583366394043 |
19| 00df35db6c5b34ed57dd896fa1bdca9d | 1863 | 6.5917825698852539 |
20| 0003fd7b5f9c2a16b6a94cd7df601bee | 3157 | 6.5908031463623047 |
21+----------------------------------+-------------------+--------------------+
2210 rows in set (0.41 sec)OR 条件匹配
使用 OR 查询包含“百度”或“知道”的文档：
SQL
1SELECT F_id, length(F_content), __SCORE
2FROM test_score
3WHERE F_content MATCH "百度"
4 OR F_content MATCH "知道"
5ORDER BY __SCORE DESC, F_id
6LIMIT 10;
7
8+----------------------------------+-------------------+--------------------+
9| F_id | length(F_content) | __SCORE |
10+----------------------------------+-------------------+--------------------+
11| 0055446903c7512318533bbd1287ce44 | 1864 | 6.661651611328125 |
12| 00a34300c094e3552c8f0749ea712d03 | 3264 | 6.6296517848968506 |
13| 00a5fabdd53f9d36853601c673f2512d | 2021 | 6.62890362739563 |
14| 003bb89b5ea8a4d2e4d98813526a3124 | 3595 | 6.6162307262420654 |
15| 01a876ea622e5db23ad612a843ab720c | 2872 | 6.6099758148193359 |
16| 00e74c024f7268c96f614976447a22fa | 1316 | 6.6003093719482422 |
17| 01eb394371bb834132a3da01543965e3 | 2924 | 6.5982217788696289 |
18| 010c765834acc2c09712157ec822aecc | 4652 | 6.596583366394043 |
19| 00df35db6c5b34ed57dd896fa1bdca9d | 1863 | 6.5917825698852539 |
20| 0003fd7b5f9c2a16b6a94cd7df601bee | 3157 | 6.5908031463623047 |
21+----------------------------------+-------------------+--------------------+
2210 rows in set (0.35 sec)解释评分明细
如果需要了解 __SCORE 的计算细节，可以在 SELECT 子句中增加 __SCORE_EXPLAIN：
SQL
1SELECT F_id,
2 length(F_content),
3 __SCORE,
4 __SCORE_EXPLAIN
5FROM test_score
6WHERE F_content MATCH "百度"
7ORDER BY __SCORE DESC, F_id
8LIMIT 10;
9
10+----------------------------------+-------------------+--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
11| F_id | length(F_content) | __SCORE | __SCORE_EXPLAIN |
12+----------------------------------+-------------------+--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
13| 00bc5beb7352ccb09b631d301e7f2320 | 7933 | 3.7965038673780445 | Score(F_content: 百度): 3.7965038673780445 product:
14 boost:1
15 idf, computed as log(1 + (N - n + 0.5) / (n + 0.5)) from: 3.85459
16 n, number of documents containing term: 943884
17 N, total number of documents with field: 44560188
18 tf, freq / (freq + k1 * (1 - b + b * dl / avgdl)): 0.98493
19 freq, occurrences of term within document: 107
20 k1, term saturation parameter: 1.2
21 b, length normalization parameter: 0.75
22 dl, length of field (approximate): 1560
23 avgdl, average length of field: 1049.99
24 |
25| 00c4f4f3207e42c51f2cb5034d0a3ff8 | 4253 | 3.7931689368851393 | Score(F_content: 百度): 3.7931689368851393 product:
26 boost:1
27 idf, computed as log(1 + (N - n + 0.5) / (n + 0.5)) from: 3.85459
28 n, number of documents containing term: 943884
29 N, total number of documents with field: 44560188
30 tf, freq / (freq + k1 * (1 - b + b * dl / avgdl)): 0.984065
31 freq, occurrences of term within document: 74
32 k1, term saturation parameter: 1.2
33 b, length normalization parameter: 0.75
34 dl, length of field (approximate): 1048
35 avgdl, average length of field: 1049.99
36 |
37| 01b0915180f38a9a06108a3fe179099f | 2708 | 3.7900026587744797 | Score(F_content: 百度): 3.7900026587744797 product:
38 boost:1
39 idf, computed as log(1 + (N - n + 0.5) / (n + 0.5)) from: 3.85459
40 n, number of documents containing term: 943884
41 N, total number of documents with field: 44560188
42 tf, freq / (freq + k1 * (1 - b + b * dl / avgdl)): 0.983243
43 freq, occurrences of term within document: 51
44 k1, term saturation parameter: 1.2
45 b, length normalization parameter: 0.75
46 dl, length of field (approximate): 664
47 avgdl, average length of field: 1049.99
48 |
49| 01f85b44af8717620bec712cd24eea51 | 4182 | 3.7899128725164184 | Score(F_content: 百度): 3.7899128725164184 product:
50 boost:1
51 idf, computed as log(1 + (N - n + 0.5) / (n + 0.5)) from: 3.85459
52 n, number of documents containing term: 943884
53 N, total number of documents with field: 44560188
54 tf, freq / (freq + k1 * (1 - b + b * dl / avgdl)): 0.98322
55 freq, occurrences of term within document: 67
56 k1, term saturation parameter: 1.2
57 b, length normalization parameter: 0.75
58 dl, length of field (approximate): 984
59 avgdl, average length of field: 1049.99
60 |
61| 00408e93140526457d4b58dc6c374294 | 651 | 3.7877651468446087 | Score(F_content: 百度): 3.7877651468446087 product:
62 boost:1
63 idf, computed as log(1 + (N - n + 0.5) / (n + 0.5)) from: 3.85459
64 n, number of documents containing term: 943884
65 N, total number of documents with field: 44560188
66 tf, freq / (freq + k1 * (1 - b + b * dl / avgdl)): 0.982663
67 freq, occurrences of term within document: 24
68 k1, term saturation parameter: 1.2
69 b, length normalization parameter: 0.75
70 dl, length of field (approximate): 144
71 avgdl, average length of field: 1049.99
72 |
73| 01577d8eca555ed548c267f382360125 | 2678 | 3.7875453339334335 | Score(F_content: 百度): 3.7875453339334335 product:
74 boost:1
75 idf, computed as log(1 + (N - n + 0.5) / (n + 0.5)) from: 3.85459
76 n, number of documents containing term: 943884
77 N, total number of documents with field: 44560188
78 tf, freq / (freq + k1 * (1 - b + b * dl / avgdl)): 0.982606
79 freq, occurrences of term within document: 46
80 k1, term saturation parameter: 1.2
81 b, length normalization parameter: 0.75
82 dl, length of field (approximate): 600
83 avgdl, average length of field: 1049.99
84 |
85| 00e5b7b31382881d6ece211c2dab6918 | 4074 | 3.7870957739553504 | Score(F_content: 百度): 3.7870957739553504 product:
86 boost:1
87 idf, computed as log(1 + (N - n + 0.5) / (n + 0.5)) from: 3.85459
88 n, number of documents containing term: 943884
89 N, total number of documents with field: 44560188
90 tf, freq / (freq + k1 * (1 - b + b * dl / avgdl)): 0.982489
91 freq, occurrences of term within document: 58
92 k1, term saturation parameter: 1.2
93 b, length normalization parameter: 0.75
94 dl, length of field (approximate): 856
95 avgdl, average length of field: 1049.99
96 |
97| 005018c3e1a7118ff5cc25a59d865236 | 5209 | 3.7866377869261569 | Score(F_content: 百度): 3.786637786926157 product:
98 boost:1
99 idf, computed as log(1 + (N - n + 0.5) / (n + 0.5)) from: 3.85459
100 n, number of documents containing term: 943884
101 N, total number of documents with field: 44560188
102 tf, freq / (freq + k1 * (1 - b + b * dl / avgdl)): 0.98237
103 freq, occurrences of term within document: 79
104 k1, term saturation parameter: 1.2
105 b, length normalization parameter: 0.75
106 dl, length of field (approximate): 1304
107 avgdl, average length of field: 1049.99
108 |
109| 00c392c5a51532f2c6713783c76b5c9a | 6030 | 3.7848650316018566 | Score(F_content: 百度): 3.7848650316018566 product:
110 boost:1
111 idf, computed as log(1 + (N - n + 0.5) / (n + 0.5)) from: 3.85459
112 n, number of documents containing term: 943884
113 N, total number of documents with field: 44560188
114 tf, freq / (freq + k1 * (1 - b + b * dl / avgdl)): 0.981911
115 freq, occurrences of term within document: 71
116 k1, term saturation parameter: 1.2
117 b, length normalization parameter: 0.75
118 dl, length of field (approximate): 1176
119 avgdl, average length of field: 1049.99
120 |
121| 00cf4c41f1061e0e1c593cc732b0af8a | 3002 | 3.7846515384885584 | Score(F_content: 百度): 3.7846515384885584 product:
122 boost:1
123 idf, computed as log(1 + (N - n + 0.5) / (n + 0.5)) from: 3.85459
124 n, number of documents containing term: 943884
125 N, total number of documents with field: 44560188
126 tf, freq / (freq + k1 * (1 - b + b * dl / avgdl)): 0.981855
127 freq, occurrences of term within document: 50
128 k1, term saturation parameter: 1.2
129 b, length normalization parameter: 0.75
130 dl, length of field (approximate): 728
131 avgdl, average length of field: 1049.99
132 |
133+----------------------------------+-------------------+--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
13410 rows in set (38.799 sec)__SCORE_EXPLAIN 会展示影响评分的主要因素，例如：
boost：字段或查询的权重。idf：词项在文档集合中的区分度。tf：词项在当前文档中的出现频率。dl：当前字段长度。avgdl：字段平均长度。k1、b：BM25 计算参数。
注意：增加
__SCORE_EXPLAIN会返回更多计算细节，可能导致查询变慢。只建议在调试、验证和问题排查时使用。
注意事项
- 如果出现
now version unsupportes score.，表示当前索引版本不支持相关性评分，需要重新导入数据或重建索引。 - 该功能不支持双向兼容：新版本可以读取旧版本数据，但旧版本无法读取新版本数据。
- 如果表中包含 JSON 列，不能在
SELECT表达式中指定 JSON 类型的列；表中存在 JSON 列时，不能使用SELECT *。 - 使用相关性评分时，必须确认查询列已创建带
parser参数的倒排索引。 ORDER BY的第一个排序表达式必须是__SCORE DESC，其他排序字段应放在其后。
评价此篇文章