向量检索支持SQL全集
本文汇总 Doris 向量检索当前支持和不支持的 SQL 写法,包含建表 DDL、查询语法、多路召回与重排序示例,供开发和测试对照使用。
概览
Doris 是一款集向量检索、全文检索、OLAP 分析功能于一体的数据库,其中存算分离版具有高可用、高性能、低成本的特点。向量检索既可以用标准 SQL 完成,也可以通过 SDK 获得更高 QPS。
| 功能 | 说明 |
|---|---|
| 向量维度 | 2-32,768(更高维度也支持) |
| 度量类型 | 余弦、IP(点积)、L2(欧几里得)、BM25(全文检索) |
| 索引类型 | HNSW、DiskANN、Puck(IVF_PQ) |
| 搜索类型 | 向量检索、标量过滤、全文检索、OLAP 分析 |
| 全文检索 | 支持,BM25 算法 |
| 支持的 SDK | 标准 SQL、JDBC/ODBC、Python/Go SDK |
DDL
索引属性说明
| 属性 | 说明 |
|---|---|
| 索引名称 | 如 idx_ann,在 INDEX 关键字后指定 |
algorithm |
向量索引算法,示例中使用 hnsw |
dimension |
指定向量维度 |
distance |
指定向量距离计算方式,示例中使用 l2 |
Duplicate 表
1CREATE TABLE `ann_table` (
2 `F_id` int NOT NULL COMMENT 'id',
3 `F_meta_id` char(6) NOT NULL COMMENT 'meta id',
4 `F_content` text NULL COMMENT 'content1',
5 `embedding` array<float> NOT NULL COMMENT 'embedding',
6 `len` INT DEFAULT '1' COMMENT 'meta length',
7 INDEX idx_ann (`embedding`) USING ANN PROPERTIES("algorithm"="hnsw", 'dimension'='4', 'distance'='l2') COMMENT 'hnsw index'
8) ENGINE=OLAP
9DUPLICATE KEY(`F_id`)
10COMMENT 'OLAP'
11DISTRIBUTED BY HASH(`F_id`) BUCKETS 2
12PROPERTIES (
13);
Unique 表
与 Duplicate 表的差别只在 KEY 类型,索引定义方式相同。
1CREATE TABLE `ann_table_unique` (
2 `F_id` int NOT NULL COMMENT 'id',
3 `F_meta_id` char(6) NOT NULL COMMENT 'meta id',
4 `F_content` text NULL COMMENT 'content1',
5 `embedding` array<float> NOT NULL COMMENT 'embedding',
6 `len` INT DEFAULT '1' COMMENT 'meta length',
7 INDEX idx_ann (`embedding`) USING ANN PROPERTIES("algorithm"="hnsw", 'dimension'='4', 'distance'='l2') COMMENT 'hnsw index'
8) ENGINE=OLAP
9UNIQUE KEY(`F_id`)
10COMMENT 'OLAP'
11DISTRIBUTED BY HASH(`F_id`) BUCKETS 2
12PROPERTIES (
13);
多列索引
支持为多个字段分别创建向量 ANN 索引、文本倒排索引,查询时可组合多个索引实现混合检索,提升多条件查询性能。
1CREATE TABLE `hybrid_dup_table` (
2 `id` int NOT NULL COMMENT 'id',
3 `name` varchar(50) NOT NULL COMMENT 'name',
4 `age` int NULL COMMENT 'age',
5 `embedding1` array<float> NOT NULL COMMENT 'embedding1',
6 `embedding2` array<float> NOT NULL COMMENT 'embedding2',
7 `embedding3` array<float> NOT NULL COMMENT 'embedding2',
8 `content1` String NOT NULL COMMENT 'content1',
9 `content2` String NOT NULL COMMENT 'content2',
10 INDEX idx_ann1 (`embedding1`) USING ANN PROPERTIES("algorithm"="hnsw", 'hnsw_m'='1000', 'hnsw_efConstruction'='2000', 'dimension'='4', 'distance'='l2') COMMENT 'hnsw index',
11 INDEX idx_ann2 (`embedding2`) USING ANN PROPERTIES("algorithm"="hnsw", 'hnsw_m'='1000', 'hnsw_efConstruction'='2000', 'dimension'='4', 'distance'='l2') COMMENT 'hnsw index',
12 INDEX idx_ann3 (`embedding3`) USING ANN PROPERTIES("algorithm"="hnsw", 'hnsw_m'='1000', 'hnsw_efConstruction'='2000', 'dimension'='4', 'distance'='l2') COMMENT 'hnsw index',
13 INDEX idx_content1 (`content1`) USING INVERTED PROPERTIES("parser" = "english") COMMENT 'inverted index',
14 INDEX idx_content2 (`content2`) USING INVERTED PROPERTIES("parser" = "english") COMMENT 'inverted index'
15) ENGINE=OLAP
16DUPLICATE KEY(`id`)
17COMMENT 'OLAP'
18DISTRIBUTED BY HASH(`id`) BUCKETS 2
19PROPERTIES (
20);
全文索引
支持对文本字段创建倒排全文索引,可实现关键词、短语检索,同时可搭配向量索引完成文本 + 向量混合检索。
1CREATE TABLE `docs` (
2 `id` int NOT NULL,
3 `title` varchar(50) NOT NULL,
4 `content` text NOT NULL,
5 INDEX idx_content (`content`) USING INVERTED PROPERTIES("parser" = "english", "lower_case" = "true", "support_phrase" = "true")
6) ENGINE=OLAP
7DUPLICATE KEY(`id`)
8DISTRIBUTED BY HASH(`id`) BUCKETS 1;
9
10CREATE TABLE `emb` (
11 `doc_id` int NOT NULL,
12 `tag` varchar(50) NOT NULL,
13 `embedding` array<float> NOT NULL,
14 INDEX idx_ann (`embedding`) USING ANN PROPERTIES("hnsw_m" = "1000", "distance" = "l2", "dimension" = "4", "algorithm" = "hnsw", "hnsw_efConstruction" = "2000")
15) ENGINE=OLAP
16DUPLICATE KEY(`doc_id`)
17DISTRIBUTED BY HASH(`doc_id`) BUCKETS 1;
DQL
下面是两张示例表的检索结果,可作为后续用例的数据参照。
1-- duplicate 表里的数据
2SELECT * FROM ann_table WHERE ann_distance(embedding, [1.1, 2.1, 3.1, 4.1]) BETWEEN 0 AND 1 ORDER BY __DISTANCE LIMIT 100;
3+------+-----------+------------+----------------------+------+--------------+
4| F_id | F_meta_id | F_content | embedding | len | __DISTANCE |
5+------+-----------+------------+----------------------+------+--------------+
6| 1 | meta1 | content1 | [1.1, 2.1, 3.1, 4.1] | 2 | 0 |
7| 2 | meta1 | content2 | [1.1, 2.1, 3.1, 4.1] | 2 | 0 |
8| 3 | meta2 | content1 | [1.1, 2.1, 3.1, 4.2] | 3 | 0.0099999812 |
9| 1 | meta1 | content2 | [1.1, 2.2, 3.1, 4.1] | 6 | 0.010000029 |
10| 5 | meta5 | content111 | [1.1, 2.1, 3.2, 4.2] | 6 | 0.020000011 |
11| 4 | meta3 | content3 | [2.1, 2.1, 3.1, 4.1] | 5 | 0.99999976 |
12+------+-----------+------------+----------------------+------+--------------+
1-- unique 表里的数据,F_id:6 这条是被 update 过的数据
2SELECT * FROM ann_table_unique WHERE ann_distance(embedding, [1.1, 2.1, 3.1, 4.1]) BETWEEN 0 AND 1 ORDER BY __DISTANCE;
3+------+-----------+-----------+----------------------+------+--------------+
4| F_id | F_meta_id | F_content | embedding | len | __DISTANCE |
5+------+-----------+-----------+----------------------+------+--------------+
6| 5 | meta1 | content2 | [1.1, 2.1, 3.1, 4.1] | 2 | 0 |
7| 1 | meta1 | content1 | [1.1, 2.1, 3.1, 4.1] | 2 | 0 |
8| 2 | meta1 | content2 | [1.1, 2.1, 3.1, 4.1] | 2 | 0 |
9| 4 | meta1 | content1 | [1.1, 2.1, 3.1, 4.1] | 2 | 0 |
10| 3 | meta2 | content1 | [1.1, 2.1, 3.1, 4.2] | 3 | 0.0099999812 |
11| 6 | meta2 | content1 | [1.1, 2.1, 3.1, 4.2] | 4 | 0.0099999812 |
12+------+-----------+-----------+----------------------+------+--------------+
向量检索 范围查询必须要传入期望查询返回的结果数(即limit)。如果没传,当前会设置默认值:100 范围查询没有 等于。因为是近似查询。
纯向量检索
TopK 查询
标准 TopK 查询:按 ann_distance() 排序并用 LIMIT 取前 K 条。支持。
1SELECT * FROM ann_table ORDER BY ann_distance(embedding, [1.1, 2.1, 3.1, 4.1]) LIMIT 3;

不设置 LIMIT 的 TopK 查询。支持。
1SELECT * FROM ann_table ORDER BY ann_distance(embedding, [1.1, 2.1, 3.1, 4.1]);

把 ann_distance() 放在 SELECT 字段里、再按别名排序。不支持。
1SELECT *, ann_distance(embedding, [1.1, 2.1, 3.1, 4.1]) AS distance FROM ann_table ORDER BY distance ASC LIMIT 5;

范围查询
BETWEEN 范围查询 + LIMIT,不加 ORDER BY 时结果次序不固定。支持。
1SELECT * FROM ann_table WHERE ann_distance(embedding, [1.1, 2.1, 3.1, 4.1]) BETWEEN 0.01 AND 0.03 LIMIT 5;
2SELECT *, __DISTANCE FROM ann_table WHERE ann_distance(embedding, [1.1, 2.1, 3.1, 4.1]) BETWEEN 0.01 AND 0.03 LIMIT 5;

BETWEEN 范围查询不设置 LIMIT,使用默认值 100。支持。
1SELECT * FROM ann_table WHERE ann_distance(embedding, [1.1, 2.1, 3.1, 4.1]) BETWEEN 0.01 AND 0.03;

BETWEEN 范围查询 + 按距离全局排序,支持升序、降序和叠加 LIMIT。支持。
1SELECT * FROM ann_table WHERE ann_distance(embedding, [1.1, 2.1, 3.1, 4.1]) BETWEEN 0.01 AND 0.03 ORDER BY __DISTANCE;
2SELECT * FROM ann_table WHERE ann_distance(embedding, [1.1, 2.1, 3.1, 4.1]) BETWEEN 0.01 AND 0.03 ORDER BY __DISTANCE LIMIT 5;
3SELECT * FROM ann_table WHERE ann_distance(embedding, [1.1, 2.1, 3.1, 4.1]) BETWEEN 0.01 AND 0.03 ORDER BY __DISTANCE DESC LIMIT 5;

用大于、小于组合出范围 + 按距离全局排序。支持。
1SELECT * FROM ann_table
2WHERE ann_distance(embedding, [1.1, 2.1, 3.1, 4.1]) > 0.01
3 AND ann_distance(embedding, [1.1, 2.1, 3.1, 4.1]) < 0.03
4ORDER BY __DISTANCE DESC LIMIT 2;

单个距离边界查询 + 按距离全局排序,内部会自动补全另一边界的阈值。支持。
1SELECT * FROM ann_table WHERE ann_distance(embedding, [1.1, 2.1, 3.1, 4.1]) < 0.5 ORDER BY __DISTANCE LIMIT 10;
2SELECT * FROM ann_table WHERE ann_distance(embedding, [1.1, 2.1, 3.1, 4.1]) > 0.02 ORDER BY __DISTANCE LIMIT 10;

查询指定列。支持。
1SELECT F_id, embedding FROM ann_table WHERE ann_distance(embedding, [1.1, 2.1, 3.1, 4.1]) BETWEEN 0.01 AND 0.03 LIMIT 5;

查询指定列并显式取出隐藏的距离列 __DISTANCE。支持。
1SELECT F_id, __DISTANCE FROM ann_table WHERE ann_distance(embedding, [1.1, 2.1, 3.1, 4.1]) BETWEEN 0.01 AND 0.03 ORDER BY __DISTANCE LIMIT 5;

子查询
派生表子查询 + 隐藏距离列。支持。
1SELECT sc.*, __DISTANCE
2FROM (SELECT embedding, len FROM ann_table_dup WHERE len > 2) AS sc
3ORDER BY ann_distance(sc.embedding, [1.1, 2.1, 3.1, 4.1])
4LIMIT 5;

WITH 子查询 + 隐藏距离列。支持。
1WITH subquery_result AS (
2 SELECT embedding, len
3 FROM ann_table_dup
4 WHERE len > 2
5)
6SELECT *, __DISTANCE
7FROM subquery_result
8ORDER BY ann_distance(embedding, [1.1, 2.1, 3.1, 4.1]) DESC
9LIMIT 5;

分页查询
TopK 查询 + 分页。支持。
1SELECT *, __DISTANCE FROM ann_table ORDER BY ann_distance(embedding, [1.1, 2.1, 3.1, 4.1]) LIMIT 1, 5;

查询指定列 + 隐藏距离列 + 距离排序 + 分页。支持。
1SELECT F_id, __DISTANCE FROM ann_table WHERE ann_distance(embedding, [1.1, 2.1, 3.1, 4.1]) BETWEEN 0.01 AND 0.03 ORDER BY __DISTANCE LIMIT 1, 5;

复杂检索
标量过滤 + 向量检索
标量过滤 + 向量范围查询 + ORDER BY,结果顺序固定。支持。
1SELECT * FROM ann_table WHERE ann_distance(embedding, [1.1, 2.1, 3.1, 4.1]) BETWEEN 0 AND 1 AND len >= 3 ORDER BY __DISTANCE DESC;

标量过滤 + 向量 TopK 查询。支持。
1SELECT *, __DISTANCE FROM ann_table WHERE len > 2 ORDER BY ann_distance(embedding, [1.1, 2.1, 3.1, 4.1]) LIMIT 10;

标量过滤 + 向量范围查询,不加 ORDER BY,结果顺序不固定。支持。
1SELECT * FROM ann_table WHERE ann_distance(embedding, [1.1, 2.1, 3.1, 4.1]) BETWEEN 0 AND 1 AND len >= 3;

分组检索
向量范围检索 + 标量分组 + 距离聚合。支持。
1SELECT F_id, max(__DISTANCE) FROM ann_table_dup WHERE ann_distance(embedding, [1.1, 2.1, 3.1, 4.1]) BETWEEN 0 AND 1 GROUP BY F_id LIMIT 10;

向量范围检索 + 标量分组 + 距离聚合 + 按聚合后的距离排序。支持。
1SELECT F_id, max(__DISTANCE) AS id_max_distance
2FROM ann_table_dup
3WHERE ann_distance(embedding, [1.1, 2.1, 3.1, 4.1]) BETWEEN 0 AND 1
4GROUP BY F_id
5ORDER BY id_max_distance DESC
6LIMIT 10;

全文检索 + 向量检索
全文检索与向量检索写在同一个 WHERE 中。不支持。
1SELECT * FROM ann_table WHERE ann_distance(embedding, [1.1, 2.1, 3.1, 4.1]) BETWEEN 0 AND 1 AND content MATCH_ANY 'content1';
批量向量检索
批量向量检索,即一次传入多个查询向量。不支持
1SELECT * FROM ann_table WHERE ann_distance(embedding, [[1.1, 2.1, 3.1, 4.1], [1.2, 2.1, 3.1, 4.2], [2.1, 1.1, 3.1, 4.1]]) BETWEEN 0 AND 1 AND len >= 3;
多列向量检索
用 multi_vector_search() 融合同一张表多个向量列的检索结果,融合排名结果在 __RANK 列返回。
RRF 排序
RRF排序的C参数可以不传,默认值为:60。支持。
- c越小 → 高位排名加分越大(尤其是 c < 1 时,高位结果非常占优势);
- c越大 → 多个排名的权重差异变小。
1SELECT id, name, __RANK FROM multi_ann_table
2ORDER BY multi_vector_search(
3 '{"rerank":"RRF", "c":30}',
4 ann_distance(embedding1, [1.0, 2.0, 3.0, 4.0]),
5 ann_distance(embedding2, [1.1, 2.1, 3.1, 4.1])
6)
7LIMIT 5;

WEIGHTED 排序
weight排序多列的结果。weights数组第一个值,代表后面第一列的检索结果的权重值,第二个值,代表后面第二列的检索结果的权重值。支持。
weights默认值为每列结果的权重都为1。
1SELECT id, name, __RANK FROM multi_ann_table
2ORDER BY multi_vector_search(
3 '{"rerank":"WEIGHT", "weights":[0.6,0.4]}',
4 ann_distance(embedding1, [1.0, 2.0, 3.0, 4.0]),
5 ann_distance(embedding2, [1.1, 2.1, 3.1, 4.1])
6)
7LIMIT 5;

混合检索(多列向量 + 多列全文 + 标量过滤)
语法与多列向量检索相同,把 bm25_score() 与 ann_distance() 一起传给 hybrid_search(),标量过滤仍写在 WHERE 中。全文检索支持的 Match 类型:all、any、phrase。
RRF 排序
语法同多列向量检索。全文检索支持的Match类型:all、any、phrase。支持。
1SELECT id, name, __RANK FROM hybrid_dup_table2 WHERE age > 26 AND id > 1
2ORDER BY hybrid_search(
3 '{"rerank":"RRF"}',
4 ann_distance(embedding1, '[1.0, 2.0, 3.0, 4.0]'),
5 bm25_score(content3, 'any', 'content3'),
6 ann_distance(embedding2, '[1.1, 2.1, 3.3, 4.4]'),
7 bm25_score(content1, 'any', 'content1'),
8 bm25_score(content2, 'any', 'content2')
9) LIMIT 8;

WEIGHTED 排序
语法同多列向量检索。全文检索支持的Match类型:all、any、phrase。支持。
1SELECT id, name, __RANK FROM hybrid_dup_table2 WHERE age > 26 AND id > 1
2ORDER BY hybrid_search(
3 '{"rerank":"WEIGHT", weights:[0.5, 0.2, 0.5, 0.2, 0.2]}',
4 ann_distance(embedding1, '[1.0, 2.0, 3.0, 4.0]'),
5 bm25_score(content3, 'any', 'content3'),
6 ann_distance(embedding2, '[1.1, 2.1, 3.3, 4.4]'),
7 bm25_score(content1, 'any', 'content1'),
8 bm25_score(content2, 'any', 'content2')
9) LIMIT 8;

多表混合检索
用 HYBRID ... ON 连接两张表,各自召回后再融合。注意:ON 中的行键类型必须一致,例如都为 INT。
RRF 排序
docs 表全文检索 top10 与 emb 表向量检索 top10 按 RRF 融合后取 top5。支持。
要求on中的行键类型必须一致(例如都为INT)。
1SELECT __RANK,
2 COALESCE(d.id, e.doc_id) AS rid,
3 d.title,
4 d.content,
5 e.tag
6FROM docs_mt d HYBRID emb_mt e ON d.id = e.doc_id
7WHERE d.title LIKE '%apple%' AND e.tag = 'a'
8ORDER BY hybrid_search(
9 '{"rerank":"RRF","c":60,"candidates":10}',
10 bm25_score(d.content, 'any', 'apple market'),
11 ann_distance(e.embedding, '[1.0,2.0,3.0,4.0]')
12)
13LIMIT 5;

WEIGHTED 排序
两路召回按权重融合后取 top5。支持。
1SELECT __RANK,
2 COALESCE(a.doc_id, b.doc_id) AS rid,
3 a.tag AS tag_a,
4 b.tag AS tag_b
5FROM emb_mt a HYBRID emb_mt b ON a.doc_id = b.doc_id
6WHERE a.tag IN ('a','b') AND b.doc_id <= 7
7ORDER BY hybrid_search(
8 '{"rerank":"WEIGHT","weights":[0.7,0.3]}',
9 ann_distance(a.embedding, '[1.0,2.0,3.0,4.0]'),
10 ann_distance(b.embedding, '[1.4,2.0,3.0,4.0]')
11)
12LIMIT 5;

模型重排序(仅预览版支持)
在 hybrid_search() 中通过 reranker 指定重排序模型、query 和参与重排的文本列。
子查询中重排序
docs_mt 全文检索 top5 与 emb_mt 向量检索 top5 UNION 后,将 title、content 两列送入模型重排序。支持。
1SELECT id, title, content
2FROM (
3 -- 第一路:docs_mt 上全文召回
4 SELECT id, title, content
5 FROM (
6 SELECT id, title, content
7 FROM docs_mt
8 WHERE content MATCH_ANY 'apple market'
9 ORDER BY __SCORE DESC
10 LIMIT 5
11 ) text_recall
12 UNION ALL
13 -- 第二路:emb_mt 上向量召回
14 SELECT id, title, content
15 FROM (
16 SELECT e.doc_id AS id,
17 e.tag AS title,
18 e.tag AS content
19 FROM emb_mt e
20 WHERE e.tag IN ('a', 'b')
21 ORDER BY ann_distance(e.embedding, '[1.0,2.0,3.0,4.0]')
22 LIMIT 5
23 ) vector_recall
24) union_recall
25ORDER BY hybrid_search(
26 '{"candidates":10,
27 "reranker":{
28 "model":"jina-reranker-v2-base-multilingual",
29 "query":"apple market cap",
30 "text_columns":["title","content"]
31 }}'
32)
33LIMIT 5;

多表 RRF 后接模型重排序
多表 RRF 融合结果再送入模型重排序。支持。
1SELECT COALESCE(d.id, e.doc_id) AS id,
2 d.title,
3 d.content,
4 e.tag
5FROM docs_mt d HYBRID emb_mt e ON d.id = e.doc_id
6ORDER BY hybrid_search(
7 '{"rerank":"RRF","c":60,"candidates":50,
8 "reranker":{
9 "model":"jina-reranker-v2-base-multilingual",
10 "query":"apple market cap",
11 "text_columns":["d.content"]
12 }}',
13 bm25_score(d.content, 'any', 'apple market'),
14 ann_distance(e.embedding, '[1.0,2.0,3.0,4.0]')
15)
16LIMIT 5;

重排序后处理(仅预览版支持)
在 reranker 之后通过 post_process 追加后处理阶段。下例为两路检索结果依次应用 RRF 融合、模型重排序和各后处理阶段。支持。
1SELECT id, title, content, publish_time
2FROM documents
3ORDER BY hybrid_search(
4 '{
5 "rerank": "RRF",
6 "c": 60,
7 "candidates": 50,
8 "reranker": {
9 "model": "jina-reranker-v2-base-multilingual",
10 "query": "苹果公司市值",
11 "text_columns": ["title", "content"]
12 },
13 "post_process": {
14 "min_score": 0.35,
15 "length_norm": {
16 "field": "content",
17 "min_length": 50
18 },
19 "blend": {
20 "fusion_weight": 0.3,
21 "reranker_weight": 0.7
22 },
23 "decay": {
24 "function": "gauss",
25 "field": "publish_time",
26 "origin": "2026-04-20",
27 "scale": "7d",
28 "offset": "1d",
29 "decay": 0.5
30 },
31 "mmr": {
32 "lambda": 0.7,
33 "vector_column": "vec",
34 "distance_type": "cosine",
35 "top_n": 10
36 },
37 "topn_post": 10
38 }
39 }',
40 ann_distance(vec, [0.1, 0.2, 0.3, 0.4]),
41 bm25_score(content, 'any', '苹果公司市值')
42)
43LIMIT 10;

评价此篇文章
