Gabriel39 commented on code in PR #68028:
URL: https://github.com/apache/doris/pull/68028#discussion_r4025263342
##########
fe/fe-core/src/main/java/org/apache/doris/tablefunction/VectorSearchTableValuedFunction.java:
##########
@@ -77,12 +77,16 @@ private static PreparedSearch prepare(Map<String, String>
properties)
.setQueryVector(queryVector)
.setTopK(common.topK())
.setOffset(common.offset());
- if (params.containsKey(METRIC)) {
- vectorParams.setMetric(parseMetric(params.get(METRIC)));
- }
+ // Pin the planner's default on every split; Lance otherwise inherits
an index metric.
+ vectorParams.setMetric(params.containsKey(METRIC) ?
parseMetric(params.get(METRIC)) : TVectorMetric.L2);
+ LanceVectorQuery.validateMultiVectorBudget(queryVector, common.topK(),
common.offset(),
+ params.containsKey(REFINE_FACTOR) ?
parsePositiveInt(params.get(REFINE_FACTOR), REFINE_FACTOR) : 1);
Review Comment:
Yes. An explicitly supplied refine_factor must be positive: the pinned Lance
indexed-search planner rejects Some(0) with 'Refine factor cannot be zero'.
Doris already enforced this through parsePositiveInt before this PR. A factor
of 1 still re-ranks candidates using original vectors; it does not disable
refinement. Multi-vector queries default to 1 to refine scores and validate the
stored values.
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/lance/LanceVectorQuery.java:
##########
@@ -38,6 +38,10 @@
/** Validates and encodes one Lance vector-search query against its Arrow
vector column. */
public final class LanceVectorQuery {
+ // Keep aligned with the BE and Lance-C limits: each subvector expands
into an ANN branch.
+ public static final int MAX_QUERY_VECTORS = 128;
+ public static final long MAX_QUERY_VECTOR_CANDIDATES = 100_000;
Review Comment:
The limit is a resource guard, not an algorithmic requirement. Lance creates
an ANN plan branch for each query subvector and overfetches candidates in each
branch, so a small input matrix can still expand into substantial planning and
execution work. FE validation rejects oversized requests before dispatch; BE
and the merged lance-c #83 API also enforce the same bounds. Removing only the
Doris check would therefore defer the rejection to the backend. I agree that
128 subvectors and the 100,000 candidate budget are policy choices and that
using the same budget for exact and indexed searches is conservative. We can
revisit the thresholds and separate the execution paths with workload evidence,
but I would retain the aligned checks in this PR rather than remove the
protection.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
