Copilot commented on code in PR #20555:
URL: https://github.com/apache/druid/pull/20555#discussion_r4240780246
##########
docs/querying/query-context-reference.md:
##########
@@ -43,36 +43,35 @@ Unless otherwise noted, the following parameters apply to
all query types, and t
|Parameter |Default | Description
|
|-------------------|----------------------------------------|----------------------|
-|`timeout` | `druid.server.http.defaultQueryTimeout`| Query timeout
in millis, beyond which unfinished queries will be cancelled. 0 timeout means
`no timeout` (up to the server-side maximum query timeout,
`druid.server.http.maxQueryTimeout`). To set the default timeout and maximum
timeout, see [Broker configuration](../configuration/index.md#broker) |
-|`perSegmentTimeout`| `null` | Per-segment
processing timeout in millis, beyond which unfinished queries will be
cancelled. Should be ≤ `timeout`. 0 `perSegmentTimeout` means `no per-segment
timeout`. Generally, a standard default should be O(X seconds). A cluster-wide
default value for this query context can be specified via
`druid.query.default.context.perSegmentTimeout`.|
-|`priority` | The default priority is one of the following:
<ul><li>Value of `priority` in the query context, if set</li><li>The value of
the runtime property `druid.query.default.context.priority`, if set and not
null</li><li>`0` if the priority is not set in the query context or runtime
properties</li></ul>| Query priority. Queries with higher priority get
precedence for computational resources.|
-|`lane` | `null` | Query lane,
used to control usage limits on classes of queries. See [Broker
configuration](../configuration/index.md#broker) for more details.|
-|`queryId` | auto-generated | Unique
identifier given to this query. If a query ID is set or known, this can be used
to cancel the query |
-|`brokerService` | `null` | Broker service
to which this query should be routed. This parameter is honored only by a
broker selector strategy of type *manual*. See [Router
strategies](../design/router.md#router-strategies) for more details.|
-|`useCache` | `true` | Flag indicating
whether to leverage the query cache for this query. When set to false, it
disables reading from the query cache for this query. When set to true, Apache
Druid uses `druid.broker.cache.useCache` or `druid.historical.cache.useCache`
to determine whether or not to read from the query cache |
-|`populateCache` | `true` | Flag indicating
whether to save the results of the query to the query cache. Primarily used for
debugging. When set to false, it disables saving the results of this query to
the query cache. When set to true, Druid uses
`druid.broker.cache.populateCache` or `druid.historical.cache.populateCache` to
determine whether or not to save the results of this query to the query cache |
+|`timeout`| `druid.server.http.defaultQueryTimeout` | Query timeout in millis,
beyond which unfinished queries will be cancelled. 0 timeout means `no timeout`
(up to the server-side maximum query timeout,
`druid.server.http.maxQueryTimeout`). To set the default timeout and maximum
timeout, see [Broker configuration](../configuration/index.md#broker).| <!--
GENERATED QUERY CONTEXT PARAMETER: timeout -->
+|`perSegmentTimeout`| `null` | Per-segment processing timeout in millis,
beyond which unfinished queries will be cancelled. Should be ≤ `timeout`. 0
`perSegmentTimeout` means `no per-segment timeout`. Generally, a standard
default should be O(X seconds). A cluster-wide default value for this query
context can be specified via `druid.query.default.context.perSegmentTimeout`.|
<!-- GENERATED QUERY CONTEXT PARAMETER: perSegmentTimeout -->
+|`priority`| The default priority is one of the following: <ul><li>Value of
`priority` in the query context, if set</li><li>The value of the runtime
property `druid.query.default.context.priority`, if set and not
null</li><li>`0` if the priority is not set in the query context or runtime
properties</li></ul> | Query priority. Queries with higher priority get
precedence for computational resources.| <!-- GENERATED QUERY CONTEXT
PARAMETER: priority -->
+|`lane`| `null` | Query lane, used to control usage limits on classes of
queries. See [Broker configuration](../configuration/index.md#broker) for more
details.| <!-- GENERATED QUERY CONTEXT PARAMETER: lane -->
+|`queryId`| auto-generated | Unique identifier given to this query. If a query
ID is set or known, this can be used to cancel the query.| <!-- GENERATED QUERY
CONTEXT PARAMETER: queryId -->
Review Comment:
The generated rows discard the existing aligned column spacing throughout
this Markdown table. Update the documentation generator to emit the established
aligned table format, then regenerate the rows so future generated updates
preserve the file's formatting.
##########
processing/src/main/java/org/apache/druid/query/groupby/GroupByQuery.java:
##########
@@ -1243,6 +1245,11 @@ public Builder overrideContext(Map<String, Object>
contextOverride)
return this;
}
+ public <V> Builder overrideContext(final QueryContextParameter<V>
parameter, @Nullable final V value)
+ {
+ return overrideContext(QueryContext.ofMap(parameter, value));
Review Comment:
`QueryContext.ofMap` drops null values, so `overrideContext(parameter,
null)` becomes an empty override and leaves any existing value in the builder
context intact. Preserve the explicit null entry here, matching
`Query.withOverriddenContext`, so callers can actually unset a previously
supplied parameter.
##########
docs/querying/query-context-reference.md:
##########
@@ -43,36 +43,35 @@ Unless otherwise noted, the following parameters apply to
all query types, and t
|Parameter |Default | Description
|
|-------------------|----------------------------------------|----------------------|
-|`timeout` | `druid.server.http.defaultQueryTimeout`| Query timeout
in millis, beyond which unfinished queries will be cancelled. 0 timeout means
`no timeout` (up to the server-side maximum query timeout,
`druid.server.http.maxQueryTimeout`). To set the default timeout and maximum
timeout, see [Broker configuration](../configuration/index.md#broker) |
-|`perSegmentTimeout`| `null` | Per-segment
processing timeout in millis, beyond which unfinished queries will be
cancelled. Should be ≤ `timeout`. 0 `perSegmentTimeout` means `no per-segment
timeout`. Generally, a standard default should be O(X seconds). A cluster-wide
default value for this query context can be specified via
`druid.query.default.context.perSegmentTimeout`.|
-|`priority` | The default priority is one of the following:
<ul><li>Value of `priority` in the query context, if set</li><li>The value of
the runtime property `druid.query.default.context.priority`, if set and not
null</li><li>`0` if the priority is not set in the query context or runtime
properties</li></ul>| Query priority. Queries with higher priority get
precedence for computational resources.|
-|`lane` | `null` | Query lane,
used to control usage limits on classes of queries. See [Broker
configuration](../configuration/index.md#broker) for more details.|
-|`queryId` | auto-generated | Unique
identifier given to this query. If a query ID is set or known, this can be used
to cancel the query |
-|`brokerService` | `null` | Broker service
to which this query should be routed. This parameter is honored only by a
broker selector strategy of type *manual*. See [Router
strategies](../design/router.md#router-strategies) for more details.|
-|`useCache` | `true` | Flag indicating
whether to leverage the query cache for this query. When set to false, it
disables reading from the query cache for this query. When set to true, Apache
Druid uses `druid.broker.cache.useCache` or `druid.historical.cache.useCache`
to determine whether or not to read from the query cache |
-|`populateCache` | `true` | Flag indicating
whether to save the results of the query to the query cache. Primarily used for
debugging. When set to false, it disables saving the results of this query to
the query cache. When set to true, Druid uses
`druid.broker.cache.populateCache` or `druid.historical.cache.populateCache` to
determine whether or not to save the results of this query to the query cache |
+|`timeout`| `druid.server.http.defaultQueryTimeout` | Query timeout in millis,
beyond which unfinished queries will be cancelled. 0 timeout means `no timeout`
(up to the server-side maximum query timeout,
`druid.server.http.maxQueryTimeout`). To set the default timeout and maximum
timeout, see [Broker configuration](../configuration/index.md#broker).| <!--
GENERATED QUERY CONTEXT PARAMETER: timeout -->
+|`perSegmentTimeout`| `null` | Per-segment processing timeout in millis,
beyond which unfinished queries will be cancelled. Should be ≤ `timeout`. 0
`perSegmentTimeout` means `no per-segment timeout`. Generally, a standard
default should be O(X seconds). A cluster-wide default value for this query
context can be specified via `druid.query.default.context.perSegmentTimeout`.|
<!-- GENERATED QUERY CONTEXT PARAMETER: perSegmentTimeout -->
+|`priority`| The default priority is one of the following: <ul><li>Value of
`priority` in the query context, if set</li><li>The value of the runtime
property `druid.query.default.context.priority`, if set and not
null</li><li>`0` if the priority is not set in the query context or runtime
properties</li></ul> | Query priority. Queries with higher priority get
precedence for computational resources.| <!-- GENERATED QUERY CONTEXT
PARAMETER: priority -->
+|`lane`| `null` | Query lane, used to control usage limits on classes of
queries. See [Broker configuration](../configuration/index.md#broker) for more
details.| <!-- GENERATED QUERY CONTEXT PARAMETER: lane -->
+|`queryId`| auto-generated | Unique identifier given to this query. If a query
ID is set or known, this can be used to cancel the query.| <!-- GENERATED QUERY
CONTEXT PARAMETER: queryId -->
+|`brokerService`| `null` | Broker service to which this query should be
routed. This parameter is honored only by a broker selector strategy of type
*manual*. See [Router strategies](../design/router.md#router-strategies) for
more details.| <!-- GENERATED QUERY CONTEXT PARAMETER: brokerService -->
+|`useCache`| `true` | Flag indicating whether to leverage the query cache for
this query. When set to false, it disables reading from the query cache for
this query. When set to true, Apache Druid uses `druid.broker.cache.useCache`
or `druid.historical.cache.useCache` to determine whether or not to read from
the query cache.| <!-- GENERATED QUERY CONTEXT PARAMETER: useCache -->
+|`populateCache`| `true` | Flag indicating whether to save the results of the
query to the query cache. Primarily used for debugging. When set to false, it
disables saving the results of this query to the query cache. When set to true,
Druid uses `druid.broker.cache.populateCache` or
`druid.historical.cache.populateCache` to determine whether or not to save the
results of this query to the query cache.| <!-- GENERATED QUERY CONTEXT
PARAMETER: populateCache -->
|`useResultLevelCache`| `true` | Flag indicating whether to leverage the
result level cache for this query. When set to false, it disables reading from
the query cache for this query. When set to true, Druid uses
`druid.broker.cache.useResultLevelCache` to determine whether or not to read
from the result-level query cache.| <!-- GENERATED QUERY CONTEXT PARAMETER:
useResultLevelCache -->
-|`populateResultLevelCache` | `true` | Flag indicating
whether to save the results of the query to the result level cache. Primarily
used for debugging. When set to false, it disables saving the results of this
query to the query cache. When set to true, Druid uses
`druid.broker.cache.populateResultLevelCache` to determine whether or not to
save the results of this query to the result-level query cache |
-|`bySegment` | `false` | Native queries
only. Return "by segment" results. Primarily used for debugging, setting it to
`true` returns results associated with the data segment they came from |
-|`finalize` | `N/A` | Flag indicating
whether to "finalize" aggregation results. Primarily used for debugging. For
instance, the `hyperUnique` aggregator returns the full HyperLogLog sketch
instead of the estimated cardinality when this flag is set to `false` |
-|`maxScatterGatherBytes`| `druid.server.http.maxScatterGatherBytes` | Maximum
number of bytes gathered from data processes such as Historicals and realtime
processes to execute a query. This parameter can be used to further reduce
`maxScatterGatherBytes` limit at query time. See [Broker
configuration](../configuration/index.md#broker) for more details.|
-|`maxQueuedBytes` | `druid.broker.http.maxQueuedBytes` | Maximum
number of bytes queued per query before exerting backpressure on the channel to
the data server. Similar to `maxScatterGatherBytes`, except unlike that
configuration, this one will trigger backpressure rather than query failure.
Zero means disabled.|
-|`maxSubqueryRows`| `druid.server.http.maxSubqueryRows` | Upper limit on the
number of rows a subquery can generate. See [Broker
configuration](../configuration/index.md#broker) and [subquery
guardrails](../configuration/index.md#Guardrails for materialization of
subqueries) for more details.|
-|`maxSubqueryBytes`| `druid.server.http.maxSubqueryBytes` | Upper limit on the
number of bytes a subquery can generate. See [Broker
configuration](../configuration/index.md#broker) and [subquery
guardrails](../configuration/index.md#Guardrails for materialization of
subqueries) for more details.|
-|`serializeDateTimeAsLong`| `false` | If true, DateTime is serialized as
long in the result returned by Broker and the data transportation between
Broker and compute process|
-|`serializeDateTimeAsLongInner`| `false` | If true, DateTime is serialized as
long in the data transportation between Broker and compute process|
-|`enableParallelMerge`|`true`|Enable parallel result merging on the Broker.
Note that `druid.processing.merge.useParallelMergePool` must be enabled for
this setting to be set to `true`. See [Broker
configuration](../configuration/index.md#broker) for more details.|
-|`parallelMergeParallelism`|`druid.processing.merge.parallelism`|Maximum
number of parallel threads to use for parallel result merging on the Broker.
See [Broker configuration](../configuration/index.md#broker) for more details.|
-|`parallelMergeInitialYieldRows`|`druid.processing.merge.initialYieldNumRows`|Number
of rows to yield per ForkJoinPool merge task for parallel result merging on
the Broker, before forking off a new task to continue merging sequences. See
[Broker configuration](../configuration/index.md#broker) for more details.|
-|`parallelMergeSmallBatchRows`|`druid.processing.merge.smallBatchNumRows`|Size
of result batches to operate on in ForkJoinPool merge tasks for parallel result
merging on the Broker. See [Broker
configuration](../configuration/index.md#broker) for more details.|
-|`useFilterCNF`|`false`| If true, Druid will attempt to convert the query
filter to Conjunctive Normal Form (CNF). During query processing, columns can
be pre-filtered by intersecting the bitmap indexes of all values that match the
eligible filters, often greatly reducing the raw number of rows which need to
be scanned. But this effect only happens for the top level filter, or
individual clauses of a top level 'and' filter. As such, filters in CNF
potentially have a higher chance to utilize a large amount of bitmap indexes on
string columns during pre-filtering. However, this setting should be used with
great caution, as it can sometimes have a negative effect on performance, and
in some cases, the act of computing CNF of a filter can be expensive. We
recommend hand tuning your filters to produce an optimal form if possible, or
at least verifying through experimentation that using this parameter actually
improves your query performance with no ill-effects.|
-|`secondaryPartitionPruning`|`true`|Enable secondary partition pruning on the
Broker. The Broker will always prune unnecessary segments from the input scan
based on a filter on time intervals, but if the data is further partitioned
with hash or range partitioning, this option will enable additional pruning
based on a filter on secondary partition dimensions.|
-|`debug`| `false` | Flag indicating whether to enable debugging outputs for
the query. When set to false, no additional logs will be produced (logs
produced will be entirely dependent on your logging level). When set to true,
the following addition logs will be produced:<br />- Log the stack trace of the
exception (if any) produced by the query |
-|`setProcessingThreadNames`|`false`| Flag indicating whether processing thread
names will be set to `processing_<queryId>` while processing a query. Thread
renaming aids in interpreting thread dumps, but has measurable thread renaming
overhead when segment scans are very quick. |
-|`sqlPlannerBloat`|`1000`|Calcite parameter which controls whether to merge
two Project operators when inlining expressions causes complexity to increase.
Implemented as a workaround to exception `There are not enough rules to produce
a node with desired properties: convention=DRUID, sort=[]` thrown after
rejecting the merge of two projects.|
-|`cloneQueryMode`|`excludeClones`| Indicates whether clone Historicals should
be queried by brokers. Clone servers are created by the `cloneServers`
Coordinator dynamic configuration. Possible values are `excludeClones`,
`includeClones` and `preferClones`. `excludeClones` means that clone
Historicals are not queried by the broker. `preferClones` indicates that when
given a choice between the clone Historical and the original Historical which
is being cloned, the broker chooses the clones. Historicals which are not
involved in the cloning process will still be queried. `includeClones` means
that broker queries any Historical without regarding clone status. This
parameter only affects native queries. MSQ does not query Historicals directly.|
-|`realtimeSegmentsMode` |`include`| Controls whether realtime segments are
queried. `include` queries all segments, including realtime. `exclude` skips
realtime segments. `exclusive` queries only realtime segments. |
-|`realtimeSegmentsOnly` |`false`| **Deprecated.** Use
`realtimeSegmentsMode=exclusive` instead. When set to `true`, this is
equivalent to `realtimeSegmentsMode=exclusive`. When set to `false`, this is
equivalent to `realtimeSegmentsMode=include`.|
+|`populateResultLevelCache`| `true` | Flag indicating whether to save the
results of the query to the result level cache. Primarily used for debugging.
When set to false, it disables saving the results of this query to the query
cache. When set to true, Druid uses
`druid.broker.cache.populateResultLevelCache` to determine whether or not to
save the results of this query to the result-level query cache.| <!-- GENERATED
QUERY CONTEXT PARAMETER: populateResultLevelCache -->
+|`bySegment`| `false` | Native queries only. Return "by segment" results.
Primarily used for debugging, setting it to `true` returns results associated
with the data segment they came from.| <!-- GENERATED QUERY CONTEXT PARAMETER:
bySegment -->
+|`finalize`| N/A | Flag indicating whether to "finalize" aggregation results.
Primarily used for debugging. For instance, the `hyperUnique` aggregator
returns the full HyperLogLog sketch instead of the estimated cardinality when
this flag is set to `false`.| <!-- GENERATED QUERY CONTEXT PARAMETER: finalize
-->
+|`maxScatterGatherBytes`| `druid.server.http.maxScatterGatherBytes` | Maximum
number of bytes gathered from data processes such as Historicals and realtime
processes to execute a query. This parameter can be used to further reduce
`maxScatterGatherBytes` limit at query time. See [Broker
configuration](../configuration/index.md#broker) for more details.| <!--
GENERATED QUERY CONTEXT PARAMETER: maxScatterGatherBytes -->
+|`maxQueuedBytes`| `druid.broker.http.maxQueuedBytes` | Maximum number of
bytes queued per query before exerting backpressure on the channel to the data
server. Similar to `maxScatterGatherBytes`, except unlike that configuration,
this one will trigger backpressure rather than query failure. Zero means
disabled.| <!-- GENERATED QUERY CONTEXT PARAMETER: maxQueuedBytes -->
+|`maxSubqueryRows`| `druid.server.http.maxSubqueryRows` | Upper limit on the
number of rows a subquery can generate. See [Broker
configuration](../configuration/index.md#broker) and [subquery
guardrails](../configuration/index.md#Guardrails for materialization of
subqueries) for more details.| <!-- GENERATED QUERY CONTEXT PARAMETER:
maxSubqueryRows -->
+|`maxSubqueryBytes`| `druid.server.http.maxSubqueryBytes` | Upper limit on the
number of bytes a subquery can generate. See [Broker
configuration](../configuration/index.md#broker) and [subquery
guardrails](../configuration/index.md#Guardrails for materialization of
subqueries) for more details.| <!-- GENERATED QUERY CONTEXT PARAMETER:
maxSubqueryBytes -->
+|`serializeDateTimeAsLong`| `false` | If true, DateTime is serialized as long
in the result returned by Broker and the data transportation between Broker and
compute process| <!-- GENERATED QUERY CONTEXT PARAMETER:
serializeDateTimeAsLong -->
+|`serializeDateTimeAsLongInner`| `false` | If true, DateTime is serialized as
long in the data transportation between Broker and compute process| <!--
GENERATED QUERY CONTEXT PARAMETER: serializeDateTimeAsLongInner -->
+|`enableParallelMerge`| `true` | Enable parallel result merging on the Broker.
Note that `druid.processing.merge.useParallelMergePool` must be enabled for
this setting to be set to `true`. See [Broker
configuration](../configuration/index.md#broker) for more details.| <!--
GENERATED QUERY CONTEXT PARAMETER: enableParallelMerge -->
+|`parallelMergeParallelism`| `druid.processing.merge.parallelism` | Maximum
number of parallel threads to use for parallel result merging on the Broker.
See [Broker configuration](../configuration/index.md#broker) for more details.|
<!-- GENERATED QUERY CONTEXT PARAMETER: parallelMergeParallelism -->
+|`parallelMergeInitialYieldRows`| `druid.processing.merge.initialYieldNumRows`
| Number of rows to yield per ForkJoinPool merge task for parallel result
merging on the Broker, before forking off a new task to continue merging
sequences. See [Broker configuration](../configuration/index.md#broker) for
more details.| <!-- GENERATED QUERY CONTEXT PARAMETER:
parallelMergeInitialYieldRows -->
+|`parallelMergeSmallBatchRows`| `druid.processing.merge.smallBatchNumRows` |
Size of result batches to operate on in ForkJoinPool merge tasks for parallel
result merging on the Broker. See [Broker
configuration](../configuration/index.md#broker) for more details.| <!--
GENERATED QUERY CONTEXT PARAMETER: parallelMergeSmallBatchRows -->
+|`useFilterCNF`| `false` | If true, Druid will attempt to convert the query
filter to Conjunctive Normal Form (CNF). During query processing, columns can
be pre-filtered by intersecting the bitmap indexes of all values that match the
eligible filters, often greatly reducing the raw number of rows which need to
be scanned. But this effect only happens for the top level filter, or
individual clauses of a top level 'and' filter. As such, filters in CNF
potentially have a higher chance to utilize a large amount of bitmap indexes on
string columns during pre-filtering. However, this setting should be used with
great caution, as it can sometimes have a negative effect on performance, and
in some cases, the act of computing CNF of a filter can be expensive. We
recommend hand tuning your filters to produce an optimal form if possible, or
at least verifying through experimentation that using this parameter actually
improves your query performance with no ill-effects.| <!-- GENERATED QUERY CO
NTEXT PARAMETER: useFilterCNF -->
+|`secondaryPartitionPruning`| `true` | Enable secondary partition pruning on
the Broker. The Broker will always prune unnecessary segments from the input
scan based on a filter on time intervals, but if the data is further
partitioned with hash or range partitioning, this option will enable additional
pruning based on a filter on secondary partition dimensions.| <!-- GENERATED
QUERY CONTEXT PARAMETER: secondaryPartitionPruning -->
+|`debug`| `false` | Flag indicating whether to enable debugging outputs for
the query. When set to false, no additional logs will be produced (logs
produced will be entirely dependent on your logging level). When set to true,
the following addition logs will be produced:<br />- Log the stack trace of the
exception (if any) produced by the query| <!-- GENERATED QUERY CONTEXT
PARAMETER: debug -->
+|`setProcessingThreadNames`| `false` | Flag indicating whether processing
thread names will be set to `processing_<queryId>` while processing a query.
Thread renaming aids in interpreting thread dumps, but has measurable thread
renaming overhead when segment scans are very quick.| <!-- GENERATED QUERY
CONTEXT PARAMETER: setProcessingThreadNames -->
+|`cloneQueryMode`| `excludeClones` | Indicates whether clone Historicals
should be queried by brokers. Clone servers are created by the `cloneServers`
Coordinator dynamic configuration. Possible values are `excludeClones`,
`includeClones` and `preferClones`. `excludeClones` means that clone
Historicals are not queried by the broker. `preferClones` indicates that when
given a choice between the clone Historical and the original Historical which
is being cloned, the broker chooses the clones. Historicals which are not
involved in the cloning process will still be queried. `includeClones` means
that broker queries any Historical without regarding clone status. This
parameter only affects native queries. MSQ does not query Historicals
directly.| <!-- GENERATED QUERY CONTEXT PARAMETER: cloneQueryMode -->
Review Comment:
The migration removes `sqlPlannerBloat` from the query-context reference
even though `CalciteRulesManager.BLOAT_PROPERTY` still defines and consumes
this supported context key. The generator preserves unmarked rows, so restore
its manually maintained row rather than silently dropping the parameter from
user documentation.
##########
processing/src/main/java/org/apache/druid/query/context/QueryContextParameters.java:
##########
@@ -53,6 +56,519 @@
/// ```
public final class QueryContextParameters
{
+ public static final QueryContextParameter<String> QUERY_ID =
stringParameter("queryId")
+ .description(
+ """
+ Unique identifier given to this query.
+ If a query ID is set or known, this can be used to cancel the query.
+ """
+ )
+ .since("0.6.62")
+ .defaultDescription("auto-generated")
+ .query(Query.JSON, Query.SQL)
+ .engine(Engine.NATIVE, Engine.MSQ, Engine.DART)
+ .build();
+
+ public static final QueryContextParameter<Boolean> FINALIZE =
booleanParameter("finalize")
+ .description(
+ """
+ Flag indicating whether to "finalize" aggregation results.
+ Primarily used for debugging.
+ For instance, the `hyperUnique` aggregator returns the full
HyperLogLog sketch instead of the estimated
+ cardinality when this flag is set to `false`.
+ """
+ )
+ .since("0.1.0")
+ .query(Query.JSON, Query.SQL)
+ .engine(Engine.NATIVE)
+ .build();
+
+ public static final QueryContextParameter<Integer> PRIORITY =
integerParameter("priority")
+ .defaultValue(QueryContexts.DEFAULT_PRIORITY)
+ .description(
+ """
+ Query priority.
+ Queries with higher priority get precedence for computational
resources.
+ """
+ )
+ .since("0.5.0")
+ .defaultDescription(
+ """
+ The default priority is one of the following: <ul><li>Value of
`priority` in the query context, if
+ set</li><li>The value of the runtime property
`druid.query.default.context.priority`, if set and
+ not null</li><li>`0` if the priority is not set in the query context
or runtime properties</li></ul>
+ """
+ )
+ .query(Query.JSON, Query.SQL)
+ .engine(Engine.NATIVE)
+ .build();
+
+ public static final QueryContextParameter<String> LANE =
stringParameter("lane")
+ .description(
+ """
+ Query lane, used to control usage limits on classes of queries.
+ See [Broker configuration](../configuration/index.md#broker) for
more details.
+ """
+ )
+ .since("0.18.0")
+ .defaultDescription("`null`")
+ .query(Query.JSON, Query.SQL)
+ .engine(Engine.NATIVE)
+ .build();
+
+ public static final QueryContextParameter<Long> TIMEOUT =
longParameter("timeout")
+ .description(
+ """
+ Query timeout in millis, beyond which unfinished queries will be
cancelled.
+ 0 timeout means `no timeout` (up to the server-side maximum query
timeout, `druid.server.http.maxQueryTimeout`).
+ To set the default timeout and maximum timeout, see [Broker
configuration](../configuration/index.md#broker).
+ """
+ )
+ .since("0.6.122")
+ .defaultDescription("`druid.server.http.defaultQueryTimeout`")
+ .query(Query.JSON, Query.SQL)
+ .engine(Engine.NATIVE)
+ .build();
+
+ public static final QueryContextParameter<Long> PER_SEGMENT_TIMEOUT =
+ longParameter("perSegmentTimeout")
+ .description(
+ """
+ Per-segment processing timeout in millis, beyond which
unfinished queries will be cancelled.
+ Should be ≤ `timeout`.
+ 0 `perSegmentTimeout` means `no per-segment timeout`.
+ Generally, a standard default should be O(X seconds).
+ A cluster-wide default value for this query context can be
specified via `druid.query.default.context.perSegmentTimeout`.
+ """
+ )
+ .since("35.0.0")
+ .defaultDescription("`null`")
+ .query(Query.JSON, Query.SQL)
+ .engine(Engine.NATIVE)
+ .build();
+
+ public static final QueryContextParameter<Long> MAX_SCATTER_GATHER_BYTES =
+ longParameter("maxScatterGatherBytes")
+ .description(
+ """
+ Maximum number of bytes gathered from data processes such as
Historicals and realtime processes to execute a query.
+ This parameter can be used to further reduce
`maxScatterGatherBytes` limit at query time.
+ See [Broker configuration](../configuration/index.md#broker) for
more details.
+ """
+ )
+ .since("0.10.1")
+ .defaultDescription("`druid.server.http.maxScatterGatherBytes`")
+ .query(Query.JSON, Query.SQL)
+ .engine(Engine.NATIVE)
+ .build();
+
+ public static final QueryContextParameter<Long> MAX_QUEUED_BYTES =
+ longParameter("maxQueuedBytes")
+ .description(
+ """
+ Maximum number of bytes queued per query before exerting
backpressure on the channel to the data server.
+ Similar to `maxScatterGatherBytes`, except unlike that
configuration, this one will trigger backpressure rather than query failure.
+ Zero means disabled.
+ """
+ )
+ .since("0.13.0-incubating")
+ .defaultDescription("`druid.broker.http.maxQueuedBytes`")
+ .query(Query.JSON, Query.SQL)
+ .engine(Engine.NATIVE)
+ .build();
+
+ public static final QueryContextParameter<Boolean> ENABLE_PARALLEL_MERGE =
+ booleanParameter("enableParallelMerge")
+ .defaultValue(QueryContexts.DEFAULT_ENABLE_PARALLEL_MERGE)
+ .description(
+ """
+ Enable parallel result merging on the Broker.
+ Note that `druid.processing.merge.useParallelMergePool` must be
enabled for this setting to be set to `true`.
+ See [Broker configuration](../configuration/index.md#broker) for
more details.
+ """
+ )
+ .since("0.17.0")
+ .query(Query.JSON, Query.SQL)
+ .engine(Engine.NATIVE)
+ .build();
+
+ public static final QueryContextParameter<Integer>
PARALLEL_MERGE_INITIAL_YIELD_ROWS =
+ integerParameter("parallelMergeInitialYieldRows")
+ .description(
+ """
+ Number of rows to yield per ForkJoinPool merge task for parallel
result merging on the Broker, before forking off a new task to continue merging
sequences.
+ See [Broker configuration](../configuration/index.md#broker) for
more details.
+ """
+ )
+ .since("0.17.0")
+ .defaultDescription("`druid.processing.merge.initialYieldNumRows`")
+ .query(Query.JSON, Query.SQL)
+ .engine(Engine.NATIVE)
+ .build();
+
+ public static final QueryContextParameter<Integer>
PARALLEL_MERGE_SMALL_BATCH_ROWS =
+ integerParameter("parallelMergeSmallBatchRows")
+ .description(
+ """
+ Size of result batches to operate on in ForkJoinPool merge tasks
for parallel result merging on the Broker.
+ See [Broker configuration](../configuration/index.md#broker) for
more details.
+ """
+ )
+ .since("0.17.0")
+ .defaultDescription("`druid.processing.merge.smallBatchNumRows`")
+ .query(Query.JSON, Query.SQL)
+ .engine(Engine.NATIVE)
+ .build();
+
+ public static final QueryContextParameter<Integer>
PARALLEL_MERGE_PARALLELISM =
+ integerParameter("parallelMergeParallelism")
+ .description(
+ """
+ Maximum number of parallel threads to use for parallel result
merging on the Broker.
+ See [Broker configuration](../configuration/index.md#broker) for
more details.
+ """
+ )
+ .since("0.17.0")
+ .defaultDescription("`druid.processing.merge.parallelism`")
+ .query(Query.JSON, Query.SQL)
+ .engine(Engine.NATIVE)
+ .build();
+
+ public static final QueryContextParameter<QueryContexts.Vectorize> VECTORIZE
=
+ enumParameter("vectorize", QueryContexts.Vectorize.class)
+ .defaultValue(QueryContexts.DEFAULT_VECTORIZE)
+ .description(
+ """
+ Enables or disables vectorized query execution.
+ Possible values are `false` (disabled), `true` (enabled if
possible, disabled otherwise, on a per-segment basis), and `force` (enabled,
and query types that support vectorization will fail if they cannot be
vectorized).
+ The `"force"` setting is meant to aid in testing, and is not
generally useful in production (since real-time segments can never be processed
with vectorized execution, any queries on real-time data will fail).
+ This will override `druid.query.default.context.vectorize` if
it's set.
+ """
+ )
+ .since("0.16.0-incubating")
+ .query(Query.JSON, Query.SQL)
+ .engine(Engine.NATIVE)
+ .build();
+
+ public static final QueryContextParameter<QueryContexts.Vectorize>
VECTORIZE_VIRTUAL_COLUMNS =
+ enumParameter("vectorizeVirtualColumns", QueryContexts.Vectorize.class)
+ .defaultValue(QueryContexts.DEFAULT_VECTORIZE_VIRTUAL_COLUMN)
+ .description(
+ """
+ Enables or disables vectorized query processing of queries with
virtual columns, layered on top of `vectorize` (`vectorize` must also be set to
true for a query to utilize vectorization).
+ Possible values are `false` (disabled), `true` (enabled if
possible, disabled otherwise, on a per-segment basis), and `force` (enabled,
and groupBy or timeseries queries with virtual columns that cannot be
vectorized will fail).
+ The `"force"` setting is meant to aid in testing, and is not
generally useful in production.
+ This will override
`druid.query.default.context.vectorizeVirtualColumns` if it's set.
+ """
+ )
+ .since("0.20.0")
+ .query(Query.JSON, Query.SQL)
+ .engine(Engine.NATIVE)
+ .build();
+
+ public static final QueryContextParameter<Integer> VECTOR_SIZE =
integerParameter("vectorSize")
+ .defaultValue(QueryContexts.DEFAULT_VECTOR_SIZE)
+ .description(
+ """
+ Sets the row batching size for a particular query.
+ This will override `druid.query.default.context.vectorSize` if it's
set.
+ """
+ )
+ .since("0.16.0-incubating")
+ .query(Query.JSON, Query.SQL)
+ .engine(Engine.NATIVE)
+ .build();
+
+ public static final QueryContextParameter<Integer> MAX_SUBQUERY_ROWS =
+ integerParameter("maxSubqueryRows")
+ .description(
+ """
+ Upper limit on the number of rows a subquery can generate.
+ See [Broker configuration](../configuration/index.md#broker) and
[subquery guardrails](../configuration/index.md#Guardrails for materialization
of subqueries) for more details.
+ """
+ )
+ .since("0.18.0")
+ .defaultDescription("`druid.server.http.maxSubqueryRows`")
+ .query(Query.JSON, Query.SQL)
+ .engine(Engine.NATIVE)
+ .build();
+
+ public static final QueryContextParameter<Object> MAX_SUBQUERY_BYTES =
+ objectParameter("maxSubqueryBytes")
+ .description(
+ """
+ Upper limit on the number of bytes a subquery can generate.
+ See [Broker configuration](../configuration/index.md#broker) and
[subquery guardrails](../configuration/index.md#Guardrails for materialization
of subqueries) for more details.
+ """
+ )
+ .since("27.0.0")
+ .defaultDescription("`druid.server.http.maxSubqueryBytes`")
+ .query(Query.JSON, Query.SQL)
+ .engine(Engine.NATIVE)
+ .build();
+
+ public static final QueryContextParameter<Boolean>
ENABLE_JOIN_FILTER_PUSH_DOWN =
+ booleanParameter("enableJoinFilterPushDown")
+ .defaultValue(QueryContexts.DEFAULT_ENABLE_JOIN_FILTER_PUSH_DOWN)
+ .description(
+ """
+ Controls whether a join query will attempt filter push down,
which reduces the number of rows that have to be compared in a join operation.
+ """
+ )
+ .since("0.18.0")
+ .query(Query.JSON, Query.SQL)
+ .engine(Engine.NATIVE)
+ .build();
+
+ public static final QueryContextParameter<Boolean>
ENABLE_JOIN_FILTER_REWRITE =
+ booleanParameter("enableJoinFilterRewrite")
+ .defaultValue(QueryContexts.DEFAULT_ENABLE_JOIN_FILTER_REWRITE)
+ .description(
+ """
+ Controls whether filter clauses that reference non-base table
columns will be rewritten into filters on base table columns.
+ """
+ )
+ .since("0.18.0")
+ .query(Query.JSON, Query.SQL)
+ .engine(Engine.NATIVE)
+ .build();
+
+ public static final QueryContextParameter<Boolean>
ENABLE_JOIN_FILTER_REWRITE_VALUE_COLUMN_FILTERS =
+ booleanParameter("enableJoinFilterRewriteValueColumnFilters")
+
.defaultValue(QueryContexts.DEFAULT_ENABLE_JOIN_FILTER_REWRITE_VALUE_COLUMN_FILTERS)
+ .description(
+ """
+ Controls whether Druid rewrites non-base table filters on
non-key columns in the non-base table.
+ Requires a scan of the non-base table.
+ """
+ )
+ .since("0.18.0")
+ .query(Query.JSON, Query.SQL)
+ .engine(Engine.NATIVE)
+ .build();
+
+ public static final QueryContextParameter<Boolean>
ENABLE_REWRITE_JOIN_TO_FILTER =
+ booleanParameter("enableRewriteJoinToFilter")
+ .defaultValue(QueryContexts.DEFAULT_ENABLE_REWRITE_JOIN_TO_FILTER)
+ .description(
+ """
+ Controls whether a join can be pushed partial or fully to the
base table as a filter at runtime.
+ """
+ )
+ .since("0.22.0")
+ .query(Query.JSON, Query.SQL)
+ .engine(Engine.NATIVE)
+ .build();
+
+ public static final QueryContextParameter<Long> JOIN_FILTER_REWRITE_MAX_SIZE
=
+ longParameter("joinFilterRewriteMaxSize")
+
.defaultValue(QueryContexts.DEFAULT_ENABLE_JOIN_FILTER_REWRITE_MAX_SIZE)
+ .description(
+ """
+ The maximum size of the correlated value set used for filter
rewrites.
+ Set this limit to prevent excessive memory use.
+ """
+ )
+ .since("0.18.0")
+ .query(Query.JSON, Query.SQL)
+ .engine(Engine.NATIVE)
+ .build();
+
+ public static final QueryContextParameter<CloneQueryMode> CLONE_QUERY_MODE =
+ enumParameter("cloneQueryMode", CloneQueryMode.class)
+ .defaultValue(QueryContexts.DEFAULT_CLONE_QUERY_MODE)
+ .description(
+ """
+ Indicates whether clone Historicals should be queried by brokers.
+ Clone servers are created by the `cloneServers` Coordinator
dynamic configuration.
+ Possible values are `excludeClones`, `includeClones` and
`preferClones`.
+ `excludeClones` means that clone Historicals are not queried by
the broker.
+ `preferClones` indicates that when given a choice between the
clone Historical and the original Historical which is being cloned, the broker
chooses the clones.
+ Historicals which are not involved in the cloning process will
still be queried.
+ `includeClones` means that broker queries any Historical without
regarding clone status.
+ This parameter only affects native queries.
+ MSQ does not query Historicals directly.
+ """
+ )
+ .since("34.0.0")
+ .query(Query.JSON)
+ .engine(Engine.NATIVE)
+ .build();
+
+ public static final QueryContextParameter<Boolean> USE_FILTER_CNF =
booleanParameter("useFilterCNF")
+ .defaultValue(QueryContexts.DEFAULT_USE_FILTER_CNF)
+ .description(
+ """
+ If true, Druid will attempt to convert the query filter to
Conjunctive Normal Form (CNF).
+ During query processing, columns can be pre-filtered by intersecting
the bitmap indexes of all values that match the eligible filters, often greatly
reducing the raw number of rows which need to be scanned.
+ But this effect only happens for the top level filter, or individual
clauses of a top level 'and' filter.
+ As such, filters in CNF potentially have a higher chance to utilize
a large amount of bitmap indexes on string columns during pre-filtering.
+ However, this setting should be used with great caution, as it can
sometimes have a negative effect on performance, and in some cases, the act of
computing CNF of a filter can be expensive.
+ We recommend hand tuning your filters to produce an optimal form if
possible, or at least verifying through experimentation that using this
parameter actually improves your query performance with no ill-effects.
+ """
+ )
+ .since("0.19.0")
+ .query(Query.JSON, Query.SQL)
+ .engine(Engine.NATIVE)
+ .build();
+
+ public static final QueryContextParameter<Boolean> USE_CACHE =
booleanParameter("useCache")
+ .defaultValue(QueryContexts.DEFAULT_USE_CACHE)
+ .description(
+ """
+ Flag indicating whether to leverage the query cache for this query.
+ When set to false, it disables reading from the query cache for this
query.
+ When set to true, Apache Druid uses `druid.broker.cache.useCache` or
`druid.historical.cache.useCache` to determine whether or not to read from the
query cache.
+ """
+ )
+ .since("0.1.0")
+ .query(Query.JSON, Query.SQL)
+ .engine(Engine.NATIVE)
+ .build();
+
+ public static final QueryContextParameter<Boolean>
SECONDARY_PARTITION_PRUNING =
+ booleanParameter("secondaryPartitionPruning")
+ .defaultValue(QueryContexts.DEFAULT_SECONDARY_PARTITION_PRUNING)
+ .description(
+ """
+ Enable secondary partition pruning on the Broker.
+ The Broker will always prune unnecessary segments from the input
scan based on a filter on time intervals, but if the data is further
partitioned with hash or range partitioning, this option will enable additional
pruning based on a filter on secondary partition dimensions.
+ """
+ )
+ .since("0.20.0")
+ .query(Query.JSON, Query.SQL)
+ .engine(Engine.NATIVE)
+ .build();
+
+ public static final QueryContextParameter<Boolean> DEBUG =
booleanParameter("debug")
+ .defaultValue(QueryContexts.DEFAULT_ENABLE_DEBUG)
+ .description(
+ """
+ Flag indicating whether to enable debugging outputs for the query.
+ When set to false, no additional logs will be produced (logs
produced will be entirely dependent on your logging level).
+ When set to true, the following addition logs will be produced:<br
/>- Log the stack trace of the exception (if any) produced by the query
+ """
+ )
+ .since("0.22.0")
+ .query(Query.JSON, Query.SQL)
+ .engine(Engine.NATIVE)
+ .build();
+
+ public static final QueryContextParameter<Boolean>
SET_PROCESSING_THREAD_NAMES =
+ booleanParameter("setProcessingThreadNames")
+ .defaultValue(false)
+ .description(
+ """
+ Flag indicating whether processing thread names will be set to
`processing_<queryId>` while processing a query.
+ Thread renaming aids in interpreting thread dumps, but has
measurable thread renaming overhead when segment scans are very quick.
+ """
+ )
+ .since("24.0.0")
+ .query(Query.JSON)
+ .engine(Engine.NATIVE)
+ .build();
+
+ public static final QueryContextParameter<Boolean> BY_SEGMENT =
booleanParameter("bySegment")
+ .defaultValue(QueryContexts.DEFAULT_BY_SEGMENT)
+ .description(
+ """
+ Native queries only.
+ Return "by segment" results.
+ Primarily used for debugging, setting it to `true` returns results
associated with the data segment they came from.
+ """
+ )
+ .since("0.1.0")
+ .query(Query.JSON)
+ .engine(Engine.NATIVE)
+ .build();
+
+ public static final QueryContextParameter<String> BROKER_SERVICE =
+ stringParameter("brokerService")
+ .description(
+ """
+ Broker service to which this query should be routed.
+ This parameter is honored only by a broker selector strategy of
type *manual*.
+ See [Router strategies](../design/router.md#router-strategies)
for more details.
+ """
+ )
+ .since("0.22.0")
+ .defaultDescription("`null`")
+ .query(Query.JSON)
Review Comment:
This metadata incorrectly limits `brokerService` to native JSON queries.
`ManualTieredBrokerSelectorStrategy.getBrokerServiceName(TieredBrokerConfig,
SqlQuery)` reads this parameter directly from the SQL query context, and the
migrated SQL routing tests exercise that path, so catalog consumers would
receive an incomplete applicability declaration.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
