Copilot commented on code in PR #20687:
URL: https://github.com/apache/druid/pull/20687#discussion_r4350894385
##########
server/src/main/java/org/apache/druid/query/lookup/LookupListeningResource.java:
##########
@@ -139,4 +161,17 @@ public Object delete(String id)
}
);
}
+
+ @Override
+ @GET
+ @Produces({MediaType.APPLICATION_JSON,
SmileMediaTypes.APPLICATION_JACKSON_SMILE})
+ public Response getAll()
+ {
+ final Response response = super.getAll();
+ if (response.getStatus() == Response.Status.OK.getStatusCode()) {
+ final LookupsState<?> state = (LookupsState<?>) response.getEntity();
+ return Response.ok(LookupResourceFilter.filterBulk(request,
state)).build();
Review Comment:
This rebuilds the response from scratch and can drop important metadata from
`super.getAll()` (e.g., headers like caching controls, content-type already
selected, etc.). Prefer creating the new response via
`Response.fromResponse(response).entity(filteredEntity)...` to preserve
existing headers/status while swapping the entity.
##########
server/src/main/java/org/apache/druid/server/http/security/LookupResourceFilter.java:
##########
@@ -0,0 +1,288 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.druid.server.http.security;
+
+import com.google.common.collect.ImmutableList;
+import com.google.common.collect.ImmutableSet;
+import com.google.common.collect.Iterables;
+import com.google.inject.Inject;
+import com.google.inject.Injector;
+import com.sun.jersey.spi.container.ContainerRequest;
+import org.apache.druid.discovery.LookupNodeService;
+import org.apache.druid.query.lookup.LookupsState;
+import org.apache.druid.server.security.Action;
+import org.apache.druid.server.security.AuthConfig;
+import org.apache.druid.server.security.AuthorizationResult;
+import org.apache.druid.server.security.AuthorizationUtils;
+import org.apache.druid.server.security.AuthorizerMapper;
+import org.apache.druid.server.security.ForbiddenException;
+import org.apache.druid.server.security.Resource;
+import org.apache.druid.server.security.ResourceAction;
+import org.apache.druid.server.security.ResourceType;
+
+import javax.servlet.http.HttpServletRequest;
+import javax.ws.rs.core.Context;
+import javax.ws.rs.core.MultivaluedMap;
+import javax.ws.rs.core.UriInfo;
+import java.util.Collection;
+import java.util.LinkedHashMap;
+import java.util.LinkedHashSet;
+import java.util.Map;
+import java.util.Set;
+import java.util.function.Supplier;
+
+/**
+ * Use this resource filter for API endpoints that contain {@code lookups} in
their request path.
+ *
+ * The resource name is {@code tier/lookup}, or {@code tier} when the tier has
no lookups.
+ * Coordinator-wide requests with no tier or lookup use {@code LOOKUP:LOOKUP}.
+ * When lookup authorization is disabled, all requests use {@code
CONFIG:CONFIG}.
+ *
+ * Here are some example paths where this filter is used -
+ * - druid/coordinator/v1/lookups/config/{tier}/{lookup}
+ * - druid/coordinator/v1/lookups/status/{tier}/{lookup}
+ * - druid/coordinator/v1/lookups/config/{tier}
+ * - druid/coordinator/v1/lookups/config/all
+ */
+public class LookupResourceFilter extends AbstractResourceFilter
+{
+ private static final String BULK_FILTER =
LookupResourceFilter.class.getName() + ".bulkFilter";
+
+ private final boolean enabled;
+ private final Injector injector;
+
+ @Context
+ UriInfo uriInfo;
+
+ @Inject
+ public LookupResourceFilter(AuthorizerMapper authorizerMapper, AuthConfig
authConfig, Injector injector)
+ {
+ super(authorizerMapper);
+ this.enabled = authConfig.isEnableLookupAuthorization();
+ this.injector = injector;
+ }
+
+ @Override
+ public ContainerRequest filter(ContainerRequest request)
+ {
+ if (!enabled) {
+ authorize(ImmutableList.of(new ResourceAction(new Resource("CONFIG",
ResourceType.CONFIG), getAction(request))));
+ return request;
+ }
+ final MultivaluedMap<String, String> pathParams =
uriInfo.getPathParameters();
+ final String lookup = lookupPathParam(pathParams);
+ if (lookup == null) {
+ // Bulk routes provide their lookup names after reading the request body
or response data.
+ getReq().setAttribute(BULK_FILTER, this);
+ } else {
+ authorize(pathParams.getFirst("tier"), lookup, getAction(request));
+ }
+ return request;
+ }
+
+ private static String lookupPathParam(MultivaluedMap<String, String>
pathParams)
+ {
+ for (final String param : ImmutableList.of("lookup", "lookupId", "id")) {
+ if (pathParams.containsKey(param)) {
+ return pathParams.getFirst(param);
+ }
+ }
+ return null;
+ }
+
+ public static <T> T filterBulk(HttpServletRequest request, T value)
+ {
+ return filterBulk(request, null, value);
+ }
+
+ /**
+ * Loads a tier's current lookups only when bulk authorization is active for
this request.
+ */
+ public static void filterBulk(HttpServletRequest request, String tier,
Supplier<?> value)
+ {
+ if (request != null && request.getAttribute(BULK_FILTER) != null) {
+ filterBulk(request, tier, value.get());
+ }
+ }
+
+ /**
+ * Checks every lookup in a bulk write or returns only entries allowed by
{@link Action#READ} for a GET.
+ * An empty value uses the tier name, or {@link ResourceType#LOOKUP} when no
tier is available.
+ */
+ public static <T> T filterBulk(HttpServletRequest request, String tier, T
value)
+ {
+ final LookupResourceFilter filter = request == null ? null :
(LookupResourceFilter) request.getAttribute(BULK_FILTER);
+ if (filter == null) {
+ return value;
+ }
+ final String method = request.getMethod();
+ final Action action = "GET".equals(method) || "HEAD".equals(method) ?
Action.READ : Action.WRITE;
+ final String targetTier = tier == null && value instanceof LookupsState
+ ?
filter.injector.getInstance(LookupNodeService.class).getLookupTier()
+ : tier;
+ final Set<String> names = new LinkedHashSet<>();
+ addNames(names, targetTier, value);
+ if (names.isEmpty()) {
+ names.add(name(targetTier, null));
+ }
+ if ("GET".equals(method)) {
+ return readable(filter.readableNames(request, names), targetTier, value);
+ }
+ filter.authorize(Iterables.transform(names, target -> lookupAction(target,
action)));
+ return value;
+ }
+
+ /**
+ * Filters a tier listing using known lookup names and tier grants where no
lookups are known.
+ */
+ public static Set<String> filterBulk(
+ HttpServletRequest request,
+ Set<String> tiers,
+ Map<String, ? extends Map<String, ?>> knownLookups
+ )
+ {
+ final LookupResourceFilter filter = request == null ? null :
(LookupResourceFilter) request.getAttribute(BULK_FILTER);
+ if (filter == null) {
+ return tiers;
+ }
+ final Set<String> names = new LinkedHashSet<>();
+ addNames(names, null, knownLookups);
+ tiers.forEach(tier -> {
+ if (knownLookups == null || !knownLookups.containsKey(tier)) {
+ names.add(tier);
+ }
+ });
+ final Set<String> readable = filter.readableNames(request, names);
+ tiers.removeIf(tier -> !canReadTier(readable, tier));
+ return tiers;
+ }
+
+ private Set<String> readableNames(HttpServletRequest request, Set<String>
names)
+ {
+ if (names.isEmpty()) {
+ names.add(name(null, null));
+ }
+ return ImmutableSet.copyOf(AuthorizationUtils.filterAuthorizedResources(
+ request,
+ names,
+ target -> ImmutableList.of(lookupAction(target, Action.READ)),
+ getAuthorizerMapper()
+ ));
+ }
+
+ private void authorize(Iterable<ResourceAction> actions)
+ {
+ final AuthorizationResult authResult =
AuthorizationUtils.authorizeAllResourceActions(
+ getReq(), actions, getAuthorizerMapper()
+ );
+ if (!authResult.allowAccessWithNoRestriction()) {
+ throw new ForbiddenException(authResult.getErrorMessage());
+ }
+ }
+
+ /**
+ * Uses the serving node's tier for lookup routes without a tier path
parameter.
+ */
+ private void authorize(String tier, String lookup, Action action)
+ {
+ final String targetTier = tier == null ?
injector.getInstance(LookupNodeService.class).getLookupTier() : tier;
+ authorize(ImmutableList.of(lookupAction(name(targetTier, lookup),
action)));
+ }
+
+ private static ResourceAction lookupAction(String name, Action action)
+ {
+ return new ResourceAction(new Resource(name, ResourceType.LOOKUP), action);
+ }
+
+ private static String name(String tier, String lookup)
+ {
+ return tier == null ? ResourceType.LOOKUP : lookup == null ? tier : tier +
"/" + lookup;
+ }
+
+ /**
+ * Collects resource names from nested tier maps, node maps, and {@link
LookupsState} values.
+ */
+ private static void addNames(Set<String> names, String tier, Object value)
+ {
+ if (value instanceof LookupsState) {
+ final LookupsState<?> state = (LookupsState<?>) value;
+ addNames(names, tier, state.getCurrent());
+ addNames(names, tier, state.getToLoad());
+ addNames(names, tier, state.getToDrop());
+ } else if (value instanceof Map) {
+ final Map<?, ?> map = (Map<?, ?>) value;
+ // String keys name tiers at the root and lookups within a tier. Other
keys identify nodes.
+ if (tier == null) {
+ map.forEach((key, entry) -> {
+ final int size = names.size();
+ addNames(names, (String) key, entry);
+ if (names.size() == size) {
+ names.add(name((String) key, null));
Review Comment:
When `tier == null`, `addNames` unconditionally casts `key` to `String`,
which will throw `ClassCastException` if the top-level map has non-String keys
(e.g., host maps). Consider guarding with `key instanceof String` before
casting, and for non-String keys, recurse into `entry` without changing `tier`
(or handle them as node maps).
##########
server/src/main/java/org/apache/druid/server/http/security/LookupResourceFilter.java:
##########
@@ -0,0 +1,288 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.druid.server.http.security;
+
+import com.google.common.collect.ImmutableList;
+import com.google.common.collect.ImmutableSet;
+import com.google.common.collect.Iterables;
+import com.google.inject.Inject;
+import com.google.inject.Injector;
+import com.sun.jersey.spi.container.ContainerRequest;
+import org.apache.druid.discovery.LookupNodeService;
+import org.apache.druid.query.lookup.LookupsState;
+import org.apache.druid.server.security.Action;
+import org.apache.druid.server.security.AuthConfig;
+import org.apache.druid.server.security.AuthorizationResult;
+import org.apache.druid.server.security.AuthorizationUtils;
+import org.apache.druid.server.security.AuthorizerMapper;
+import org.apache.druid.server.security.ForbiddenException;
+import org.apache.druid.server.security.Resource;
+import org.apache.druid.server.security.ResourceAction;
+import org.apache.druid.server.security.ResourceType;
+
+import javax.servlet.http.HttpServletRequest;
+import javax.ws.rs.core.Context;
+import javax.ws.rs.core.MultivaluedMap;
+import javax.ws.rs.core.UriInfo;
+import java.util.Collection;
+import java.util.LinkedHashMap;
+import java.util.LinkedHashSet;
+import java.util.Map;
+import java.util.Set;
+import java.util.function.Supplier;
+
+/**
+ * Use this resource filter for API endpoints that contain {@code lookups} in
their request path.
+ *
+ * The resource name is {@code tier/lookup}, or {@code tier} when the tier has
no lookups.
+ * Coordinator-wide requests with no tier or lookup use {@code LOOKUP:LOOKUP}.
+ * When lookup authorization is disabled, all requests use {@code
CONFIG:CONFIG}.
+ *
+ * Here are some example paths where this filter is used -
+ * - druid/coordinator/v1/lookups/config/{tier}/{lookup}
+ * - druid/coordinator/v1/lookups/status/{tier}/{lookup}
+ * - druid/coordinator/v1/lookups/config/{tier}
+ * - druid/coordinator/v1/lookups/config/all
+ */
+public class LookupResourceFilter extends AbstractResourceFilter
+{
+ private static final String BULK_FILTER =
LookupResourceFilter.class.getName() + ".bulkFilter";
+
+ private final boolean enabled;
+ private final Injector injector;
+
+ @Context
+ UriInfo uriInfo;
+
+ @Inject
+ public LookupResourceFilter(AuthorizerMapper authorizerMapper, AuthConfig
authConfig, Injector injector)
+ {
+ super(authorizerMapper);
+ this.enabled = authConfig.isEnableLookupAuthorization();
+ this.injector = injector;
+ }
+
+ @Override
+ public ContainerRequest filter(ContainerRequest request)
+ {
+ if (!enabled) {
+ authorize(ImmutableList.of(new ResourceAction(new Resource("CONFIG",
ResourceType.CONFIG), getAction(request))));
+ return request;
+ }
+ final MultivaluedMap<String, String> pathParams =
uriInfo.getPathParameters();
+ final String lookup = lookupPathParam(pathParams);
+ if (lookup == null) {
+ // Bulk routes provide their lookup names after reading the request body
or response data.
+ getReq().setAttribute(BULK_FILTER, this);
+ } else {
+ authorize(pathParams.getFirst("tier"), lookup, getAction(request));
+ }
+ return request;
+ }
+
+ private static String lookupPathParam(MultivaluedMap<String, String>
pathParams)
+ {
+ for (final String param : ImmutableList.of("lookup", "lookupId", "id")) {
+ if (pathParams.containsKey(param)) {
+ return pathParams.getFirst(param);
+ }
+ }
+ return null;
+ }
+
+ public static <T> T filterBulk(HttpServletRequest request, T value)
+ {
+ return filterBulk(request, null, value);
+ }
+
+ /**
+ * Loads a tier's current lookups only when bulk authorization is active for
this request.
+ */
+ public static void filterBulk(HttpServletRequest request, String tier,
Supplier<?> value)
+ {
+ if (request != null && request.getAttribute(BULK_FILTER) != null) {
+ filterBulk(request, tier, value.get());
+ }
+ }
+
+ /**
+ * Checks every lookup in a bulk write or returns only entries allowed by
{@link Action#READ} for a GET.
+ * An empty value uses the tier name, or {@link ResourceType#LOOKUP} when no
tier is available.
+ */
+ public static <T> T filterBulk(HttpServletRequest request, String tier, T
value)
+ {
+ final LookupResourceFilter filter = request == null ? null :
(LookupResourceFilter) request.getAttribute(BULK_FILTER);
+ if (filter == null) {
+ return value;
+ }
+ final String method = request.getMethod();
+ final Action action = "GET".equals(method) || "HEAD".equals(method) ?
Action.READ : Action.WRITE;
+ final String targetTier = tier == null && value instanceof LookupsState
+ ?
filter.injector.getInstance(LookupNodeService.class).getLookupTier()
+ : tier;
+ final Set<String> names = new LinkedHashSet<>();
+ addNames(names, targetTier, value);
+ if (names.isEmpty()) {
+ names.add(name(targetTier, null));
+ }
+ if ("GET".equals(method)) {
+ return readable(filter.readableNames(request, names), targetTier, value);
+ }
+ filter.authorize(Iterables.transform(names, target -> lookupAction(target,
action)));
+ return value;
+ }
+
+ /**
+ * Filters a tier listing using known lookup names and tier grants where no
lookups are known.
+ */
+ public static Set<String> filterBulk(
+ HttpServletRequest request,
+ Set<String> tiers,
+ Map<String, ? extends Map<String, ?>> knownLookups
+ )
+ {
+ final LookupResourceFilter filter = request == null ? null :
(LookupResourceFilter) request.getAttribute(BULK_FILTER);
+ if (filter == null) {
+ return tiers;
+ }
+ final Set<String> names = new LinkedHashSet<>();
+ addNames(names, null, knownLookups);
+ tiers.forEach(tier -> {
+ if (knownLookups == null || !knownLookups.containsKey(tier)) {
+ names.add(tier);
+ }
+ });
+ final Set<String> readable = filter.readableNames(request, names);
+ tiers.removeIf(tier -> !canReadTier(readable, tier));
+ return tiers;
Review Comment:
`filterBulk(HttpServletRequest, Set, Map)` mutates the provided `tiers` set
in-place via `removeIf`. This is surprising for a method that also returns the
set, and it can break callers passing unmodifiable sets (or reusing the set
later). Prefer returning a new filtered `Set` (e.g., a `LinkedHashSet`) and
leaving the input unmodified, or explicitly document the mutation in the
Javadoc.
##########
server/src/test/java/org/apache/druid/server/http/LookupCoordinatorResourceTest.java:
##########
@@ -1125,6 +1126,8 @@ private HttpServletRequest createMockRequest(boolean
hasJsonPayload)
if (hasJsonPayload) {
EasyMock.expect(request.getContentType()).andReturn(MediaType.APPLICATION_JSON).once();
}
+ EasyMock.expect(request.getAttribute(LookupResourceFilter.class.getName()
+ ".bulkFilter"))
+ .andReturn(null).anyTimes();
Review Comment:
The attribute key string is duplicated here
(`LookupResourceFilter.class.getName() + \".bulkFilter\"`) but the source of
truth (`BULK_FILTER`) is private in `LookupResourceFilter`. To avoid test
brittleness if the attribute key changes, consider exposing a
package-private/public constant or a small accessor (e.g.,
`LookupResourceFilter.bulkFilterAttributeKey()`) that tests can reference.
##########
docs/operations/security-user-auth.md:
##########
@@ -85,6 +86,16 @@ Resource names for this type are datasource names.
Specifying a datasource permi
Any authenticated user can read the contents of any
[lookup](../querying/lookups.md). Avoid storing sensitive data in lookups.
:::
+### `LOOKUP`
+
+When `druid.auth.enableLookupAuthorization` is true, lookup endpoints
authorize against `LOOKUP` resources named `tier/lookup`. The tier comes from
the URL, bulk request body, coordinator lookup state or the serving node.
`READ` covers GET requests, including lookup configuration, status, node status
and introspection. `WRITE` covers POST, PUT and DELETE requests.
+
+This setting applies to lookup HTTP endpoints. It does not currently change
lookup use in SQL or native queries ie., query execution does not check
`LOOKUP` permissions.
Review Comment:
Correct grammar/punctuation: use `i.e.,` instead of `ie.,`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
