This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new f5da8fb165e [fix](ranger) Apply Ranger policy items written against
groups, from Ranger's user store (#68220)
f5da8fb165e is described below
commit f5da8fb165e3abf339354e7d9799c0eb916a3762
Author: Mingyu Chen (Rayner) <[email protected]>
AuthorDate: Tue Sep 22 20:34:01 2026 +0800
[fix](ranger) Apply Ranger policy items written against groups, from
Ranger's user store (#68220)
### What problem does this PR solve?
Issue Number: None
Related PR: #68203 (documents the behaviour; this PR changes it)
Problem Summary:
A Ranger policy item can name users, groups and roles. The requests the
`ranger-doris` and `ranger-hive`
sources build carry no groups - Doris has none of its own to hand over -
so every policy item written
against a group never matches: an allow on a group grants nothing to a
Doris user, a row filter or a mask
on a group does not apply, and a **deny on a group is silently ignored**
while a user-level allow on the same
table still admits the user. Operators who manage access by Ranger group
(so that nobody edits a policy
every time somebody joins a team) see the user denied on everything,
down to `SHOW DATABASES` and
`SWITCH`, and it starts working the moment the same user is put on the
item directly.
Ranger's answer for a plugin that cannot compute groups itself is the
**user store** - the users and groups
Ranger Admin holds, kept current by usersync - which a plugin downloads
next to its policies. Ranger 2.5+
reads the requesting user's groups out of it when
`ranger.plugin.<type>.use.rangerGroups=true`; the
plugin library on branch-3.0/2.1 is 2.4.0, which has no such setting, so
on those branches no
configuration can make group items work.
What this PR does:
- `RangerUserStoreGroups` (ranger-common): the two Ranger plugins Doris
embeds put the user store
enricher on the service definition they download (`setPolicies`),
exactly as `RangerBasePlugin` does for
`use.rangerGroups`, and the request builders read the user's groups out
of the downloaded store and set
them on every request (`RangerAccessRequestImpl.setUserGroups`). This
does not depend on the Ranger
version the plugin was built against.
- On by default. `ranger.plugin.doris.use.rangerGroups=false`
(`ranger.plugin.hive.` for the catalog-level
source) switches it off and restores the previous requests; the property
is the one Ranger itself reads
for the same thing, so one setting decides both. The refresh interval
and retriever class are read under
Ranger's own option names (`userStoreRefresherPollingInterval`, default
60000 ms).
- Each plugin logs one line at start-up saying whether groups are
attached and how to switch it off.
- Regression: `ranger_p2/test_ranger_group_policy` writes an access
policy, a row filter, a mask and a
deny against a Ranger group only, and checks them as a member of that
group, then after leaving it.
`httpTest` gains `op "put"` for the Ranger user update that bumps the
user store version.
Verified end to end against the `docker/thirdparties` Ranger (Admin
2.4.0): group allow / row filter /
mask apply, a group deny outranks a user allow, a Ranger role granted to
a group resolves, leaving the group
revokes within one user store refresh, and `use.rangerGroups=false`
gives the previous behaviour.
What this PR also changes: a Ranger plugin is built with its service's
policies, or not at all
`RangerBasePlugin.init()` survives a Ranger Admin it cannot reach: it
answers out of the local policy cache
when there is one, and out of no policies at all when there is not,
refusing every check until a later poll
succeeds. That was already so before this PR, and it is the wrong state
for Doris to start in: an FE with
`access_controller_type=ranger-doris` and no policies is one nobody can
use (no account bypasses the source),
and a catalog bound to `ranger-hive` in that state refuses every
statement, with nothing but a line in fe.log
to say why. So the plugins now refuse that state at the moment they are
built, and Ranger's own resilience
takes over afterwards. The load order at start-up, and where it fails:
```text
FE start with access_controller_type=ranger-doris (every FE:
master, follower, observer)
Env()
└─ AccessControllerManager()
└─ RangerDorisAccessControllerFactory.create()
└─ new RangerDorisPlugin("doris") ... on the starting
thread, synchronously
└─ LoadedRangerPlugin.init()
├─ RangerBasePlugin.init()
│ ├─ audit subsystem
│ ├─ PolicyRefresher.startRefresher()
│ │ ├─ loadRoles() ──REST──► Ranger Admin
(fails: logged; roles from cache, or none)
│ │ ├─ loadPolicy() ──REST──► Ranger Admin
│ │ │ ├─ answered →
setPolicies(policies) policies version ≥ 0
│ │ │ ├─ unreachable, cache found →
setPolicies(cached policies) policies version ≥ 0
│ │ │ ├─ unreachable, no cache →
setPolicies(null): no engine policies version -1
│ │ │ └─ service not found → cache
cleared, no engine policies version -1
│ │ │ setPolicies() puts the user store
enricher on the service definition:
│ │ │ user store ──REST──► Ranger Admin
(fails: logged; no groups until it arrives)
│ │ └─ refresher thread + download timer start
(from here on: Ranger's own resilience)
│ └─ chained plugins (none configured)
└─ policies version < 0 ?
├─ no → plugin built. INFO "loaded in N ms:
policies version ...", WARN if no user store
└─ yes → cleanup(), throw IllegalStateException("...
has no policies to authorize against ...")
└─ Env() fails → the FE does not start
CREATE CATALOG ... 'access_controller.class'='ranger-hive'
CatalogFactory.createCatalog()
└─ catalog.initAccessController(dryRun = true)
└─ RangerHiveAccessControllerFactory.create()
└─ RangerHiveAuditStack.startFor(serviceName)
└─ new RangerHivePlugin(serviceName) → the same
LoadedRangerPlugin.init() as above
├─ built → controller released again (the stack stays up
300 s for the bind that follows) → CREATE succeeds
└─ threw → DdlException "Failed to init access
controller: ..." → CREATE fails
Later binds of a ranger-hive catalog (the first statement against it, after
ALTER CATALOG, after the 300 s idle stop):
the same init(); a failure fails that statement and the next statement
tries again. Never a fallback to the
built-in source.
```
Configuration errors (no `policy.rest.url`, a REST timeout that is not a
number, an audit destination Doris
does not ship) fail the way they always did: inside
`RangerBasePlugin.init()`, before anything is started. A
missing user store is not a failure: an Admin that serves none cannot be
told from one that has not answered
yet, so policy items written against a group simply do not apply until
it arrives, and the plugin says so
with a WARN.
---
.../authorizer/ranger/LoadedRangerPlugin.java | 159 ++++++++
.../authorizer/ranger/RangerAccessController.java | 9 +
.../authorizer/ranger/RangerUserStoreGroups.java | 212 +++++++++++
.../ranger/doris/RangerDorisAccessController.java | 7 +-
.../doris/RangerDorisAccessControllerFactory.java | 10 +-
.../authorizer/ranger/doris/RangerDorisPlugin.java | 15 +-
.../authorizer/ranger/LoadedRangerPluginTest.java | 410 +++++++++++++++++++++
.../ranger/RangerUserStoreGroupsTest.java | 206 +++++++++++
.../authorizer/ranger/doris/RangerTest.java | 91 ++++-
.../ranger/hive/RangerHiveAccessController.java | 3 +
.../hive/RangerHiveAccessControllerFactory.java | 8 +-
.../ranger/hive/RangerHiveAuditStack.java | 2 +-
.../authorizer/ranger/hive/RangerHivePlugin.java | 16 +-
.../RangerHiveAccessControllerFactoryTest.java | 2 +-
.../hive/RangerHiveAccessControllerTest.java | 36 ++
.../apache/doris/datasource/CatalogFactory.java | 5 +-
.../mysql/privilege/AccessControllerManager.java | 5 +-
.../data/ranger_p2/test_ranger_group_policy.out | 9 +
.../doris/regression/action/HttpCliAction.groovy | 6 +
regression-test/plugins/plugin_ranger.groovy | 77 +++-
.../ranger_p2/test_ranger_group_policy.groovy | 218 +++++++++++
21 files changed, 1480 insertions(+), 26 deletions(-)
diff --git
a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-common/src/main/java/org/apache/doris/catalog/authorizer/ranger/LoadedRangerPlugin.java
b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-common/src/main/java/org/apache/doris/catalog/authorizer/ranger/LoadedRangerPlugin.java
new file mode 100644
index 00000000000..e5c7f897155
--- /dev/null
+++
b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-common/src/main/java/org/apache/doris/catalog/authorizer/ranger/LoadedRangerPlugin.java
@@ -0,0 +1,159 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.catalog.authorizer.ranger;
+
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+import org.apache.ranger.plugin.service.RangerBasePlugin;
+import org.apache.ranger.plugin.util.ServicePolicies;
+
+import java.util.concurrent.TimeUnit;
+
+/**
+ * What the two Ranger plugins Doris embeds have in common: they are built
with their service's policies or
+ * not at all, and the requests built over them carry the groups Ranger keeps
for a user.
+ *
+ * <p><b>Built with the policies, or not at all.</b> {@code
RangerBasePlugin.init()} downloads the service's
+ * roles and policies - and, with the enricher below on them, its user store -
before it returns, and it
+ * survives a Ranger admin it cannot reach: it answers out of the local policy
cache
+ * ({@code ranger.plugin.<type>.policy.cache.dir}) when there is one, and out
of no policies at all when there
+ * is not, refusing every check until a later poll succeeds. That is the right
thing for a plugin embedded in
+ * a service that has to keep running, and the wrong thing at the moment Doris
binds one: an FE whose instance
+ * scope {@code access_controller_type=ranger-doris} governs, and which has no
policies, is an FE nobody can
+ * use - no account bypasses the source - and a catalog bound to a {@code
ranger-hive} source in that state
+ * refuses every statement against it, in both cases with nothing but a line
in fe.log to say why. So
+ * {@link #init()} refuses that state instead: a load that ended with no
policies, from the admin or the
+ * cache - or with policies Ranger could build no engine out of, which it also
survives, by logging - stops
+ * what it started and throws, with the cause. That fails the FE start, the
{@code CREATE CATALOG} (whose
+ * dry run builds the plugin), or the statement binding a catalog to its
source again, which is where an
+ * operator sees it. A configuration the load could not use is refused before
it starts, for the same
+ * reader; see {@link RangerUserStoreGroups#validate}. Once built, an outage
of the admin is Ranger's to
+ * survive, as it always was: the refresher keeps the last policies it
downloaded and keeps polling.
+ *
+ * <p>The user store is not part of that. A load that found the policies but
no user store is logged and
+ * accepted - policy items written against a group do not apply until the
store arrives, which the enricher
+ * keeps asking for - because "no store has arrived" cannot be told apart from
"this admin serves none", and
+ * an admin from before the user store download existed, or one that fails
that download while serving the
+ * policies, ran every deployment before groups were attached at all; see
{@link RangerUserStoreGroups}.
+ *
+ * <p><b>Requests carry Ranger's own groups.</b> Doris has none to offer, and
a policy item written against a
+ * group matches nothing without them; the store they are read from is asked
for with the policies, see
+ * {@link #setPolicies} and {@link RangerUserStoreGroups}.
+ */
+public abstract class LoadedRangerPlugin extends RangerBasePlugin {
+ private static final Logger LOG =
LogManager.getLogger(LoadedRangerPlugin.class);
+
+ /**
+ * The version of the policies the last {@link #setPolicies} was handed
and could not build an engine
+ * out of; null when it built one, or was handed none. {@code
RangerBasePlugin.setPolicies} catches
+ * whatever the engine's construction throws and leaves the plugin without
an engine, which is the state
+ * "no policies" leaves it in as well, and not the same cause: {@link
#init} reads this to tell the two
+ * apart when it refuses the plugin.
+ */
+ private volatile Long policiesWithoutEngine;
+
+ protected LoadedRangerPlugin(String serviceType, String serviceName,
String appId) {
+ super(serviceType, serviceName, appId);
+ }
+
+ /**
+ * Loads the plugin - {@code RangerBasePlugin.init()}: roles, policies and
user store, on this thread -
+ * and refuses to leave it without policies; see the class comment.
+ *
+ * @throws IllegalArgumentException for a configuration the load could not
use, before it starts; see
+ * {@link RangerUserStoreGroups#validate}
+ * @throws IllegalStateException when the load ended with no policies from
either the admin or the
+ * cache, or with policies no engine could be built out of
+ */
+ @Override
+ public void init() {
+ RangerUserStoreGroups.validate(getConfig());
+ LOG.info(RangerUserStoreGroups.describe(getConfig()));
+ long startedAtNanos = System.nanoTime();
+ try {
+ super.init();
+ if (getPoliciesVersion() < 0) {
+ Long refusedVersion = policiesWithoutEngine;
+ throw new IllegalStateException(refusedVersion == null
+ ? describeNoPolicies() :
describeNoEngine(refusedVersion));
+ }
+ } catch (RuntimeException | Error e) {
+ // Stops what the load had started before it failed - the policy
refresher and its download
+ // timer, and the engine if one was built - so that a plugin
nobody will hold does not go on
+ // polling the admin. A configuration the load could not use fails
inside RangerBasePlugin.init()
+ // before any of that is started, and stopping nothing is a no-op.
+ try {
+ cleanup();
+ } catch (RuntimeException | Error stopFailure) {
+ e.addSuppressed(stopFailure);
+ }
+ throw e;
+ }
+ LOG.info("Ranger service {} loaded in {} ms: policies version {},
roles version {}, user store"
+ + " version {}", getServiceName(),
+ TimeUnit.NANOSECONDS.toMillis(System.nanoTime() -
startedAtNanos), getPoliciesVersion(),
+ getRolesVersion(), getUserStoreVersion());
+ if (RangerUserStoreGroups.enabledFor(getConfig()) &&
getUserStoreVersion() < 0) {
+ // Policies to answer out of, but no user store to read groups
from: the admin could not be reached
+ // for it and none was cached. Requests carry no groups until one
arrives - the enricher keeps
+ // asking - which is what this source sent before groups were
attached at all, and which a policy
+ // item written against a group, a deny included, does not match.
+ LOG.warn("Ranger service {} loaded its policies but no user store:
until one arrives, requests"
+ + " carry no groups and policy items written against a
group do not apply", getServiceName());
+ }
+ }
+
+ /** Why the plugin was refused: what was asked for the policies, and where
they could have been cached. */
+ private String describeNoPolicies() {
+ String prefix = getConfig().getPropertyPrefix();
+ String adminUrl = getConfig().get(prefix + ".policy.rest.url");
+ String cacheDir = getConfig().get(prefix + ".policy.cache.dir");
+ return "Ranger service " + getServiceName() + " (type " +
getServiceType() + ") has no policies to"
+ + " authorize against: Ranger Admin at " + adminUrl + " could
not be reached or does not know"
+ + " the service, and "
+ + (cacheDir == null
+ ? "no policy cache directory is configured (" + prefix
+ ".policy.cache.dir)"
+ : "no policy cache was found under " + cacheDir)
+ + ". Fix ranger-" + getServiceType() + "-security.xml in
fe/conf, or bring Ranger Admin back,"
+ + " and try again";
+ }
+
+ /** Why the plugin was refused when its policies did arrive: nothing could
be built out of them. */
+ private String describeNoEngine(long policiesVersion) {
+ return "Ranger service " + getServiceName() + " (type " +
getServiceType() + ") has no policies to"
+ + " authorize against: its policies (version " +
policiesVersion + ") were downloaded, but no"
+ + " policy engine could be built out of them;
RangerBasePlugin.setPolicies logged the cause"
+ + " just before this. Fix what it names and try again";
+ }
+
+ /**
+ * Takes the policies Ranger downloaded, and asks for the user store with
them, so that the requests the
+ * source over this plugin builds can carry the groups Ranger keeps for a
user; see
+ * {@link RangerUserStoreGroups}. Done on the way in rather than left to
Ranger, because Ranger only does
+ * it on its own from 2.5 on, and behind a property.
+ *
+ * <p>Notes, for {@link #init}, whether the policies handed over left the
plugin without an engine.
+ */
+ @Override
+ public void setPolicies(ServicePolicies policies) {
+ RangerUserStoreGroups.addUserStoreEnricher(getConfig(), policies);
+ super.setPolicies(policies);
+ Long version = policies == null ? null : policies.getPolicyVersion();
+ policiesWithoutEngine = version != null && getPoliciesVersion() < 0 ?
version : null;
+ }
+}
diff --git
a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-common/src/main/java/org/apache/doris/catalog/authorizer/ranger/RangerAccessController.java
b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-common/src/main/java/org/apache/doris/catalog/authorizer/ranger/RangerAccessController.java
index eb36f2c45ea..81b2134130c 100644
---
a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-common/src/main/java/org/apache/doris/catalog/authorizer/ranger/RangerAccessController.java
+++
b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-common/src/main/java/org/apache/doris/catalog/authorizer/ranger/RangerAccessController.java
@@ -238,6 +238,15 @@ public abstract class RangerAccessController implements
AuthorizationPlugin {
return
context.getClientIp().filter(StringUtils::isNotBlank).orElseGet(subject::getHost);
}
+ /**
+ * The groups to put on a request about {@code subject}: the ones Ranger's
own user store puts the user
+ * in, see {@link RangerUserStoreGroups}. Empty when the plugin has no
user store, for a user the store does
+ * not know, and when the deployment has switched this off.
+ */
+ protected Set<String> groupsOf(AuthorizedSubject subject) {
+ return RangerUserStoreGroups.groupsOf(getPlugin(), subject.getUser());
+ }
+
private static boolean deferenceFrom(Map<String, String> properties) {
String configured = properties == null ? null :
properties.get(DEFER_TO_GLOBAL_SCOPE_AUTHORITY);
if (configured == null) {
diff --git
a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-common/src/main/java/org/apache/doris/catalog/authorizer/ranger/RangerUserStoreGroups.java
b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-common/src/main/java/org/apache/doris/catalog/authorizer/ranger/RangerUserStoreGroups.java
new file mode 100644
index 00000000000..3053b0a29a4
--- /dev/null
+++
b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-common/src/main/java/org/apache/doris/catalog/authorizer/ranger/RangerUserStoreGroups.java
@@ -0,0 +1,212 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.catalog.authorizer.ranger;
+
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+import org.apache.ranger.authorization.hadoop.config.RangerPluginConfig;
+import org.apache.ranger.plugin.contextenricher.RangerAdminUserStoreRetriever;
+import org.apache.ranger.plugin.contextenricher.RangerUserStoreEnricher;
+import org.apache.ranger.plugin.policyengine.RangerPluginContext;
+import org.apache.ranger.plugin.service.RangerAuthContext;
+import org.apache.ranger.plugin.service.RangerBasePlugin;
+import org.apache.ranger.plugin.util.RangerUserStoreUtil;
+import org.apache.ranger.plugin.util.ServiceDefUtil;
+import org.apache.ranger.plugin.util.ServicePolicies;
+
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.Set;
+
+/**
+ * Where a request built by a Doris source gets the groups a Ranger policy
item may be written against.
+ *
+ * <p>A Ranger policy item names users, groups and roles, and Ranger's own
plugins fill in the groups from
+ * the service they run in - the Hive plugin asks Hadoop's group mapping.
Doris has nothing to ask: an
+ * account holds roles, and no part of the engine knows which groups a user is
in. So a request that carries
+ * no groups matches no group item at all, and a service whose policies are
kept by group - the normal way
+ * to run Ranger, so that nobody edits a policy each time somebody joins a
team - grants nothing to a Doris
+ * user; and, worse, denies nothing either. An item that denies a group is
exactly as silent about it.
+ *
+ * <p>What Ranger provides for a plugin in that position is its <b>user
store</b>: the users and groups Ranger
+ * Admin itself holds, kept current by usersync from LDAP or the OS, which a
plugin downloads next to its
+ * policies through a {@link RangerUserStoreEnricher}. Ranger 2.5 and later
reads the groups of the request's
+ * user out of it when {@code ranger.plugin.<type>.use.rangerGroups} is set;
the two sources Doris embeds do
+ * the same thing here, in the request builder, so that it neither depends on
the Ranger version they were
+ * built against - branches still on 2.4 have no such setting - nor on an
operator finding a Ranger property
+ * no Doris document names.
+ *
+ * <p>It is on by default, because a group item that is silently ignored is a
bug in the deployment's eyes,
+ * not a behaviour anybody opted into; the property Ranger reads for this is
the one that switches it off,
+ * so that a deployment which has already decided the question in Ranger's
terms has decided it here too.
+ * Switched off, requests carry no groups and this source matches what it
matched before.
+ */
+public final class RangerUserStoreGroups {
+ private static final Logger LOG =
LogManager.getLogger(RangerUserStoreGroups.class);
+
+ /**
+ * Suffix of the property switching this off, {@code
ranger.plugin.<type>.use.rangerGroups} in full: the
+ * name Ranger itself reads for the same thing, so that one setting
decides both.
+ */
+ public static final String USE_RANGER_GROUPS = ".use.rangerGroups";
+
+ /** How often the plugin asks Ranger Admin for a newer user store, when
nothing configures it. */
+ private static final long DEFAULT_REFRESH_INTERVAL_MS = 60 * 1000L;
+
+ private RangerUserStoreGroups() {
+ }
+
+ /**
+ * Whether the plugin configured by {@code config} attaches user store
groups; on unless switched off.
+ *
+ * <p>Read strictly - {@code true}, {@code false}, or nothing - and not
through Hadoop's {@code getBoolean},
+ * which takes any value it cannot read as the default: a mistyped opt-out
({@code flase}) would switch this
+ * on, in a deployment that has just decided the opposite, and Ranger
reads the same property with the
+ * opposite default, so that the two would disagree about what a single
setting says.
+ *
+ * @throws IllegalArgumentException for a value that is neither; {@link
#validate} raises it before the
+ * load, so that a request never meets it
+ */
+ public static boolean enabledFor(RangerPluginConfig config) {
+ // No configuration at all - a plugin stubbed out in a test - has not
switched anything off.
+ if (config == null) {
+ return true;
+ }
+ String property = config.getPropertyPrefix() + USE_RANGER_GROUPS;
+ String value = config.getTrimmed(property);
+ if (value == null || value.isEmpty() ||
value.equalsIgnoreCase("true")) {
+ return true;
+ }
+ if (value.equalsIgnoreCase("false")) {
+ return false;
+ }
+ throw new IllegalArgumentException("Ranger service " +
config.getServiceName() + ": " + property + "="
+ + value + " is neither true nor false; leave it unset or set
it to false to switch off the"
+ + " groups requests carry");
+ }
+
+ /**
+ * How often the plugin asks Ranger Admin for a newer user store, in
milliseconds: what
+ * {@code userStoreRefresherPollingInterval} says, or a minute.
+ *
+ * <p>Read here rather than left to the enricher, which parses the option
inside the policy engine's
+ * construction: a value that is not a number fails there, and one that is
not positive fails a step
+ * later in {@code Timer.schedule}, after the enricher has downloaded the
store and started its refresher
+ * thread. {@code RangerBasePlugin.setPolicies} catches both and leaves
the plugin without an engine, which
+ * {@link LoadedRangerPlugin#init} would refuse for the wrong reason -
and, in the second case, with that
+ * thread left behind. So both are refused before the load, by {@link
#validate}.
+ *
+ * @throws IllegalArgumentException for a value that is not a positive
number of milliseconds
+ */
+ static long refreshIntervalMsOf(RangerPluginConfig config) {
+ String property =
RangerUserStoreEnricher.USERSTORE_REFRESHER_POLLINGINTERVAL_OPTION;
+ String value = config.getTrimmed(property);
+ if (value == null || value.isEmpty()) {
+ return DEFAULT_REFRESH_INTERVAL_MS;
+ }
+ long intervalMs;
+ try {
+ intervalMs = Long.parseLong(value);
+ } catch (NumberFormatException e) {
+ throw new IllegalArgumentException("Ranger service " +
config.getServiceName() + ": " + property + "="
+ + value + " is not a number of milliseconds", e);
+ }
+ if (intervalMs <= 0) {
+ throw new IllegalArgumentException("Ranger service " +
config.getServiceName() + ": " + property + "="
+ + value + " is not a positive number of milliseconds");
+ }
+ return intervalMs;
+ }
+
+ /**
+ * Refuses a configuration this cannot run with, before {@code
RangerBasePlugin.init()} starts anything:
+ * an opt-out that is neither true nor false, and a refresh interval that
is not a positive number of
+ * milliseconds. See {@link #enabledFor} and {@link #refreshIntervalMsOf}
for where each would fail
+ * otherwise, and how much worse.
+ *
+ * @throws IllegalArgumentException naming the property and its value
+ */
+ public static void validate(RangerPluginConfig config) {
+ if (config == null || !enabledFor(config)) {
+ return;
+ }
+ refreshIntervalMsOf(config);
+ }
+
+ /**
+ * Puts a user store enricher on the service definition Ranger just
downloaded, unless one is there or
+ * this is switched off, so that the plugin fetches and refreshes the user
store {@link #groupsOf} reads.
+ *
+ * <p>Called by {@link LoadedRangerPlugin#setPolicies} before handing the
policies on, on every
+ * call and not only the first: a download of policy deltas comes with its
own copy of the service
+ * definition, which is why {@code RangerBasePlugin} re-adds the enricher
on deltas too. The retriever
+ * class and the refresh interval are read under the option names Ranger
itself reads them under, so that
+ * an operator who has tuned them for Ranger's own {@code
use.rangerGroups} has tuned them here.
+ */
+ public static void addUserStoreEnricher(RangerPluginConfig config,
ServicePolicies policies) {
+ if (policies == null || config == null || !enabledFor(config)) {
+ return;
+ }
+ String retriever =
config.get(RangerUserStoreEnricher.USERSTORE_RETRIEVER_CLASSNAME_OPTION,
+ RangerAdminUserStoreRetriever.class.getCanonicalName());
+ String refreshIntervalMs = Long.toString(refreshIntervalMsOf(config));
+ // Ranger logs the addition itself, once per download that needed it;
the operator-facing line about
+ // why the store is downloaded at all is the plugin's, written once
when it starts (see describe).
+ if (ServiceDefUtil.addUserStoreEnricher(policies, retriever,
refreshIntervalMs) && LOG.isDebugEnabled()) {
+ LOG.debug("Ranger service {} will download its user store every {}
ms", policies.getServiceName(),
+ refreshIntervalMs);
+ }
+ }
+
+ /** One line for the plugin's start-up log, saying what this does for it
and how to switch it off. */
+ public static String describe(RangerPluginConfig config) {
+ String property = config.getPropertyPrefix() + USE_RANGER_GROUPS;
+ return enabledFor(config)
+ ? "Ranger service " + config.getServiceName() + ": requests
carry the groups Ranger's user store"
+ + " puts the user in, so that policy items written
against a group apply; set "
+ + property + "=false to switch that off"
+ : "Ranger service " + config.getServiceName() + ": " +
property + "=false, so requests carry no"
+ + " groups and policy items written against a group
never apply";
+ }
+
+ /**
+ * The groups the user store {@code plugin} has downloaded puts {@code
user} in.
+ *
+ * <p>Empty when this is switched off, when no store has arrived - Ranger
Admin could not be reached for
+ * it and nothing was cached, which {@link LoadedRangerPlugin} says why it
does not refuse - and when the
+ * store does not know the user, which is the case for every account that
exists in Doris only. Empty
+ * and not null on purpose: a request with an empty group set matches
items written against users and
+ * roles exactly as it did before.
+ */
+ public static Set<String> groupsOf(RangerBasePlugin plugin, String user) {
+ if (user == null || !enabledFor(plugin.getConfig())) {
+ return Collections.emptySet();
+ }
+ // The auth context the policy engine publishes is where Ranger's own
request processing reads the
+ // store from; it is replaced together with the engine, and the store
is carried over when it is.
+ RangerPluginContext pluginContext = plugin.getPluginContext();
+ RangerAuthContext authContext = pluginContext == null ? null :
pluginContext.getAuthContext();
+ RangerUserStoreUtil userStore = authContext == null ? null :
authContext.getUserStoreUtil();
+ Set<String> groups = userStore == null ? null :
userStore.getUserGroups(user);
+ if (groups == null || groups.isEmpty()) {
+ return Collections.emptySet();
+ }
+ // A copy, because the set belongs to the store shared by every
request until the next download.
+ return new HashSet<>(groups);
+ }
+}
diff --git
a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerDorisAccessController.java
b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerDorisAccessController.java
index 68c9c7cb4c0..88f93d66363 100644
---
a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerDorisAccessController.java
+++
b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerDorisAccessController.java
@@ -258,11 +258,16 @@ public class RangerDorisAccessController extends
RangerAccessController {
protected RangerAccessRequestImpl createRequest(AuthorizedSubject subject,
AccessContext context) {
RangerAccessRequestImpl request = new RangerAccessRequestImpl();
request.setUser(subject.getUser());
+ // The groups are Ranger's own, out of the user store this source's
plugin downloads: Doris has no
+ // groups to offer, and without them a policy item written against a
group never matches, allow or
+ // deny. See RangerUserStoreGroups, including for how a deployment
switches it off.
+ request.setUserGroups(groupsOf(subject));
// No user roles, unlike ranger-hive, which does send them. Not an
oversight and not free to change:
// a request carrying roles matches policy items written against a
role, so sending them would start
// granting - and denying - on policies this source has never matched,
in every deployment that has
// any. That is a change to what an existing Ranger service decides
and belongs with a release note
- // of its own, not here.
+ // of its own, not here. (Ranger's own roles are unaffected: with none
sent, Ranger resolves the ones
+ // it holds for the user and now also for the user's groups.)
request.setClientIPAddress(clientAddressOf(subject, context));
request.setClusterType(CLIENT_TYPE_DORIS);
request.setClientType(CLIENT_TYPE_DORIS);
diff --git
a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerDorisAccessControllerFactory.java
b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerDorisAccessControllerFactory.java
index ae8d3802786..69023fe0028 100644
---
a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerDorisAccessControllerFactory.java
+++
b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerDorisAccessControllerFactory.java
@@ -59,9 +59,9 @@ public class RangerDorisAccessControllerFactory implements
AuthorizationPluginFa
* one Ranger service, which is bounded and not a leak that grows.
Stopping it would cost more than that:
* a plain {@code ALTER CATALOG} detaches and re-attaches the catalog's
access controller, and a plugin
* torn down and rebuilt between those two pays {@code cleanup()} on the
DDL thread - it interrupts the
- * policy refresher and joins it without a timeout - and then two
synchronous admin REST calls on the way
- * back up, since {@code RangerBasePlugin.init()} loads the service's
roles and policies before it
- * returns.
+ * policy refresher and joins it without a timeout - and then three
synchronous admin REST calls on the
+ * way back up, since {@code RangerBasePlugin.init()} loads the service's
roles, policies and user store
+ * before it returns.
*/
private static RangerBasePlugin sharedPlugin;
private static final Map<Map<String, String>, Held> byConfiguration = new
LinkedHashMap<>();
@@ -131,8 +131,8 @@ public class RangerDorisAccessControllerFactory implements
AuthorizationPluginFa
return held.controller;
}
}
- // Built with no lock held: RangerBasePlugin.init() loads the
service's roles and its policies
- // over REST before it returns, so against a slow or
unreachable Ranger admin doing it under
+ // Built with no lock held: RangerBasePlugin.init() loads the
service's roles, policies and user
+ // store over REST before it returns, so against a slow or
unreachable Ranger admin doing it under
// the lock queues every other binding's create - and close -
behind the whole REST timeout.
// Losing the race that opens costs one plugin, stopped in the
finally below.
built = new RangerDorisPlugin(SERVICE_NAME);
diff --git
a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerDorisPlugin.java
b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerDorisPlugin.java
index 0da65aaeb7f..1a6c6b128f2 100644
---
a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerDorisPlugin.java
+++
b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerDorisPlugin.java
@@ -17,17 +17,24 @@
package org.apache.doris.catalog.authorizer.ranger.doris;
+import org.apache.doris.catalog.authorizer.ranger.LoadedRangerPlugin;
+
import org.apache.ranger.plugin.service.RangerAuthContextListener;
-import org.apache.ranger.plugin.service.RangerBasePlugin;
-public class RangerDorisPlugin extends RangerBasePlugin {
+/**
+ * The plugin over a Ranger service of type {@code doris}: built with the
service's policies or not at all, so
+ * that an FE whose instance scope it governs starts with them or does not
start; see
+ * {@link LoadedRangerPlugin}.
+ */
+public class RangerDorisPlugin extends LoadedRangerPlugin {
public RangerDorisPlugin(String serviceName) {
this(serviceName, null);
}
public RangerDorisPlugin(String serviceName, RangerAuthContextListener
rangerAuthContextListener) {
super(serviceName, null, null);
- super.init();
- super.registerAuthContextEventListener(rangerAuthContextListener);
+ // Registered before the load, so that the listener hears of the
engine the load installs.
+ registerAuthContextEventListener(rangerAuthContextListener);
+ init();
}
}
diff --git
a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/test/java/org/apache/doris/catalog/authorizer/ranger/LoadedRangerPluginTest.java
b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/test/java/org/apache/doris/catalog/authorizer/ranger/LoadedRangerPluginTest.java
new file mode 100644
index 00000000000..bc1038bbc29
--- /dev/null
+++
b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/test/java/org/apache/doris/catalog/authorizer/ranger/LoadedRangerPluginTest.java
@@ -0,0 +1,410 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.catalog.authorizer.ranger;
+
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.ImmutableSet;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.ranger.admin.client.RangerAdminClient;
+import org.apache.ranger.authorization.utils.JsonUtils;
+import org.apache.ranger.plugin.contextenricher.RangerAbstractContextEnricher;
+import org.apache.ranger.plugin.model.RangerRole;
+import org.apache.ranger.plugin.model.RangerServiceDef;
+import org.apache.ranger.plugin.policyengine.RangerAccessRequest;
+import org.apache.ranger.plugin.util.GrantRevokeRequest;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.apache.ranger.plugin.util.RangerRoles;
+import org.apache.ranger.plugin.util.RangerServiceNotFoundException;
+import org.apache.ranger.plugin.util.RangerUserStore;
+import org.apache.ranger.plugin.util.ServicePolicies;
+import org.apache.ranger.plugin.util.ServiceTags;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.io.TempDir;
+
+import java.io.IOException;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+
+/**
+ * The contract of {@link LoadedRangerPlugin}, run through Ranger's real
{@code RangerBasePlugin.init()} -
+ * policy refresher, policy cache and user store enricher included - against a
Ranger admin this test stands
+ * in for: the plugin is built out of what the admin serves or what the cache
holds, and not at all otherwise.
+ */
+public class LoadedRangerPluginTest {
+ private static final String PREFIX = "ranger.plugin.test";
+ /** Named in a refusal; never dialed, since {@link Admin} takes the REST
client's place. */
+ private static final String ADMIN_URL = "http://ranger.invalid:6080";;
+
+ /**
+ * Stands in for Ranger Admin. Ranger builds it by class name ({@code
policy.source.impl}) and asks it for
+ * the roles, the policies and the user store exactly as it asks its REST
client. Down, every call throws,
+ * which is all an unreachable admin is to Ranger; up, it serves what the
test put here.
+ */
+ public static final class Admin implements RangerAdminClient {
+ private static volatile ServicePolicies policies;
+ private static volatile RangerUserStore userStore;
+ private static volatile boolean serviceUnknown;
+
+ private static void down() {
+ policies = null;
+ userStore = null;
+ serviceUnknown = false;
+ }
+
+ @Override
+ public void init(String serviceName, String appId, String
configPropertyPrefix, Configuration config) {
+ }
+
+ @Override
+ public ServicePolicies getServicePoliciesIfUpdated(long
lastKnownVersion, long lastActivationTimeInMillis)
+ throws Exception {
+ if (serviceUnknown) {
+ throw new RangerServiceNotFoundException("test");
+ }
+ ServicePolicies current = policies;
+ if (current == null) {
+ throw new Exception("connection refused");
+ }
+ return lastKnownVersion < current.getPolicyVersion() ? current :
null;
+ }
+
+ @Override
+ public RangerRoles getRolesIfUpdated(long lastKnownRoleVersion, long
lastActivationTimeInMills)
+ throws Exception {
+ if (policies == null) {
+ throw new Exception("connection refused");
+ }
+ return null;
+ }
+
+ @Override
+ public RangerUserStore getUserStoreIfUpdated(long
lastKnownUserStoreVersion, long lastActivationTimeInMillis)
+ throws Exception {
+ RangerUserStore current = userStore;
+ if (current == null) {
+ throw new Exception("connection refused");
+ }
+ return lastKnownUserStoreVersion < current.getUserStoreVersion() ?
current : null;
+ }
+
+ @Override
+ public ServiceTags getServiceTagsIfUpdated(long lastKnownVersion, long
lastActivationTimeInMillis) {
+ return null;
+ }
+
+ @Override
+ public List<String> getTagTypes(String tagTypePattern) {
+ return Collections.emptyList();
+ }
+
+ // Administration this plugin never performs.
+
+ @Override
+ public RangerRole createRole(RangerRole request) {
+ throw new UnsupportedOperationException();
+ }
+
+ @Override
+ public void dropRole(String execUser, String roleName) {
+ throw new UnsupportedOperationException();
+ }
+
+ @Override
+ public List<String> getAllRoles(String execUser) {
+ throw new UnsupportedOperationException();
+ }
+
+ @Override
+ public List<String> getUserRoles(String execUser) {
+ throw new UnsupportedOperationException();
+ }
+
+ @Override
+ public RangerRole getRole(String execUser, String roleName) {
+ throw new UnsupportedOperationException();
+ }
+
+ @Override
+ public void grantRole(GrantRevokeRoleRequest request) {
+ throw new UnsupportedOperationException();
+ }
+
+ @Override
+ public void revokeRole(GrantRevokeRoleRequest request) {
+ throw new UnsupportedOperationException();
+ }
+
+ @Override
+ public void grantAccess(GrantRevokeRequest request) {
+ throw new UnsupportedOperationException();
+ }
+
+ @Override
+ public void revokeAccess(GrantRevokeRequest request) {
+ throw new UnsupportedOperationException();
+ }
+ }
+
+ /**
+ * A context enricher the policy engine cannot be built with: Ranger
builds the enrichers a service
+ * definition names while it builds the engine, and does not catch what
their init() throws there.
+ */
+ public static final class BrokenEnricher extends
RangerAbstractContextEnricher {
+ @Override
+ public void init() {
+ throw new IllegalStateException("broken on purpose");
+ }
+
+ @Override
+ public void enrich(RangerAccessRequest request) {
+ }
+ }
+
+ private static final class TestPlugin extends LoadedRangerPlugin {
+ private TestPlugin(String cacheDir) {
+ // Service type "test" reads ranger-test-*.xml, none of which
exist here: what the load needs is set
+ // below, and the admin it asks is the class above.
+ super("test", "test", null);
+ getConfig().set(PREFIX + ".policy.source.impl",
Admin.class.getName());
+ getConfig().set(PREFIX + ".policy.rest.url", ADMIN_URL);
+ if (cacheDir != null) {
+ getConfig().set(PREFIX + ".policy.cache.dir", cacheDir);
+ }
+ }
+
+ /** Whether there is a policy engine to answer out of; what a refused
plugin must not leave behind. */
+ private boolean hasEngine() {
+ return getPolicyEngine() != null;
+ }
+ }
+
+ @TempDir
+ Path cacheDir;
+
+ private TestPlugin plugin;
+
+ @BeforeEach
+ public void takeTheAdminDown() {
+ Admin.down();
+ }
+
+ @AfterEach
+ public void stopThePlugin() {
+ if (plugin != null) {
+ plugin.cleanup();
+ }
+ }
+
+ private TestPlugin build(String cacheDir) {
+ plugin = new TestPlugin(cacheDir);
+ plugin.init();
+ return plugin;
+ }
+
+ /** Policies of {@code version} with nothing in them, enough for Ranger to
build an engine out of. */
+ private static ServicePolicies policies(long version) {
+ RangerServiceDef serviceDef = new RangerServiceDef();
+ serviceDef.setName("test");
+ ServicePolicies policies = new ServicePolicies();
+ policies.setServiceName("test");
+ policies.setServiceDef(serviceDef);
+ policies.setPolicyVersion(version);
+ policies.setPolicies(new ArrayList<>());
+ return policies;
+ }
+
+ /** What Ranger's refresher writes when the admin answers, where it reads
it back when the admin does not. */
+ private void cache(ServicePolicies policies) throws IOException {
+ // <appId>_<serviceName>.json, the appId defaulting to the service
type.
+ Files.write(cacheDir.resolve("test_test.json"),
+
JsonUtils.objectToJson(policies).getBytes(StandardCharsets.UTF_8));
+ }
+
+ private static boolean refresherRunning() {
+ return Thread.getAllStackTraces().keySet().stream()
+ .anyMatch(thread ->
thread.getName().startsWith("PolicyRefresher(serviceName=test)"));
+ }
+
+ /** The thread the user store enricher starts to download the store, which
cleanup() reaches through the engine. */
+ private static boolean userStoreRefresherRunning() {
+ return Thread.getAllStackTraces().keySet().stream()
+ .anyMatch(thread ->
thread.getName().startsWith("RangerUserStoreRefresher(serviceName=test)"));
+ }
+
+ /** Built out of what the admin serves, groups included: the user store
came with the policies. */
+ @Test
+ public void testBuiltWithWhatTheAdminServes() {
+ Admin.policies = policies(3L);
+ Admin.userStore = new RangerUserStore(1L, null, null,
+ ImmutableMap.of("user1", ImmutableSet.of("readers")));
+
+ build(cacheDir.toString());
+
+ Assertions.assertEquals(3L, plugin.getPoliciesVersion());
+ Assertions.assertEquals(1L, plugin.getUserStoreVersion());
+ // Through Ranger's own enricher, which the policies were handed on
with: what a request carries.
+ Assertions.assertEquals(ImmutableSet.of("readers"),
RangerUserStoreGroups.groupsOf(plugin, "user1"));
+ Assertions.assertTrue(RangerUserStoreGroups.groupsOf(plugin,
"nobody").isEmpty(),
+ "a user the store does not know is in a group");
+ Assertions.assertTrue(refresherRunning(), "built, the plugin keeps
polling");
+ }
+
+ /**
+ * The admin is down and what it served last time is in the cache: built
out of that, the way Ranger's
+ * own plugins ride out an outage. No store came with it, which is logged
and not refused.
+ */
+ @Test
+ public void testBuiltOutOfTheCacheWhenTheAdminIsDown() throws IOException {
+ cache(policies(7L));
+
+ build(cacheDir.toString());
+
+ Assertions.assertEquals(7L, plugin.getPoliciesVersion());
+ Assertions.assertTrue(plugin.getUserStoreVersion() < 0);
+ Assertions.assertTrue(RangerUserStoreGroups.groupsOf(plugin,
"user1").isEmpty(),
+ "groups came from somewhere with no store");
+ }
+
+ /** Nothing from the admin, nothing in the cache: refused with the cause,
and nothing left running. */
+ @Test
+ public void testRefusedWithNoPoliciesFromTheAdminOrTheCache() {
+ plugin = new TestPlugin(cacheDir.toString());
+
+ IllegalStateException refused =
Assertions.assertThrows(IllegalStateException.class, plugin::init);
+
+ Assertions.assertTrue(refused.getMessage().contains("has no policies
to authorize against"),
+ refused.getMessage());
+ Assertions.assertTrue(refused.getMessage().contains(ADMIN_URL),
refused.getMessage());
+
Assertions.assertTrue(refused.getMessage().contains(cacheDir.toString()),
refused.getMessage());
+ Assertions.assertFalse(plugin.hasEngine(), "an engine was left
behind");
+ Assertions.assertFalse(refresherRunning(), "the policy refresher
init() started is still running");
+ }
+
+ @Test
+ public void testRefusedWithNoCacheDirectoryConfigured() {
+ plugin = new TestPlugin(null);
+
+ IllegalStateException refused =
Assertions.assertThrows(IllegalStateException.class, plugin::init);
+
+ Assertions.assertTrue(refused.getMessage().contains(PREFIX +
".policy.cache.dir"), refused.getMessage());
+ Assertions.assertFalse(refresherRunning());
+ }
+
+ /** A service the admin does not know - a mistyped service name - has no
policies, cache or no cache. */
+ @Test
+ public void testRefusedForAServiceTheAdminDoesNotKnow() throws IOException
{
+ cache(policies(7L));
+ Admin.serviceUnknown = true;
+ plugin = new TestPlugin(cacheDir.toString());
+
+ Assertions.assertThrows(IllegalStateException.class, plugin::init);
+
+ Assertions.assertFalse(plugin.hasEngine(), "an engine was left
behind");
+ Assertions.assertFalse(refresherRunning());
+ }
+
+ /** A configuration the load cannot use fails as it always did: on the
calling thread, with its cause. */
+ @Test
+ public void testAConfigurationTheLoadCannotUseIsRefusedWithItsCause() {
+ plugin = new TestPlugin(cacheDir.toString());
+ plugin.getConfig().set(PREFIX + ".policy.pollIntervalMs", "often");
+
+ Assertions.assertThrows(NumberFormatException.class, plugin::init);
+
+ Assertions.assertFalse(refresherRunning());
+ }
+
+ /**
+ * A refresh interval the enricher could not schedule is refused before
the load starts. Left to the
+ * enricher, a non-positive interval fails in Timer.schedule after the
store is downloaded and the
+ * refresher thread is up, inside the engine's construction:
RangerBasePlugin.setPolicies catches it, the
+ * plugin is refused as if it had no policies, and cleanup() cannot reach
the thread through an engine
+ * that was never built.
+ */
+ @Test
+ public void
testRefusedBeforeTheLoadForAnIntervalTheEnricherCouldNotSchedule() {
+ Admin.policies = policies(3L);
+ Admin.userStore = new RangerUserStore(1L, null, null,
ImmutableMap.of("user1", ImmutableSet.of("readers")));
+ plugin = new TestPlugin(cacheDir.toString());
+ plugin.getConfig().set("userStoreRefresherPollingInterval", "0");
+
+ IllegalArgumentException refused =
Assertions.assertThrows(IllegalArgumentException.class, plugin::init);
+
+
Assertions.assertTrue(refused.getMessage().contains("userStoreRefresherPollingInterval=0"),
+ refused.getMessage());
+ Assertions.assertFalse(plugin.hasEngine(), "an engine was left
behind");
+ Assertions.assertFalse(refresherRunning(), "the policy refresher was
started");
+ Assertions.assertFalse(userStoreRefresherRunning(), "the user store
refresher was started");
+ }
+
+ /** An opt-out Hadoop's getBoolean would have read as the default is
refused, not switched on. */
+ @Test
+ public void testRefusedBeforeTheLoadForASwitchThatIsNeitherTrueNorFalse() {
+ Admin.policies = policies(3L);
+ plugin = new TestPlugin(cacheDir.toString());
+ plugin.getConfig().set(PREFIX + ".use.rangerGroups", "flase");
+
+ IllegalArgumentException refused =
Assertions.assertThrows(IllegalArgumentException.class, plugin::init);
+
+ Assertions.assertTrue(refused.getMessage().contains(PREFIX +
".use.rangerGroups=flase"), refused.getMessage());
+ Assertions.assertFalse(refresherRunning());
+ }
+
+ /**
+ * The policies arrived and Ranger could build no engine out of them -
here, a context enricher on the
+ * service definition whose init() throws; RangerBasePlugin.setPolicies
logs that and keeps going with
+ * no engine, the same state as no policies at all. Refused for what
happened, not for an admin that
+ * answered.
+ */
+ @Test
+ public void testRefusedWithPoliciesNoEngineCouldBeBuiltOutOf() {
+ ServicePolicies policies = policies(3L);
+ policies.getServiceDef().setContextEnrichers(Collections.singletonList(
+ new RangerServiceDef.RangerContextEnricherDef(1L, "broken",
BrokenEnricher.class.getName(), null)));
+ Admin.policies = policies;
+ plugin = new TestPlugin(cacheDir.toString());
+
+ IllegalStateException refused =
Assertions.assertThrows(IllegalStateException.class, plugin::init);
+
+ Assertions.assertTrue(refused.getMessage().contains("policies (version
3) were downloaded, but no policy"
+ + " engine could be built out of them"), refused.getMessage());
+ Assertions.assertFalse(refused.getMessage().contains("could not be
reached"), refused.getMessage());
+ Assertions.assertFalse(plugin.hasEngine(), "an engine was left
behind");
+ Assertions.assertFalse(refresherRunning());
+ }
+
+ /** The seam the groups hang on: policies are handed on with the user
store enricher on their definition. */
+ @Test
+ public void testThePoliciesAreHandedOnWithTheUserStoreEnricher() {
+ plugin = new TestPlugin(null);
+ ServicePolicies policies = policies(1L);
+
+ plugin.setPolicies(policies);
+
+ Assertions.assertNotNull(plugin.getUserStoreEnricher(), "no user store
enricher on the engine");
+
Assertions.assertTrue(policies.getServiceDef().getContextEnrichers().stream()
+ .anyMatch(enricher ->
"userStoreEnricher".equals(enricher.getName())));
+ }
+}
diff --git
a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/test/java/org/apache/doris/catalog/authorizer/ranger/RangerUserStoreGroupsTest.java
b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/test/java/org/apache/doris/catalog/authorizer/ranger/RangerUserStoreGroupsTest.java
new file mode 100644
index 00000000000..1d72f17626b
--- /dev/null
+++
b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/test/java/org/apache/doris/catalog/authorizer/ranger/RangerUserStoreGroupsTest.java
@@ -0,0 +1,206 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.catalog.authorizer.ranger;
+
+import org.apache.ranger.authorization.hadoop.config.RangerPluginConfig;
+import org.apache.ranger.plugin.contextenricher.RangerAdminUserStoreRetriever;
+import org.apache.ranger.plugin.contextenricher.RangerUserStoreEnricher;
+import org.apache.ranger.plugin.model.RangerServiceDef;
+import
org.apache.ranger.plugin.model.RangerServiceDef.RangerContextEnricherDef;
+import org.apache.ranger.plugin.util.ServiceDefUtil;
+import org.apache.ranger.plugin.util.ServicePolicies;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.Test;
+
+import java.util.List;
+
+/**
+ * The half of {@link RangerUserStoreGroups} that runs when policies arrive:
putting the user store enricher
+ * on the service definition, which is what makes the plugin download the
store the other half reads.
+ */
+public class RangerUserStoreGroupsTest {
+
+ private static RangerPluginConfig config() {
+ // Service type "test" reads ranger-test-*.xml, none of which exist
here: every property is at its
+ // default, which is the state a deployment that has never heard of
this is in.
+ return new RangerPluginConfig("test", "test", null, null, null, null);
+ }
+
+ /** Policies as Ranger Admin hands them over: a service definition
carrying no context enricher. */
+ private static ServicePolicies downloaded() {
+ RangerServiceDef serviceDef = new RangerServiceDef();
+ serviceDef.setName("test");
+ ServicePolicies policies = new ServicePolicies();
+ policies.setServiceName("test");
+ policies.setServiceDef(serviceDef);
+ return policies;
+ }
+
+ private static List<RangerContextEnricherDef> enrichersOf(ServicePolicies
policies) {
+ return policies.getServiceDef().getContextEnrichers();
+ }
+
+ @Test
+ public void testOnByDefault() {
+ Assertions.assertTrue(RangerUserStoreGroups.enabledFor(config()));
+ }
+
+ /** The switch is read as Ranger reads its own copy of it, case and
surrounding blanks aside. */
+ @Test
+ public void testReadsTheSwitchStrictly() {
+ RangerPluginConfig config = config();
+ config.set("ranger.plugin.test.use.rangerGroups", " False ");
+ Assertions.assertFalse(RangerUserStoreGroups.enabledFor(config));
+ config.set("ranger.plugin.test.use.rangerGroups", "TRUE");
+ Assertions.assertTrue(RangerUserStoreGroups.enabledFor(config));
+ config.set("ranger.plugin.test.use.rangerGroups", "");
+ Assertions.assertTrue(RangerUserStoreGroups.enabledFor(config));
+ }
+
+ /**
+ * Hadoop's getBoolean would take a mistyped opt-out as its default and
switch this on; an operator who
+ * has just decided the opposite is told so, by property and value, rather
than granted on groups.
+ */
+ @Test
+ public void testRefusesASwitchThatIsNeitherTrueNorFalse() {
+ RangerPluginConfig config = config();
+ config.set("ranger.plugin.test.use.rangerGroups", "flase");
+
+ IllegalArgumentException refused =
Assertions.assertThrows(IllegalArgumentException.class,
+ () -> RangerUserStoreGroups.validate(config));
+
+
Assertions.assertTrue(refused.getMessage().contains("ranger.plugin.test.use.rangerGroups=flase"),
+ refused.getMessage());
+ Assertions.assertThrows(IllegalArgumentException.class, () ->
RangerUserStoreGroups.enabledFor(config));
+ }
+
+ /**
+ * Ranger's enricher parses the interval inside the engine's construction,
and a non-positive one
+ * fails in Timer.schedule after the refresher thread is up; both are
refused before any of that.
+ */
+ @Test
+ public void testRefusesAnIntervalThatIsNotAPositiveNumberOfMilliseconds() {
+ for (String interval : new String[] {"0", "-1", "often", "5s"}) {
+ RangerPluginConfig config = config();
+
config.set(RangerUserStoreEnricher.USERSTORE_REFRESHER_POLLINGINTERVAL_OPTION,
interval);
+
+ IllegalArgumentException refused =
Assertions.assertThrows(IllegalArgumentException.class,
+ () -> RangerUserStoreGroups.validate(config), interval);
+
+ Assertions.assertTrue(refused.getMessage().contains(
+
RangerUserStoreEnricher.USERSTORE_REFRESHER_POLLINGINTERVAL_OPTION + "=" +
interval),
+ refused.getMessage());
+ }
+ }
+
+ @Test
+ public void testAcceptsADefaultOrPositiveInterval() {
+ RangerPluginConfig config = config();
+ Assertions.assertDoesNotThrow(() ->
RangerUserStoreGroups.validate(config));
+ Assertions.assertEquals(60000L,
RangerUserStoreGroups.refreshIntervalMsOf(config));
+
config.set(RangerUserStoreEnricher.USERSTORE_REFRESHER_POLLINGINTERVAL_OPTION,
" 5000 ");
+ Assertions.assertDoesNotThrow(() ->
RangerUserStoreGroups.validate(config));
+ Assertions.assertEquals(5000L,
RangerUserStoreGroups.refreshIntervalMsOf(config));
+ }
+
+ /** Switched off, no enricher reads the interval, so a bad one refuses
nothing. */
+ @Test
+ public void testAnIntervalDoesNotMatterWhenSwitchedOff() {
+ RangerPluginConfig config = config();
+ config.set("ranger.plugin.test.use.rangerGroups", "false");
+
config.set(RangerUserStoreEnricher.USERSTORE_REFRESHER_POLLINGINTERVAL_OPTION,
"0");
+
+ Assertions.assertDoesNotThrow(() ->
RangerUserStoreGroups.validate(config));
+ }
+
+ /**
+ * The enricher is the one Ranger adds for its own {@code
use.rangerGroups}, options included, so that a
+ * deployment that tuned the retriever or the interval for Ranger has
tuned them here.
+ */
+ @Test
+ public void testAddsTheUserStoreEnricherRangerWouldHaveAdded() {
+ ServicePolicies policies = downloaded();
+
+ RangerUserStoreGroups.addUserStoreEnricher(config(), policies);
+
+
Assertions.assertTrue(ServiceDefUtil.isUserStoreEnricherPresent(policies));
+ RangerContextEnricherDef enricher = enrichersOf(policies).get(0);
+ Assertions.assertEquals(RangerUserStoreEnricher.class.getName(),
enricher.getEnricher());
+
Assertions.assertEquals(RangerAdminUserStoreRetriever.class.getCanonicalName(),
+
enricher.getEnricherOptions().get(RangerUserStoreEnricher.USERSTORE_RETRIEVER_CLASSNAME_OPTION));
+ Assertions.assertEquals("60000",
+
enricher.getEnricherOptions().get(RangerUserStoreEnricher.USERSTORE_REFRESHER_POLLINGINTERVAL_OPTION));
+ }
+
+ @Test
+ public void testHonoursTheRetrieverAndIntervalRangerReads() {
+ RangerPluginConfig config = config();
+
config.set(RangerUserStoreEnricher.USERSTORE_RETRIEVER_CLASSNAME_OPTION,
"com.example.Retriever");
+
config.set(RangerUserStoreEnricher.USERSTORE_REFRESHER_POLLINGINTERVAL_OPTION,
"5000");
+ ServicePolicies policies = downloaded();
+
+ RangerUserStoreGroups.addUserStoreEnricher(config, policies);
+
+ RangerContextEnricherDef enricher = enrichersOf(policies).get(0);
+ Assertions.assertEquals("com.example.Retriever",
+
enricher.getEnricherOptions().get(RangerUserStoreEnricher.USERSTORE_RETRIEVER_CLASSNAME_OPTION));
+ Assertions.assertEquals("5000",
+
enricher.getEnricherOptions().get(RangerUserStoreEnricher.USERSTORE_REFRESHER_POLLINGINTERVAL_OPTION));
+ }
+
+ /** Called on every download, deltas included, so a second call must not
stack a second enricher. */
+ @Test
+ public void testAddsItOnce() {
+ ServicePolicies policies = downloaded();
+
+ RangerUserStoreGroups.addUserStoreEnricher(config(), policies);
+ RangerUserStoreGroups.addUserStoreEnricher(config(), policies);
+
+ Assertions.assertEquals(1, enrichersOf(policies).size());
+ }
+
+ @Test
+ public void testLeavesTheDefinitionAloneWhenSwitchedOff() {
+ RangerPluginConfig config = config();
+ config.set("ranger.plugin.test.use.rangerGroups", "false");
+ ServicePolicies policies = downloaded();
+
+ RangerUserStoreGroups.addUserStoreEnricher(config, policies);
+
+ Assertions.assertFalse(RangerUserStoreGroups.enabledFor(config));
+
Assertions.assertFalse(ServiceDefUtil.isUserStoreEnricherPresent(policies));
+ }
+
+ /** What the refresher hands over when the service is gone:
RangerBasePlugin copes with it, so must this. */
+ @Test
+ public void testToleratesNoPolicies() {
+ Assertions.assertDoesNotThrow(() ->
RangerUserStoreGroups.addUserStoreEnricher(config(), null));
+ }
+
+ /** The start-up line names the property either way, so an operator
reading the log knows the switch. */
+ @Test
+ public void testDescribesItselfAndTheSwitch() {
+ RangerPluginConfig config = config();
+
Assertions.assertTrue(RangerUserStoreGroups.describe(config).contains("policy
items written against a"
+ + " group apply; set
ranger.plugin.test.use.rangerGroups=false"));
+
+ config.set("ranger.plugin.test.use.rangerGroups", "false");
+ Assertions.assertTrue(RangerUserStoreGroups.describe(config).contains(
+ "ranger.plugin.test.use.rangerGroups=false, so requests carry
no groups"));
+ }
+}
diff --git
a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/test/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerTest.java
b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/test/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerTest.java
index b43973476a3..6eae5675713 100644
---
a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/test/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerTest.java
+++
b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/test/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerTest.java
@@ -32,10 +32,13 @@ import com.google.common.base.Strings;
import com.google.common.collect.Lists;
import com.google.common.collect.Sets;
import org.apache.ranger.plugin.policyengine.RangerAccessRequest;
+import org.apache.ranger.plugin.policyengine.RangerAccessRequestImpl;
import org.apache.ranger.plugin.policyengine.RangerAccessResource;
import org.apache.ranger.plugin.policyengine.RangerAccessResult;
import org.apache.ranger.plugin.policyengine.RangerAccessResultProcessor;
+import org.apache.ranger.plugin.service.RangerAuthContext;
import org.apache.ranger.plugin.service.RangerBasePlugin;
+import org.apache.ranger.plugin.util.RangerUserStore;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.Test;
@@ -56,7 +59,7 @@ public class RangerTest {
public DorisTestPlugin(String serviceName) {
super(serviceName, null, null);
- // super.init();
+ // Never init(): it answers out of the overrides below, not out of
a Ranger service.
}
@Override
@@ -131,7 +134,9 @@ public class RangerTest {
} else if (!Strings.isNullOrEmpty(db)) {
result.setIsAllowed("ctl3".equals(ctl) && "db3".equals(db));
} else if (!Strings.isNullOrEmpty(ctl)) {
- result.setIsAllowed("ctl4".equals(ctl));
+ // ctl5 stands for a policy item written against a group
rather than a user.
+ result.setIsAllowed("ctl4".equals(ctl)
+ || ("ctl5".equals(ctl) &&
request.getUserGroups().contains("readers")));
} else if (!Strings.isNullOrEmpty(cg)) {
result.setIsAllowed("cg1".equals(cg));
} else if (!Strings.isNullOrEmpty(sv)) {
@@ -353,4 +358,86 @@ public class RangerTest {
Assertions.assertEquals(0, plugin.requests.get(),
"the default workload group was put to the policy engine");
}
+
+ /**
+ * Hands {@code plugin} the user store its policy engine would have
published, mapping user1 onto
+ * {@code groups}: what the enricher does once Ranger Admin has answered
the user store download.
+ */
+ private static void userStoreArrived(RangerBasePlugin plugin, String...
groups) {
+ RangerUserStore userStore = new RangerUserStore(1L, null, null,
+ Collections.singletonMap("user1", Sets.newHashSet(groups)));
+ plugin.getPluginContext().setAuthContext(new RangerAuthContext(null,
null, null, userStore));
+ }
+
+ /**
+ * A request carries the groups Ranger's user store puts the user in, and
nothing else does: Doris has
+ * no groups of its own, so without this a policy item written against a
group never matches.
+ */
+ @Test
+ public void testRequestCarriesTheGroupsOfRangersUserStore() {
+ DorisTestPlugin plugin = new DorisTestPlugin("test");
+ userStoreArrived(plugin, "readers", "analysts");
+
+ RangerAccessRequestImpl request = new
RangerDorisAccessController(plugin, NOTHING_GRANTED_ELSEWHERE)
+ .createRequest(USER, AccessContext.NONE);
+
+ Assertions.assertEquals("user1", request.getUser());
+ Assertions.assertEquals(Sets.newHashSet("readers", "analysts"),
request.getUserGroups());
+ Assertions.assertTrue(request.getUserRoles().isEmpty(),
+ "roles were attached, which this source has never done; Ranger
resolves its own");
+ }
+
+ /** Until the plugin has a user store - not loaded yet, Ranger Admin
unreachable - there are no groups. */
+ @Test
+ public void testNoGroupsBeforeTheUserStoreArrives() {
+ RangerAccessRequestImpl request = controller().createRequest(USER,
AccessContext.NONE);
+
+ Assertions.assertTrue(request.getUserGroups().isEmpty());
+ }
+
+ /** A user the store does not know - one that exists in Doris only - is in
no group. */
+ @Test
+ public void testAUserUnknownToTheUserStoreIsInNoGroup() {
+ DorisTestPlugin plugin = new DorisTestPlugin("test");
+ userStoreArrived(plugin, "readers");
+
+ RangerAccessRequestImpl request = new
RangerDorisAccessController(plugin, NOTHING_GRANTED_ELSEWHERE)
+ .createRequest(AuthorizedSubject.of("somebody_else", "%"),
AccessContext.NONE);
+
+ Assertions.assertTrue(request.getUserGroups().isEmpty());
+ }
+
+ /**
+ * The property Ranger itself reads for this switches it off, so that a
deployment that has decided the
+ * question in Ranger's terms has decided it here too - and gets the
requests this source built before.
+ */
+ @Test
+ public void testTheRangerPropertySwitchesGroupsOff() {
+ DorisTestPlugin plugin = new DorisTestPlugin("test");
+ userStoreArrived(plugin, "readers");
+ plugin.getConfig().set("ranger.plugin.test.use.rangerGroups", "false");
+
+ RangerAccessRequestImpl request = new
RangerDorisAccessController(plugin, NOTHING_GRANTED_ELSEWHERE)
+ .createRequest(USER, AccessContext.NONE);
+
+ Assertions.assertTrue(request.getUserGroups().isEmpty());
+ }
+
+ /** The whole point: a policy item written against a group decides, once
the user is in that group. */
+ @Test
+ public void testAPolicyItemWrittenAgainstAGroupDecides() throws
AccessDeniedException {
+ DorisTestPlugin plugin = new DorisTestPlugin("test");
+ RangerDorisAccessController controller = new
RangerDorisAccessController(plugin, NOTHING_GRANTED_ELSEWHERE);
+ AuthorizedResource table = AuthorizedResource.table("ctl5", "db",
"tbl");
+
+ Assertions.assertThrows(AccessDeniedException.class,
+ () -> controller.checkPrivilege(USER, table,
AccessRequirements.SELECT, AccessContext.NONE));
+
+ userStoreArrived(plugin, "readers");
+ controller.checkPrivilege(USER, table, AccessRequirements.SELECT,
AccessContext.NONE);
+
+ userStoreArrived(plugin, "writers");
+ Assertions.assertThrows(AccessDeniedException.class,
+ () -> controller.checkPrivilege(USER, table,
AccessRequirements.SELECT, AccessContext.NONE));
+ }
}
diff --git
a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessController.java
b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessController.java
index b4a91da8f9e..7e3fe600311 100644
---
a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessController.java
+++
b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessController.java
@@ -343,6 +343,9 @@ public class RangerHiveAccessController extends
RangerAccessController {
AccessContext context) {
RangerAccessRequestImpl request = new RangerAccessRequestImpl();
request.setUser(subject.getUser());
+ // Ranger's groups for the user, out of the user store this service's
plugin downloads; a Hive
+ // service's policies are usually kept by group, and Doris has no
groups of its own to send.
+ request.setUserGroups(groupsOf(subject));
request.setUserRoles(roles);
request.setClientIPAddress(clientAddressOf(subject, context));
request.setClusterType(CLIENT_TYPE_DORIS);
diff --git
a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessControllerFactory.java
b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessControllerFactory.java
index 5fc7528199a..82428c5a470 100644
---
a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessControllerFactory.java
+++
b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessControllerFactory.java
@@ -70,7 +70,7 @@ public class RangerHiveAccessControllerFactory implements
AuthorizationPluginFac
* <p>A stack outlives the binding that started it, but not the process.
The last binding letting go does
* not stop it, because a plain {@code ALTER CATALOG} detaches and
re-attaches the catalog's access
* controller and a stack stopped between those two costs a {@code
cleanup()} on the DDL thread - it
- * interrupts the policy refresher and joins it without a timeout - and
two synchronous admin REST calls
+ * interrupts the policy refresher and joins it without a timeout - and
three synchronous admin REST calls
* on the way back up. What happens instead is that a stack nothing reads
any more is stopped after
* {@link #idleStackGraceSeconds}, and anything asking for that service
again in the meantime cancels
* the stop. Without it the map would grow by one entry, one policy
refresher thread and one download
@@ -168,9 +168,9 @@ public class RangerHiveAccessControllerFactory implements
AuthorizationPluginFac
return held.controller;
}
}
- // Nothing reads this service yet, and starting to read it
talks to the Ranger admin twice
- // before it returns: RangerBasePlugin.init() loads the
service's roles and its policies
- // synchronously, before the refresher thread starts. Built
with no lock held, so that a slow
+ // Nothing reads this service yet, and starting to read it
talks to the Ranger admin three
+ // times before it returns: RangerBasePlugin.init() loads the
service's roles, policies and user
+ // store synchronously, before the refresher thread starts.
Built with no lock held, so that a slow
// or unreachable admin cannot queue every other binding's
create - and close - behind it.
// Losing the race that opens costs one plugin, stopped in the
finally below.
built = RangerHiveAuditStack.startFor(serviceName);
diff --git
a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAuditStack.java
b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAuditStack.java
index e223ff60b55..ab19977c60a 100644
---
a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAuditStack.java
+++
b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAuditStack.java
@@ -33,7 +33,7 @@ import java.util.concurrent.TimeUnit;
* <p>Bundled because the three are one lifetime - the handler is configured
out of the plugin and the task
* exists to drain the handler - and because that lifetime is not a binding's.
A catalog bound to this source
* is detached and re-attached by a plain {@code ALTER CATALOG}, and tearing a
stack down between those two
- * costs a {@code cleanup()} on the DDL thread and two synchronous REST calls
to the Ranger admin on the way
+ * costs a {@code cleanup()} on the DDL thread and three synchronous REST
calls to the Ranger admin on the way
* back up. So {@link RangerHiveAccessControllerFactory} keeps one stack per
Ranger service, hands it to
* every controller reading that service, and stops it only once nothing has
read it for a while - while a
* controller built directly, a test or an embedding that owns its own, starts
and stops one of its own.
diff --git
a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHivePlugin.java
b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHivePlugin.java
index 18ddde88e39..170fe0c6d7f 100644
---
a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHivePlugin.java
+++
b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHivePlugin.java
@@ -17,17 +17,25 @@
package org.apache.doris.catalog.authorizer.ranger.hive;
+import org.apache.doris.catalog.authorizer.ranger.LoadedRangerPlugin;
+
import org.apache.ranger.plugin.service.RangerAuthContextListener;
-import org.apache.ranger.plugin.service.RangerBasePlugin;
-public class RangerHivePlugin extends RangerBasePlugin {
+/**
+ * The plugin over a Ranger service of type {@code hive}: built with the
service's policies or not at all, so
+ * that a catalog is bound to it with them or the binding fails; see {@link
LoadedRangerPlugin}. The groups
+ * its requests carry are Ranger's own, where Hive's plugin would have asked
Hadoop's group mapping, which
+ * Doris has no equivalent of.
+ */
+public class RangerHivePlugin extends LoadedRangerPlugin {
public RangerHivePlugin(String serviceName) {
this(serviceName, null);
}
public RangerHivePlugin(String serviceName, RangerAuthContextListener
rangerAuthContextListener) {
super(serviceName, null, null);
- super.init();
- super.registerAuthContextEventListener(rangerAuthContextListener);
+ // Registered before the load, so that the listener hears of the
engine the load installs.
+ registerAuthContextEventListener(rangerAuthContextListener);
+ init();
}
}
diff --git
a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/test/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessControllerFactoryTest.java
b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/test/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessControllerFactoryTest.java
index cf8be16ebd7..f3f6da86228 100644
---
a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/test/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessControllerFactoryTest.java
+++
b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/test/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessControllerFactoryTest.java
@@ -136,7 +136,7 @@ public class RangerHiveAccessControllerFactoryTest {
* both detach a catalog's access controller and attach a new one, so the
count of bindings on a service
* passes through zero as a matter of course. Stopped there and rebuilt,
the plugin costs a
* {@code cleanup()} on the DDL thread - it interrupts the policy
refresher and joins it with no timeout -
- * and two synchronous admin REST calls on the way back up.
+ * and three synchronous admin REST calls on the way back up.
*/
@Test
public void testAReAcquireWithinTheGraceKeepsThePluginUp() {
diff --git
a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/test/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessControllerTest.java
b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/test/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessControllerTest.java
index 79a48e63420..7485c5c6c59 100644
---
a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/test/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessControllerTest.java
+++
b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/test/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessControllerTest.java
@@ -28,11 +28,15 @@ import
org.apache.doris.authorization.spi.AuthorizationContext;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.ImmutableSet;
+import org.apache.ranger.authorization.hadoop.config.RangerPluginConfig;
import org.apache.ranger.plugin.model.RangerPolicy;
import org.apache.ranger.plugin.policyengine.RangerAccessRequest;
import org.apache.ranger.plugin.policyengine.RangerAccessRequestImpl;
import org.apache.ranger.plugin.policyengine.RangerAccessResult;
+import org.apache.ranger.plugin.policyengine.RangerPluginContext;
import org.apache.ranger.plugin.policyengine.RangerPolicyEngine;
+import org.apache.ranger.plugin.service.RangerAuthContext;
+import org.apache.ranger.plugin.util.RangerUserStore;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.Test;
import org.mockito.ArgumentCaptor;
@@ -102,6 +106,38 @@ public class RangerHiveAccessControllerTest {
};
}
+ /**
+ * Groups, unlike roles, are not the engine's to know: Doris has none, so
they are Ranger's own, read out
+ * of the user store the plugin downloads - which for a Hive service is
where its policies' groups come
+ * from in the first place. Without them every policy item written against
a group is dead to Doris.
+ */
+ @Test
+ public void testRequestCarriesTheGroupsOfRangersUserStore() {
+ AuthorizationContext context =
Mockito.mock(AuthorizationContext.class);
+ Mockito.when(context.rolesOf(SUBJECT)).thenReturn(ImmutableSet.of());
+ // The plugin context as the policy engine leaves it once the user
store enricher has run.
+ RangerPluginContext pluginContext = new RangerPluginContext(
+ new RangerPluginConfig("hive", "hive", null, null, null,
null));
+ pluginContext.setAuthContext(new RangerAuthContext(null, null, null,
new RangerUserStore(1L, null, null,
+ ImmutableMap.of("user1", ImmutableSet.of("analysts",
"etl")))));
+
+ try (MockedConstruction<RangerHivePlugin> plugin =
Mockito.mockConstruction(RangerHivePlugin.class,
+ (mock, settings) ->
Mockito.when(mock.getPluginContext()).thenReturn(pluginContext));
+ MockedConstruction<RangerHiveAuditHandler> audit =
+
Mockito.mockConstruction(RangerHiveAuditHandler.class)) {
+ RangerHiveAccessController controller = new
RangerHiveAccessController(
+ ImmutableMap.of("ranger.service.name", "hive"), context);
+ try {
+ Assertions.assertEquals(ImmutableSet.of("analysts", "etl"),
+ controller.createRequest(SUBJECT,
AccessContext.NONE).getUserGroups());
+
Assertions.assertTrue(controller.createRequest(AuthorizedSubject.of("nobody",
"%"),
+ AccessContext.NONE).getUserGroups().isEmpty(), "a user
the store does not know");
+ } finally {
+ controller.close();
+ }
+ }
+ }
+
/**
* Only the checks the engine asks by name map onto a Hive access type.
Anything else - a requirement put
* together for one statement, for instance - maps to one no Hive policy
grants, rather than to a
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/datasource/CatalogFactory.java
b/fe/fe-core/src/main/java/org/apache/doris/datasource/CatalogFactory.java
index cf410288032..13ffe40baa1 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/datasource/CatalogFactory.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/datasource/CatalogFactory.java
@@ -192,7 +192,10 @@ public class CatalogFactory {
catalog.setDefaultPropsIfMissing(false);
catalog.checkWhenCreating();
// This will check if the customized access controller can be
created successfully.
- // If failed, it will throw exception and the catalog will not be
created.
+ // If failed, it will throw exception and the catalog will not be
created. A Ranger source refuses
+ // to be built without its service's policies, so a Ranger admin
that cannot be reached and has
+ // left no policy cache fails the CREATE here, with the cause,
rather than leaving a catalog
+ // behind that refuses every statement.
try {
catalog.initAccessController(true);
} catch (Throwable e) {
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/AccessControllerManager.java
b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/AccessControllerManager.java
index fcb3c0424b8..ce23f2714fe 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/AccessControllerManager.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/AccessControllerManager.java
@@ -788,8 +788,9 @@ public class AccessControllerManager {
if (isDryRun) {
// CREATE CATALOG validates its authorization properties by
building the source and letting it go
// again, which is the only way to find out whether they are
usable at all - the properties are the
- // source's to interpret. It is not free: a Ranger source builds a
real plugin, policy refresher and
- // download timer included. Letting it go does not necessarily
stop it on the spot - a Ranger
+ // source's to interpret - and, for a Ranger source, whether the
service's policies can be had, since
+ // it refuses to be built without them. It is not free: a Ranger
source builds a real plugin, policy
+ // refresher and download timer included. Letting it go does not
necessarily stop it on the spot - a Ranger
// source shares one plugin per service and keeps an unread one
for a grace period, so that the
// CREATE this validated for does not immediately rebuild what it
just tore down - but a
// validation nothing follows does stop polling once that period
is up.
diff --git a/regression-test/data/ranger_p2/test_ranger_group_policy.out
b/regression-test/data/ranger_p2/test_ranger_group_policy.out
new file mode 100644
index 00000000000..2bf4e793c4f
--- /dev/null
+++ b/regression-test/data/ranger_p2/test_ranger_group_policy.out
@@ -0,0 +1,9 @@
+-- This file is automatically generated. You should know what you did if you
want to edit this
+-- !group_select --
+10 XXXXXen10 SampleJ0
+5 XXXXXiv05 SampleE5
+6 XXXXXix06 SampleF6
+7 XXXXXev07 SampleG7
+8 XXXXXig08 SampleH8
+9 XXXXXin09 SampleI9
+
diff --git
a/regression-test/framework/src/main/groovy/org/apache/doris/regression/action/HttpCliAction.groovy
b/regression-test/framework/src/main/groovy/org/apache/doris/regression/action/HttpCliAction.groovy
index b5b88089fb6..8dab41bf87e 100644
---
a/regression-test/framework/src/main/groovy/org/apache/doris/regression/action/HttpCliAction.groovy
+++
b/regression-test/framework/src/main/groovy/org/apache/doris/regression/action/HttpCliAction.groovy
@@ -25,6 +25,7 @@ import org.apache.doris.regression.suite.SuiteContext
import org.apache.http.client.methods.HttpDelete
import org.apache.http.client.methods.HttpGet
import org.apache.http.client.methods.HttpPost
+import org.apache.http.client.methods.HttpPut
import org.apache.http.entity.StringEntity
import org.apache.http.entity.ContentType
import org.apache.http.impl.client.CloseableHttpClient
@@ -152,6 +153,11 @@ class HttpCliAction implements SuiteAction {
HttpDelete httpDelete = new HttpDelete(uri)
headers.each { k, v -> httpDelete.setHeader(k, v) }
result = executeRequest(httpDelete)
+ } else if (op == "put") {
+ HttpPut httpPut = new HttpPut(uri)
+ headers.each { k, v -> httpPut.setHeader(k, v) }
+ httpPut.setEntity(new StringEntity(body,
ContentType.APPLICATION_JSON))
+ result = executeRequest(httpPut)
} else {
HttpPost httpPost = new HttpPost(uri)
headers.each { k, v -> httpPost.setHeader(k, v) }
diff --git a/regression-test/plugins/plugin_ranger.groovy
b/regression-test/plugins/plugin_ranger.groovy
index 77c4286eaee..d0907d51040 100644
--- a/regression-test/plugins/plugin_ranger.groovy
+++ b/regression-test/plugins/plugin_ranger.groovy
@@ -16,6 +16,7 @@
// under the License.
import groovy.json.JsonOutput
+import groovy.json.JsonSlurper
import org.apache.doris.regression.suite.Suite
Suite.metaClass.createRangerUser = { String user, String password, String[]
roles ->
@@ -80,7 +81,81 @@ Suite.metaClass.dropRangerUser = { String userId ->
}
}
+// A group in Ranger's own user store: what a policy item is written against
so that nobody edits the policy
+// each time somebody joins a team. Doris has no groups, so the ranger-doris
source reads a user's groups out of
+// this store; the suites put a user into a group here and write policies
against the group only.
+// Tolerates a group that is already there, the way createRangerUser tolerates
an existing user.
+Suite.metaClass.createRangerGroup = { String group ->
+ def rangerEndpoint = context.config.otherConfigs.get("rangerEndpoint")
+ def rangerUser = context.config.otherConfigs.get("rangerUser")
+ def rangerPassword = context.config.otherConfigs.get("rangerPassword")
+ def js = new JsonOutput().toJson([
+ "name": "${group}",
+ "description": "${group} desc",
+ "groupType": 1,
+ "groupSource": 0,
+ "isVisible": 1])
+ log.info("create group req: ${js} ".toString())
+ httpTest {
+ basicAuthorization "${rangerUser}","${rangerPassword}"
+ endpoint "${rangerEndpoint}"
+ uri "/service/xusers/secure/groups"
+ body js
+ op "post"
+ check { respCode, body ->
+ log.info("create group resp: ${body}
${respCode}".toString())
+ assertTrue(respCode == 200 || body.contains("Error
creating duplicate object"))
+ }
+ }
+}
+
+// Reads one object out of Ranger Admin's user/group API as a map.
+Suite.metaClass.getRangerXObject = { String path ->
+ def rangerEndpoint = context.config.otherConfigs.get("rangerEndpoint")
+ def rangerUser = context.config.otherConfigs.get("rangerUser")
+ def rangerPassword = context.config.otherConfigs.get("rangerPassword")
+ def object = null
+ httpTest {
+ basicAuthorization "${rangerUser}","${rangerPassword}"
+ endpoint "${rangerEndpoint}"
+ header "Accept", "application/json"
+ uri path
+ op "get"
+ check { respCode, body ->
+ assertEquals(200, respCode, "GET ${path}: ${body}")
+ object = new JsonSlurper().parseText(body)
+ }
+ }
+ return object
+}
+
+// Rewrites the groups a Ranger user is in, to exactly `groups`. Goes through
the user update that the
+// Ranger UI uses rather than the group-user mapping API, because in Ranger
Admin 2.4 only the former bumps
+// the user store version - and the version is what tells the plugins there is
a new store to download; a
+// membership changed without it is invisible to every plugin until something
else bumps it.
+Suite.metaClass.setRangerUserGroups = { String user, List<String> groups ->
+ def rangerEndpoint = context.config.otherConfigs.get("rangerEndpoint")
+ def rangerUser = context.config.otherConfigs.get("rangerUser")
+ def rangerPassword = context.config.otherConfigs.get("rangerPassword")
+ def vxUser = getRangerXObject("/service/xusers/users/userName/${user}")
+ vxUser.groupIdList = groups.collect {
getRangerXObject("/service/xusers/groups/groupName/${it}").id }
+ vxUser.groupNameList = groups
+ def js = new JsonOutput().toJson(vxUser)
+ log.info("set user groups req: ${js} ".toString())
+ httpTest {
+ basicAuthorization "${rangerUser}","${rangerPassword}"
+ endpoint "${rangerEndpoint}"
+ uri "/service/xusers/secure/users/${vxUser.id}"
+ body js
+ op "put"
+ check { respCode, body ->
+ log.info("set user groups resp: ${body}
${respCode}".toString())
+ assertEquals(200, respCode)
+ }
+ }
+}
+
Suite.metaClass.waitPolicyEffect {
sleep(6000)
// TODO: check if policy is effective by API
-}
\ No newline at end of file
+}
diff --git a/regression-test/suites/ranger_p2/test_ranger_group_policy.groovy
b/regression-test/suites/ranger_p2/test_ranger_group_policy.groovy
new file mode 100644
index 00000000000..83b7dd316d3
--- /dev/null
+++ b/regression-test/suites/ranger_p2/test_ranger_group_policy.groovy
@@ -0,0 +1,218 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+import org.apache.ranger.RangerClient
+import org.apache.ranger.plugin.model.RangerPolicy
+
+// Policies written against a Ranger GROUP and never against the user: how a
deployment runs Ranger so that
+// nobody edits a policy each time somebody joins a team. Doris has no groups
of its own, so these apply
+// only because the ranger-doris source reads the user's groups out of
Ranger's user store - the store the
+// plugin downloads next to its policies. Every other suite here writes its
policy items against a user.
+suite("test_ranger_group_policy", "p2,ranger,external") {
+ String enabled = context.config.otherConfigs.get("enableRangerTest")
+ String rangerEndpoint =
context.config.otherConfigs.get("rangerEndpoint")
+ String rangerUser = context.config.otherConfigs.get("rangerUser")
+ String rangerPassword =
context.config.otherConfigs.get("rangerPassword")
+ String rangerServiceName =
context.config.otherConfigs.get("rangerServiceName")
+
+ if (enabled == null || !enabled.equalsIgnoreCase("true")) {
+ return
+ }
+
+ String db = 'ranger_group_db_1'
+ String table = 'ranger_group_tbl_1'
+ // The same database, no policy of any kind: what the group's SELECT
must not open.
+ String otherTable = 'ranger_group_tbl_2'
+ String user = 'ranger_group_user_1'
+ String pwd = 'C123_567p'
+ String group = 'ranger_group_readers_1'
+ String accessPolicy = 'ranger_test_group_access_policy'
+ String rowFilterPolicy = 'ranger_test_group_row_filter_policy'
+ String maskPolicy = 'ranger_test_group_mask_policy'
+ String denyPolicy = 'ranger_test_group_deny_policy'
+ // A table the user is allowed on by name while the group is denied on
it; the deny has to win.
+ String deniedTable = 'ranger_group_tbl_3'
+
+ sql """CREATE DATABASE IF NOT EXISTS ${db}"""
+ [table, otherTable, deniedTable].each {
+ sql """DROP TABLE IF EXISTS ${db}.${it}"""
+ sql """CREATE TABLE ${db}.${it} (
+ id BIGINT,
+ c1 VARCHAR(20),
+ c2 VARCHAR(20)
+ )
+ DISTRIBUTED BY HASH(id) BUCKETS 2
+ PROPERTIES (
+ "replication_num" = "1"
+ )"""
+ sql """INSERT INTO ${db}.${it} VALUES
+ (1, 'DataOne01', 'SampleA1'),
+ (2, 'DataTwo02', 'SampleB2'),
+ (3, 'DataThr03', 'SampleC3'),
+ (4, 'DataFou04', 'SampleD4'),
+ (5, 'DataFiv05', 'SampleE5'),
+ (6, 'DataSix06', 'SampleF6'),
+ (7, 'DataSev07', 'SampleG7'),
+ (8, 'DataEig08', 'SampleH8'),
+ (9, 'DataNin09', 'SampleI9'),
+ (10, 'DataTen10', 'SampleJ0')"""
+ }
+ sql """DROP USER IF EXISTS ${user}"""
+ sql """CREATE USER '${user}' IDENTIFIED BY '${pwd}'"""
+
+ // Ranger's side of the same names: the group, the user, and the
membership. Nothing syncs users between
+ // Doris and Ranger, so the names are simply kept equal, as the other
suites do for users.
+ createRangerGroup(group)
+ createRangerUser(user, pwd, ["ROLE_USER"] as String[])
+ setRangerUserGroups(user, [group])
+
+ RangerClient rangerClient = new
RangerClient("http://${rangerEndpoint}";, "simple", rangerUser, rangerPassword,
null)
+ def dropPolicy = { String name ->
+ try {
+ rangerClient.deletePolicy(rangerServiceName, name)
+ } catch (Exception e) {
+ log.info("Policy not found: ${e.getMessage()}")
+ }
+ }
+ [accessPolicy, rowFilterPolicy, maskPolicy, denyPolicy].each {
dropPolicy(it) }
+
+ def resourcesOf = { String tbl, String column ->
+ Map<String, RangerPolicy.RangerPolicyResource> resources = new
HashMap<>()
+ resources.put("catalog", new
RangerPolicy.RangerPolicyResource("internal"))
+ resources.put("database", new
RangerPolicy.RangerPolicyResource(db))
+ resources.put("table", new
RangerPolicy.RangerPolicyResource(tbl))
+ if (column != null) {
+ resources.put("column", new
RangerPolicy.RangerPolicyResource(column))
+ }
+ return resources
+ }
+ def select = [new RangerPolicy.RangerPolicyItemAccess("SELECT")]
+
+ // Four policies, four Ranger Admin calls, and the FE polls for
policies every few seconds: a poll
+ // between two of the calls brings a generation with only the first of
them. So the access policy - the
+ // one the wait below watches for - is written last: neither a row
filter nor a mask nor a deny grants
+ // anything, and once the FE answers on the access policy it holds the
generation the other three are in.
+
+ // 1. A row filter for the group.
+ RangerPolicy policy = new RangerPolicy()
+ policy.setService(rangerServiceName)
+ policy.setName(rowFilterPolicy)
+ policy.setPolicyType(RangerPolicy.POLICY_TYPE_ROWFILTER)
+ policy.setResources(resourcesOf(table, null))
+ RangerPolicy.RangerRowFilterPolicyItem rowFilterItem = new
RangerPolicy.RangerRowFilterPolicyItem()
+ rowFilterItem.setGroups([group])
+ rowFilterItem.setAccesses(select)
+ rowFilterItem.setRowFilterInfo(new
RangerPolicy.RangerPolicyItemRowFilterInfo("id >= 5"))
+ policy.setRowFilterPolicyItems([rowFilterItem])
+ rangerClient.createPolicy(policy)
+
+ // 2. A column mask for the group.
+ policy = new RangerPolicy()
+ policy.setService(rangerServiceName)
+ policy.setName(maskPolicy)
+ policy.setPolicyType(RangerPolicy.POLICY_TYPE_DATAMASK)
+ policy.setResources(resourcesOf(table, "c1"))
+ RangerPolicy.RangerDataMaskPolicyItem maskItem = new
RangerPolicy.RangerDataMaskPolicyItem()
+ maskItem.setGroups([group])
+ maskItem.setAccesses(select)
+ maskItem.setDataMaskInfo(new
RangerPolicy.RangerPolicyItemDataMaskInfo("MASK_SHOW_LAST_4", null, null))
+ policy.setDataMaskPolicyItems([maskItem])
+ rangerClient.createPolicy(policy)
+
+ // 3. A deny written against the group, on a table the user is allowed
on by name. Before groups were
+ // attached this deny was silently ignored and the user read the table
- the dangerous half of the bug.
+ policy = new RangerPolicy()
+ policy.setService(rangerServiceName)
+ policy.setName(denyPolicy)
+ policy.setResources(resourcesOf(deniedTable, null))
+ RangerPolicy.RangerPolicyItem allowUserItem = new
RangerPolicy.RangerPolicyItem()
+ allowUserItem.setUsers([user])
+ allowUserItem.setAccesses(select)
+ policy.setPolicyItems([allowUserItem])
+ RangerPolicy.RangerPolicyItem denyGroupItem = new
RangerPolicy.RangerPolicyItem()
+ denyGroupItem.setGroups([group])
+ denyGroupItem.setAccesses(select)
+ policy.setDenyPolicyItems([denyGroupItem])
+ rangerClient.createPolicy(policy)
+
+ // 4. Access: SELECT on the table for the group. Last, see above.
+ policy = new RangerPolicy()
+ policy.setService(rangerServiceName)
+ policy.setName(accessPolicy)
+ policy.setResources(resourcesOf(table, null))
+ RangerPolicy.RangerPolicyItem accessItem = new
RangerPolicy.RangerPolicyItem()
+ accessItem.setGroups([group])
+ accessItem.setAccesses(select)
+ policy.setPolicyItems([accessItem])
+ rangerClient.createPolicy(policy)
+
+ def tokens = context.config.jdbcUrl.split('/')
+ def defaultJdbcUrl = tokens[0] + "//" + tokens[2] + "/?"
+ def readable = { String tbl ->
+ return connect("${user}", "${pwd}", "${defaultJdbcUrl}") {
+ try {
+ sql """SELECT * FROM internal.${db}.${tbl}"""
+ return true
+ } catch (Exception e) {
+ log.info("not readable yet: ${e.getMessage()}")
+ return false
+ }
+ }
+ }
+
+ // The policies reach the FE within its policy poll interval; the
membership reaches it with the next
+ // user store download, which the plugin makes every 60 seconds unless
userStoreRefresherPollingInterval
+ // in ranger-doris-security.xml says otherwise. Hence waiting on the
effect rather than a fixed sleep -
+ // and on the policy written last, which proves the whole set is there
(see above).
+ logger.info("waiting for the group's SELECT to reach ${user}")
+ awaitUntil(180, 3) { readable(table) }
+
+ connect("${user}", "${pwd}", "${defaultJdbcUrl}") {
+ // Visible on the strength of the group's policy alone.
+ def databases = sql """SHOW DATABASES"""
+ assertTrue(databases.any { it[0] == db }, "the database the
group may read is not listed")
+ sql """SWITCH internal"""
+ sql """USE ${db}"""
+
+ // Rows 5..10 only, c1 masked down to its last four characters:
the row filter and the mask written
+ // against the group both apply, as do those written against a
user.
+ order_qt_group_select """SELECT * FROM
internal.${db}.${table}"""
+
+ // The group holds SELECT on one table, so the group grants
nothing on the next one.
+ test {
+ sql """SELECT * FROM internal.${db}.${otherTable}"""
+ exception "denied"
+ }
+
+ // The deny written against the group outranks the allow
written against the user.
+ test {
+ sql """SELECT * FROM internal.${db}.${deniedTable}"""
+ exception "denied"
+ }
+ }
+
+ // Taking the user out of the group takes the access with it, once the
plugin has downloaded the change.
+ // Membership is Ranger's alone to keep, which is the whole reason a
deployment grants by group.
+ setRangerUserGroups(user, [])
+ logger.info("waiting for ${user} to lose the group's SELECT")
+ awaitUntil(180, 3) { !readable(table) }
+
+ // And the deny written against the group no longer applies either: the
user's own allow reads the table.
+ awaitUntil(180, 3) { readable(deniedTable) }
+
+ [accessPolicy, rowFilterPolicy, maskPolicy, denyPolicy].each {
dropPolicy(it) }
+}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
