Gabriel39 opened a new pull request, #69608:
URL: https://github.com/apache/doris/pull/69608
### What problem does this PR solve?
Issue Number: N/A
Problem Summary:
The external regression run `Doris_x64_Master_Doris_Regression_External
#1122` accessed Docker Kerberos catalogs before EMR catalogs in the same FE
process. Three suites created and switched their catalogs successfully, then
failed at line 43 (`USE`) while initializing external metadata:
| Failed suite | Catalog path | Failure point |
| --- | --- | --- |
| `test_hive_migrate_partition_iceberg` | Iceberg HMS | Build the Iceberg
catalog |
| `test_hive_migrate_iceberg` | Iceberg HMS | Build the Iceberg catalog |
| `test_orc_merge_io_input_streams` | Hive HMS | Create the HMS client |
All three FE stacks end in `getUGIFromSubject -> HadoopLoginModule.commit ->
KerberosName.getShortName -> NoMatchingRule`. Their catalog properties
contained explicit rules for the EMR principal, but the shared Hadoop mapping
state did not match it. Each suite aborted before its migration-table or ORC
queries ran.
Hadoop UGI and `KerberosName` are shared across FE connector plugins, while
the old `configuredAuthMethod` flag belonged to each loaded authenticator
class. After that class's first initialization, later calls skipped
configuration updates entirely, including valid `auth_to_local` rules; the
authentication-method comparison only controlled a warning. A different
plugin's first initialization could also replace the shared rules. This
explains both later-realm failures and initialization-order dependence. The
logs establish the failure location and suite order; controlled tests reproduce
both mechanisms without assuming which catalog first published the global rules.
The same run's later `test_hive_migrate_paimon` succeeded with the EMR
configuration. This is consistent with a separate plugin's first initialization
replacing shared state, rather than demonstrating that the failed suites had
invalid credentials.
The fix:
- Coordinate initialization and UGI construction on the shared
`UserGroupInformation.class` monitor, using actual Hadoop security state
instead of a per-plugin initialization flag.
- Append compatible explicit Hadoop mapping rules after existing rules,
including `DEFAULT`, preserving existing first-match precedence. Canonicalize
and deduplicate rules so repeated ticket refreshes do not accumulate copies.
- Validate mapping syntax, regular expressions, and mechanisms before
initializing UGI. Reject incompatible mechanisms and MIT rule extensions
without publishing changes: MIT accepts unmatched principals unchanged, so
extending its rules could rename existing users.
- Route the filesystem authenticator through the same initialization logic
and protect its initial and refreshed UGI construction with the same monitor.
JAAS/keytab login remains outside the shared monitor.
Mapping policy remains shared within each Hadoop runtime. Existing mappings
take precedence over overlapping later rules; independent conflicting policies
require separate Hadoop runtimes or a common policy configured from startup.
### Release note
Fix `NoMatchingRule` failures when Kerberos external catalogs for different
realms share an FE Hadoop runtime.
### Check List (For Author)
- Test
- [x] Unit Test: related Kerberos and HDFS authenticator suites,
including multiple realms in both orders, independent plugin classloaders,
concurrent initialization/refresh, rule deduplication, filesystem ticket
refresh, and invalid configurations. 61 tests passed with no failures or errors
(26 in fe-kerberos, 30 in hdfs-base, and 5 existing HDFS authenticator tests).
- [x] Manual test: replay the original Docker/EMR mapping configurations
against Hadoop JARs from the failing build with the modified authenticator;
both realm sequences and separate plugin copies succeed.
- FE Checkstyle: full reactor and affected modules.
- The complete external regression pipeline and environment-dependent
KDC/HDFS integration tests were not run locally.
- Behavior changed:
- [x] Yes. Later compatible realm rules are retained alongside earlier
mappings; incompatible rule mechanisms fail explicitly.
- Does this need documentation?
- [x] No. No new catalog properties or configuration options.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
