[
https://issues.apache.org/jira/browse/KNOX-2956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sandor Molnar updated KNOX-2956:
--------------------------------
Description:
Knox's Hadoop XML resource parser is tightly coupled with another feature
called Advanced Service Discovery configuration in Cloudera Manager.
There are several issues with that extension:
- makes the code much harder to read, understand, and maintain
- occupies a separate thread to monitor other files (we already have many file
watchers, it's always good if we can do some cleanup)
- One should really oversee the correlation between them and make the right
decision when touching one or the other (for instance, when changing the
ordering of these services)
- Since this is CM specific, lots of properties were added in the relevant
Knox [CSD
files|https://github.com/cloudera/cm_ext/wiki/Service-Descriptor-Language-Reference]
to give the flexibility for our users to enable/disable services during CM
service discovery. The management of those configurations is way too complex
and has a really negative effect on user experience on Knox's configuration
page within Cloudera Manager
Therefore, I came up with an idea that will still allow us to keep the original
idea of excluding/including certain services to be discovered/included in the
generated topology files. I plan to implement the following:
- Remove the entire {{AdvancedServiceDiscoveryConfig*}} code
- Former {{gateway.auto.discovery.address}} and
{{gateway.auto.discovery.cluster}} parameters are already taken care of in HXR
parser where descriptors are handled (they need to be set in upstream
configuration locations such as the Knox CSD)
- By default, all services are disabled even if a service available service
found in the given discovery address/cluster will be added to the descriptor.
This is because of the nature of the existing logic in
{{{}SimpleDescriptorHandler{}}}. I'll add a new parameter suffix for service
called "{{{}services{}}}" which end-users can set to "{{{}a comma-separated
list of services"{}}} to include services in the generated topology (this new
HXR parameter is similar to the existing 'discoveryAddress' or
'providerConfigRef' configs)
- since this is CM-specific, Cloudera Manager users need to make sure to adopt
their CSD files accordingly
As a result, the {{.hxr}} file(s) will be self-contained and can achieve the
same functionality as we have now with the complementary
{{auto-discovery-advanced-configuration-*}} files.
was:
Knox's Hadoop XML resource parser is tightly coupled with another feature
called Advanced Service Discovery configuration in Cloudera Manager.
There are several issues with that extension:
- makes the code much harder to read, understand, and maintain
- occupies a separate thread to monitor other files (we already have many file
watchers, it's always good if we can do some cleanup)
- One should really oversee the correlation between them and make the right
decision when touching one or the other (for instance, when changing the
ordering of these services)
- Since this is CM specific, lots of properties were added in the relevant
Knox [CSD
files|https://github.com/cloudera/cm_ext/wiki/Service-Descriptor-Language-Reference]
to give the flexibility for our users to enable/disable services during CM
service discovery. The management of those configurations is way too complex
and has a really negative effect on user experience on Knox's configuration
page within Cloudera Manager
Therefore, I came up with an idea that will still allow us to keep the original
idea of excluding/including certain services to be discovered/included in the
generated topology files. I plan to implement the following:
- Remove the entire {{AdvancedServiceDiscoveryConfig*}} code
- Former {{gateway.auto.discovery.address}} and
{{gateway.auto.discovery.cluster}} parameters are already taken care of in HXR
parser where descriptors are handled (they need to be set in upstream
configuration locations such as the Knox CSD)
- By default, all services are enabled, thus every available service found in
the given discovery address/cluster will be added to the descriptor. I'll add a
new parameter suffix for service called "{{{}exclude{}}}" which end-users can
set to {{true}} if they want to exclude a particular service from the generated
topology (this new service parameter suffix is similar to the existing 'url' or
'version' suffixes)
As a result, the {{.hxr}} file(s) will be self-contained and can achieve the
same functionality as we have now with the complementary
{{auto-discovery-advanced-configuration-*}} files.
> Refactor CM-specific advanced service discovery
> -----------------------------------------------
>
> Key: KNOX-2956
> URL: https://issues.apache.org/jira/browse/KNOX-2956
> Project: Apache Knox
> Issue Type: Bug
> Affects Versions: 2.0.0, 1.6.0, 1.6.1
> Reporter: Sandor Molnar
> Assignee: Sandor Molnar
> Priority: Critical
> Fix For: 2.1.0
>
>
> Knox's Hadoop XML resource parser is tightly coupled with another feature
> called Advanced Service Discovery configuration in Cloudera Manager.
> There are several issues with that extension:
> - makes the code much harder to read, understand, and maintain
> - occupies a separate thread to monitor other files (we already have many
> file watchers, it's always good if we can do some cleanup)
> - One should really oversee the correlation between them and make the right
> decision when touching one or the other (for instance, when changing the
> ordering of these services)
> - Since this is CM specific, lots of properties were added in the relevant
> Knox [CSD
> files|https://github.com/cloudera/cm_ext/wiki/Service-Descriptor-Language-Reference]
> to give the flexibility for our users to enable/disable services during CM
> service discovery. The management of those configurations is way too complex
> and has a really negative effect on user experience on Knox's configuration
> page within Cloudera Manager
> Therefore, I came up with an idea that will still allow us to keep the
> original idea of excluding/including certain services to be
> discovered/included in the generated topology files. I plan to implement the
> following:
> - Remove the entire {{AdvancedServiceDiscoveryConfig*}} code
> - Former {{gateway.auto.discovery.address}} and
> {{gateway.auto.discovery.cluster}} parameters are already taken care of in
> HXR parser where descriptors are handled (they need to be set in upstream
> configuration locations such as the Knox CSD)
> - By default, all services are disabled even if a service available service
> found in the given discovery address/cluster will be added to the descriptor.
> This is because of the nature of the existing logic in
> {{{}SimpleDescriptorHandler{}}}. I'll add a new parameter suffix for service
> called "{{{}services{}}}" which end-users can set to "{{{}a comma-separated
> list of services"{}}} to include services in the generated topology (this new
> HXR parameter is similar to the existing 'discoveryAddress' or
> 'providerConfigRef' configs)
> - since this is CM-specific, Cloudera Manager users need to make sure to
> adopt their CSD files accordingly
> As a result, the {{.hxr}} file(s) will be self-contained and can achieve the
> same functionality as we have now with the complementary
> {{auto-discovery-advanced-configuration-*}} files.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
