amoghrajesh commented on code in PR #63826:
URL: https://github.com/apache/airflow/pull/63826#discussion_r3015894862
##########
providers/apache/hive/provider.yaml:
##########
@@ -178,8 +179,10 @@ connection-types:
- 'null'
default: false
- hook-class-name: airflow.providers.apache.hive.hooks.hive.HiveServer2Hook
+ hook-name: "Hive Client Wrapper"
connection-type: hiveserver2
- hook-class-name: airflow.providers.apache.hive.hooks.hive.HiveMetastoreHook
+ hook-name: "Hive Client Wrapper"
connection-type: hive_metastore
Review Comment:
Earlier we seemed to use connection types, and that's why it remained
distinct, maybe do that instead
##########
airflow-core/src/airflow/api_fastapi/core_api/services/ui/connections.py:
##########
@@ -222,19 +223,39 @@ def _convert_extra_fields(form_widgets: dict[str,
ConnectionFormWidgetInfo]) ->
@staticmethod
@cache
def hook_meta_data() -> list[ConnectionHookMetaData]:
- hooks, connection_form_widgets, field_behaviours =
HookMetaService._get_hooks_with_mocked_fab()
+ from airflow.providers_manager import ProvidersManager
+
+ pm = ProvidersManager()
+ pm.initialize_providers_hooks()
+
+ widgets =
HookMetaService._convert_extra_fields(pm._connection_form_widgets)
result: list[ConnectionHookMetaData] = []
- widgets =
HookMetaService._convert_extra_fields(connection_form_widgets)
- for hook_key, hook_info in hooks.items():
- if not hook_info:
- continue
- hook_meta = ConnectionHookMetaData(
- connection_type=hook_key,
- hook_class_name=hook_info.hook_class_name,
- default_conn_name=None, # TODO: later
- hook_name=hook_info.hook_name,
-
standard_fields=HookMetaService._make_standard_fields(field_behaviours.get(hook_key)),
- extra_fields=widgets.get(hook_key),
+
+ all_conn_types = set(pm._hooks_lazy_dict) | set(pm._hook_provider_dict)
+ for conn_type in sorted(all_conn_types):
+ raw_entry = pm._hooks_lazy_dict._raw_dict.get(conn_type)
+ provider_entry = pm._hook_provider_dict.get(conn_type)
+
+ if isinstance(raw_entry, HookInfo):
+ hook_name = raw_entry.hook_name
+ hook_class_name = raw_entry.hook_class_name
+ elif provider_entry:
+ hook_name = pm._hook_name_dict.get(conn_type, conn_type)
+ hook_class_name = provider_entry.hook_class_name
+ else:
+ hook_name = pm._hook_name_dict.get(conn_type, conn_type)
+ hook_class_name = None
+
+ result.append(
+ ConnectionHookMetaData(
+ connection_type=conn_type,
+ hook_class_name=hook_class_name,
+ default_conn_name=None,
+ hook_name=hook_name,
+ standard_fields=HookMetaService._make_standard_fields(
+ pm._field_behaviours.get(conn_type)
+ ),
+ extra_fields=widgets.get(conn_type),
+ )
Review Comment:
We do a lot of private attribute access here, so I wonder if some / all of
this can be pushed into public method(s)? Maybe call it
`iter_connection_type_ui_placeholder`? Maybe even consider pushing
`_connection_form_widgets` and `_field_behaviours` into properties as well?
This is mainly to ensure that it remains public interface and doesnt make this
code brittle.
##########
providers/apache/hive/provider.yaml:
##########
@@ -178,8 +179,10 @@ connection-types:
- 'null'
default: false
- hook-class-name: airflow.providers.apache.hive.hooks.hive.HiveServer2Hook
+ hook-name: "Hive Client Wrapper"
connection-type: hiveserver2
- hook-class-name: airflow.providers.apache.hive.hooks.hive.HiveMetastoreHook
+ hook-name: "Hive Client Wrapper"
connection-type: hive_metastore
Review Comment:
This doesn't look too right to me?
##########
airflow-core/src/airflow/api_fastapi/core_api/services/ui/connections.py:
##########
@@ -222,19 +223,39 @@ def _convert_extra_fields(form_widgets: dict[str,
ConnectionFormWidgetInfo]) ->
@staticmethod
@cache
def hook_meta_data() -> list[ConnectionHookMetaData]:
- hooks, connection_form_widgets, field_behaviours =
HookMetaService._get_hooks_with_mocked_fab()
+ from airflow.providers_manager import ProvidersManager
Review Comment:
Can we import at top level please?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
