HonahX commented on code in PR #1600: URL: https://github.com/apache/polaris/pull/1600#discussion_r2096538340
########## site/content/in-dev/unreleased/policy.md: ########## @@ -0,0 +1,207 @@ +--- +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +# +title: Policy +type: docs +weight: 425 +--- + +The Polaris Policy framework empowers organizations to centrally define, manage, and enforce fine-grained governance, lifecycle, and operational rules across all data resources in the catalog. + +With the policy API, you can: +- Create and manage policies +- Attach policies to specific resources (catalogs, namespaces, tables, or views) +- Check applicable policies for any given resource + +## What is a Policy? + +A policy in Apache Polaris is a structured entity that defines rules governing actions on specified resources under +predefined conditions. Each policy contains: + +- **Name**: A unique identifier within a namespace +- **Type**: Determines the semantics and expected format of the policy content +- **Description**: Explains the purpose of the policy +- **Content**: Contains the actual rules defining the policy behavior +- **Version**: An automatically tracked revision number +- **Inheritable**: Whether the policy can be inherited by child resources, decided by its type + +### Policy Types + +Polaris supports several predefined system policy types (prefixed with `system.`): + +- **`system.data-compaction`**: Defines rules for data compaction operations + - Schema Definition: @https://polaris.apache.org/schemas/policies/system/data-compaction/2025-02-03.json + - Controls file compaction to optimize storage and query performance + +- **`system.metadata-compaction`**: Defines rules for metadata compaction operations + - Schema Definition: @https://polaris.apache.org/schemas/policies/system/metadata-compaction/2025-02-03.json + - Optimizes table metadata for improved performance + - Applicable resources: Iceberg table, namespace, catalog + +- **`system.orphan-file-removal`**: Defines rules for removing orphaned files + - Schema Definition: @https://polaris.apache.org/schemas/policies/system/orphan-file-removal/2025-02-03.json + - Identifies and safely removes files that are no longer referenced by the table metadata + - Applicable resources: Iceberg table, namespace, catalog + +- **`system.snapshot-expiry`**: Defines rules for snapshot expiration + - Schema Definition: @https://polaris.apache.org/schemas/policies/system/snapshot-expiry/2025-02-03.json + - Controls how long snapshots are retained before removal + - Applicable resources: Iceberg table, namespace, catalog + +- **Custom policy types**: Can be defined for specific organizational needs (WIP) + +- **FGAC (Fine-Grained Access Control) policies**: Row filtering, column masking, column hiding (WIP) Review Comment: IMHO, the site contains only the features/concepts that has been finalized and implemented, but I also that mentioning these features here can help form a better understanding of the general scope of policy. Should we phrase this more generally, such as: 'Support for additional predefined system policy types and custom policy type definitions is in progress. For more details, please refer to the roadmap.' This way, we avoid over-sharing implementation details while still giving users a clear sense of the feature scope. ########## site/content/in-dev/unreleased/policy.md: ########## @@ -0,0 +1,207 @@ +--- +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +# +title: Policy +type: docs +weight: 425 +--- + +The Polaris Policy framework empowers organizations to centrally define, manage, and enforce fine-grained governance, lifecycle, and operational rules across all data resources in the catalog. + +With the policy API, you can: +- Create and manage policies +- Attach policies to specific resources (catalogs, namespaces, tables, or views) +- Check applicable policies for any given resource + +## What is a Policy? + +A policy in Apache Polaris is a structured entity that defines rules governing actions on specified resources under +predefined conditions. Each policy contains: + +- **Name**: A unique identifier within a namespace +- **Type**: Determines the semantics and expected format of the policy content +- **Description**: Explains the purpose of the policy +- **Content**: Contains the actual rules defining the policy behavior +- **Version**: An automatically tracked revision number +- **Inheritable**: Whether the policy can be inherited by child resources, decided by its type + +### Policy Types + +Polaris supports several predefined system policy types (prefixed with `system.`): + +- **`system.data-compaction`**: Defines rules for data compaction operations + - Schema Definition: @https://polaris.apache.org/schemas/policies/system/data-compaction/2025-02-03.json + - Controls file compaction to optimize storage and query performance + +- **`system.metadata-compaction`**: Defines rules for metadata compaction operations + - Schema Definition: @https://polaris.apache.org/schemas/policies/system/metadata-compaction/2025-02-03.json + - Optimizes table metadata for improved performance + - Applicable resources: Iceberg table, namespace, catalog + +- **`system.orphan-file-removal`**: Defines rules for removing orphaned files + - Schema Definition: @https://polaris.apache.org/schemas/policies/system/orphan-file-removal/2025-02-03.json + - Identifies and safely removes files that are no longer referenced by the table metadata + - Applicable resources: Iceberg table, namespace, catalog + +- **`system.snapshot-expiry`**: Defines rules for snapshot expiration + - Schema Definition: @https://polaris.apache.org/schemas/policies/system/snapshot-expiry/2025-02-03.json + - Controls how long snapshots are retained before removal + - Applicable resources: Iceberg table, namespace, catalog + +- **Custom policy types**: Can be defined for specific organizational needs (WIP) + +- **FGAC (Fine-Grained Access Control) policies**: Row filtering, column masking, column hiding (WIP) + +### Policy Inheritance + +The entity hierarchy in Polaris is structured as follows: + +``` + Catalog + | + Namespace + | + +-----------+----------+ + | | | +Iceberg Iceberg Generic + Table View Table +``` + +Policies can be attached at any level, and inheritance flows from catalog down to namespace, then to tables and views. Review Comment: For now we only allow attach policy to tables. Shall we indicate that in the doc? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
