This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new 8bad9a9a992 [doc](dbt) Update adapter guide for v1.1.1 (#4229)
8bad9a9a992 is described below
commit 8bad9a9a9922364e218c73aab145b521fd884334
Author: Mingyu Chen (Rayner) <[email protected]>
AuthorDate: Tue Oct 13 18:37:02 2026 +0800
[doc](dbt) Update adapter guide for v1.1.1 (#4229)
## Summary
- Update the dbt Adapter guide for `dbt-for-apache-doris` v1.1.1,
including installation, dbt Core compatibility, and the new `database`
to Catalog mapping.
- Document External Catalog sources, Iceberg targets and incremental
strategies, unsupported database constraints, release limitations, and
runnable demos.
- Rename the page, sidebar label, and integration card to `dbt Adapter`
across current and 4.x English and Chinese docs.
## Validation
- `git diff --check`
- Changed-file link lint: no broken internal links (external URLs are
reported as informational).
- Changed-file feature-doc and i18n-sync lint: no findings.
- Changed-file SEO lint: warnings for duplicate titles across mirrored
versions/locales and the existing short Chinese integration-index
description; no errors.
Release reference:
https://github.com/velodb/dbt-for-apache-doris/releases/tag/v1.1.1
Co-authored-by: morningman <[email protected]>
---
.../data-integration/dbt-doris-adapter.md | 120 ++++++++++++------
.../data-integration/intro.mdx | 4 +-
.../data-integration/dbt-doris-adapter.md | 134 +++++++++++++--------
.../data-integration/intro.mdx | 4 +-
.../data-integration/dbt-doris-adapter.md | 134 +++++++++++++--------
.../data-integration/intro.mdx | 4 +-
.../data-integration/dbt-doris-adapter.md | 120 ++++++++++++------
.../data-integration/intro.mdx | 4 +-
8 files changed, 332 insertions(+), 192 deletions(-)
diff --git a/docs/connection-integration/data-integration/dbt-doris-adapter.md
b/docs/connection-integration/data-integration/dbt-doris-adapter.md
index 9e7414445a9..205cd1dc01a 100644
--- a/docs/connection-integration/data-integration/dbt-doris-adapter.md
+++ b/docs/connection-integration/data-integration/dbt-doris-adapter.md
@@ -1,8 +1,9 @@
---
{
- "title": "Apache Doris dbt Adapter",
+ "title": "dbt Adapter",
+ "sidebar_label": "dbt Adapter",
"language": "en",
- "description": "Install and configure dbt-for-apache-doris for Apache
Doris models, source freshness, and asynchronous materialized views.",
+ "description": "Install dbt-for-apache-doris 1.1.1 and configure models,
External Catalog sources, Iceberg targets, and asynchronous materialized
views.",
"keywords": [
"Apache Doris dbt",
"dbt Doris Adapter",
@@ -41,11 +42,13 @@ Source data must already be loaded into Doris. The adapter
performs transformati
| Component | Requirement |
| --- | --- |
| Python | 3.10 or later |
+| dbt Core | 1.12.x (installed with the adapter) |
## Released versions
| Version |
| --- |
+| [1.1.1](https://github.com/velodb/dbt-for-apache-doris/releases/tag/v1.1.1) |
| [1.1.0](https://github.com/velodb/dbt-for-apache-doris/releases/tag/v1.1.0) |
<!-- Knowledge Type: Operational Steps -->
@@ -53,7 +56,7 @@ Source data must already be loaded into Doris. The adapter
performs transformati
## Installation
-The following commands use the published [`v1.1.0`
release](https://github.com/velodb/dbt-for-apache-doris/releases/tag/v1.1.0) as
an installation example. To install another release, replace the version in the
`pip` requirement.
+The following commands use the published [`v1.1.1`
release](https://github.com/velodb/dbt-for-apache-doris/releases/tag/v1.1.1) as
an installation example. To install another release, replace the version in the
`pip` requirement.
Install the adapter from PyPI in an isolated Python environment:
@@ -61,7 +64,7 @@ Install the adapter from PyPI in an isolated Python
environment:
python3 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
-python -m pip install "dbt-for-apache-doris==1.1.0"
+python -m pip install "dbt-for-apache-doris==1.1.1"
dbt --version
```
@@ -71,11 +74,11 @@ On Windows PowerShell:
py -m venv .venv
.\.venv\Scripts\Activate.ps1
python -m pip install --upgrade pip
-python -m pip install "dbt-for-apache-doris==1.1.0"
+python -m pip install "dbt-for-apache-doris==1.1.1"
dbt --version
```
-For this example, the `Plugins` section of `dbt --version` should contain
`doris: 1.1.0`. dbt Core and MySQL Connector/Python are installed as
dependencies.
+For this example, the `Plugins` section of `dbt --version` should contain
`doris: 1.1.1`. dbt Core and MySQL Connector/Python are installed as
dependencies.
<!-- Knowledge Type: Operational Steps / Minimal Example -->
<!-- Use Case: Connect dbt to Doris / Validate an end-to-end model build -->
@@ -114,17 +117,9 @@ doris_demo:
| `password` | Read from an environment variable | Defaults to an empty string
|
| `schema` | Set the target Doris database | The `dbt init` prompt defaults to
`dbt`; the underlying credentials default to `None` |
| `threads` | Set according to FE and workload capacity | The `dbt init`
prompt defaults to `1` |
-| `database` | Omit it | If set, it must exactly equal `schema` |
+| `database` | Omit for Internal Catalog; set to an existing Doris Catalog for
an external target | Maps to the Doris Catalog in v1.1.1 |
-The adapter maps a dbt schema to a Doris database. If the target database does
not exist, the adapter attempts to create it. The Doris account used by dbt
therefore needs permission to read source tables and to create, alter, drop,
and write objects in the target database.
-
-Here, `database` is not a Doris catalog. It is a field inherited from dbt's
generic `database.schema.identifier` naming model. Doris Internal Catalog uses
only `database.table`, so set the Doris database in profile `schema` and omit
profile `database`. Source `database` has a different compatibility behavior:
it can override `schema` as the Doris database for cross-database reads.
Profiles and standard `source()` cannot represent both a Doris catalog and
database level.
-
-:::caution
-
-The adapter currently does not support External Catalog three-part namespaces
using Catalog, Database, and Table.
-
-:::
+In v1.1.1, dbt `database.schema.identifier` maps to Doris
`catalog.database.table`. For Internal Catalog, omit `database` and set
`schema` to the Doris database. For an external target, set `database` to an
existing Doris Catalog and `schema` to a database in that Catalog. The adapter
can attempt to create a missing target database if the Catalog supports it. The
dbt account needs permission to read sources and to create, alter, drop, and
write target objects. External Catalog write supp [...]
Ensure that `dbt_project.yml` references the same profile:
@@ -253,7 +248,52 @@ sources:
- name: payments
```
-For compatibility with existing dbt projects, a source may instead set only
`database: finance_raw`, or set `database` and `schema` to the same value. If
source `database` and `schema` differ, the adapter uses `database` as the Doris
database. This differs from profile validation, which requires both values to
match. New projects should consistently use source `schema` to avoid ambiguity.
+In v1.1.1, a source's `database` names a Doris Catalog. To read from another
database in Internal Catalog, set only `schema`, as above. Projects that used
`database: finance_raw` for cross-database sources in v1.1.0 must move that
value to `schema` before upgrading.
+
+## External Catalogs in v1.1.1
+
+The adapter can read sources in External Catalogs and discover their tables
and columns for `dbt docs generate`. Create the Catalog in Doris first, then
specify its name in the source's `database` field:
+
+```yaml
+sources:
+ - name: lakehouse
+ database: hive_catalog
+ schema: ods
+ tables:
+ - name: orders
+```
+
+`{{ source('lakehouse', 'orders') }}` resolves to `hive_catalog.ods.orders`.
The `schema` field names the database inside that Catalog. The adapter supports
catalog-qualified relation names; each Catalog connector still determines which
DDL and writes Doris can perform.
+
+To create an Iceberg target, first [configure a writable Iceberg
Catalog](../../lakehouse/catalogs/iceberg-catalog). Set these fields in the
existing profile output:
+
+```yaml
+database: iceberg_catalog
+schema: analytics
+```
+
+Create `models/iceberg_sample.sql`:
+
+```sql
+{{ config(materialized='table', engine='iceberg') }}
+
+select 1 as id
+```
+
+Run `dbt run --select iceberg_sample`. It creates
`iceberg_catalog.analytics.iceberg_sample`. The Catalog must exist; its
database must exist or be creatable by dbt, and the target must be writable.
`engine='iceberg'` emits an explicit engine clause; omitting `engine` lets
Doris infer it from the target Catalog. Catalog-based inference was verified
with Doris 4.1.3. Earlier Doris releases may require the explicit engine.
+
+| Iceberg target | v1.1.1 behavior |
+| --- | --- |
+| `table` | Creates an Iceberg table; reruns replace it through backup and
rename |
+| `incremental` | Supports `append`, `merge`, `insert_overwrite`, and
`microbatch` as described below |
+| Seed and Snapshot | Supported; Snapshot publication and Seed full refresh
use backup and rename. Ordinary Iceberg Seed reload uses overwrite or MERGE |
+| `view` and `materialized_view` | Targets must be in Internal Catalog;
Internal Catalog views and MVs can query External Catalog sources |
+
+Iceberg `merge` and `microbatch` require a Doris release with native Iceberg
`MERGE INTO` and an Iceberg V2 or later table. These paths were verified on
Doris 4.1.3; [Doris documents Iceberg MERGE as experimental since
4.1.0](../../lakehouse/catalogs/iceberg-catalog#merge-into). An external table
replacement is not atomic: its name is briefly unavailable between renames, and
a failure can leave the old table under a backup name. Use an isolated test
project to validate your Doris release [...]
+
+The adapter rejects model or column `constraints` declarations before writing
a Table, View, Incremental, or MV target. Model contracts still check column
names and types. Use `data_tests` such as `not_null` to check data after
writing; these do not create database-enforced constraints.
+
+The [five runnable Doris
demos](https://github.com/velodb/dbt-for-apache-doris/tree/v1.1.1/examples/doris-demos)
cover models, tests, cross-database sources, incremental processing,
snapshots, and asynchronous materialized views.
<!-- Knowledge Type: Capability Reference / Configuration Parameters /
Architecture Decision -->
<!-- Use Case: Materialization selection / Incremental processing /
Materialized views -->
@@ -264,10 +304,10 @@ For compatibility with existing dbt projects, a source
may instead set only `dat
| Materialization | Doris object or behavior | Use case |
| --- | --- | --- |
-| `view` | Doris view | Lightweight transformations that should always read
current source data |
-| `table` | A fully rebuilt Doris Duplicate Key table | Bounded results that
need stable query performance |
+| `view` | Internal Catalog Doris view | Lightweight transformations that
should always read current source data |
+| `table` | A fully rebuilt Internal Catalog Duplicate Key table or writable
Iceberg table | Bounded results that need stable query performance |
| `incremental` | Appends, upserts, or overwrites data according to a strategy
| Incremental processing for large tables |
-| `materialized_view` | Doris asynchronous materialized view | Doris-managed
refresh and transparent query rewrite |
+| `materialized_view` | Internal Catalog Doris asynchronous materialized view
| Doris-managed refresh and transparent query rewrite |
| `ephemeral` | Compiles into a CTE in downstream models | Reuse SQL without
creating a Doris object |
Seeds and snapshots can also create Doris tables. The adapter currently does
not include a custom `partition` materialization; use incremental
`insert_overwrite` for partition replacement. The examples below keep
`replication_num=1` so they also run on a single-BE development cluster; use a
production-appropriate replica count in real deployments.
@@ -311,7 +351,7 @@ select
from {{ source('raw', 'orders') }}
```
-The `table` materialization creates a Duplicate Key table and fully replaces
the target on subsequent runs. Table models accept the following table-creation
settings:
+For Internal Catalog targets, the `table` materialization creates a Duplicate
Key table and fully replaces it on subsequent runs. Iceberg targets do not use
Doris OLAP keys or default distribution. Table models accept the following
table-creation settings:
| Setting | Type and default | Description |
| --- | --- | --- |
@@ -324,7 +364,7 @@ The `table` materialization creates a Duplicate Key table
and fully replaces the
| `replication_num` | Positive integer or numeric string; optional | Replica
count; the top-level setting overrides the same key in `properties` |
| `properties` | Dictionary; optional | Key-value pairs rendered into Doris
`PROPERTIES` |
-Duplicate Key, partition, and distribution columns must be present in the
model output and comply with Doris table-creation rules. The adapter renders
`partition_by_init` and `properties` into the `CREATE TABLE` statement; Doris
validates the partition definitions, property names, and values. An ordinary
`table` model currently does not expose Aggregate Key or standalone Unique Key
configuration. Use incremental `merge` for Unique Key upserts.
+Duplicate Key, partition, and distribution columns must be present in the
model output and comply with Doris table-creation rules. The adapter renders
`partition_by_init` and `properties` into the `CREATE TABLE` statement; Doris
validates the partition definitions, property names, and values. OLAP-specific
options such as `duplicate_key` and `distributed_by` are not suitable for
Iceberg targets. An ordinary `table` model currently does not expose Aggregate
Key or standalone Unique Key co [...]
### Incremental
@@ -337,14 +377,14 @@ Microbatch does not use this filtering pattern. dbt Core
injects time filters fr
| Strategy | Doris target and behavior | Main requirements |
| --- | --- | --- |
-| `append` | Executes `INSERT INTO` against a Duplicate Key table | Do not set
`unique_key` |
-| `merge` | Performs full-row `INSERT INTO` upserts against a MOW or MOR
Unique Key table | `unique_key` is required |
-| `insert_overwrite` | Overwrites the whole table, named partitions, or
partitions touched by the batch | `unique_key` is rejected |
-| `microbatch` | Overwrites one exact RANGE partition for each dbt Core UTC
window | Configure `event_time`, `batch_size`, and `begin` |
+| `append` | `INSERT INTO` on an OLAP Duplicate Key or writable Iceberg table
| Do not set `unique_key` |
+| `merge` | Full-row `INSERT INTO` upsert on an OLAP Unique Key table; native
`MERGE INTO` on Iceberg | Requires `unique_key`; Iceberg also requires native
MERGE support and V2+ tables |
+| `insert_overwrite` | OLAP whole-table, named or dynamic partition overwrite;
Iceberg whole-table, dynamic or static partition overwrite | Rejects
`unique_key`; Iceberg static scope uses a column/value mapping |
+| `microbatch` | OLAP exact RANGE-partition overwrite; Iceberg replaces one
UTC window with native MERGE | Configure `event_time`, `batch_size`, and
`begin`; Iceberg requires native MERGE and V2+ |
When no strategy is set, `unique_key` selects `merge`; otherwise the adapter
uses `append`. The adapter supports all four `on_schema_change` modes:
`ignore`, `fail`, `append_new_columns`, and `sync_all_columns`.
-`delete+insert` and `delete_insert` are rejected. Use `merge` for Doris Unique
Key upserts. `merge` does not generate SQL `MERGE INTO`; it uses full-row
`INSERT INTO` with Doris Unique Key semantics. Each batch must contain at most
one row for each `unique_key`. To use a Sequence Column, set the visible column
through `function_column.sequence_col` in `properties` and continue using
`merge`. The bare `sequence_col` setting and `function_column.sequence_type`
with the hidden `__DORIS_SEQU [...]
+`delete+insert` and `delete_insert` are rejected. Use `merge` for Doris Unique
Key upserts. On OLAP targets, `merge` uses full-row `INSERT INTO` with Doris
Unique Key semantics. On Iceberg targets, it generates native `MERGE INTO` and
rejects duplicate source keys before writing. Each batch must contain at most
one row for each `unique_key`. To use a Sequence Column, set the visible column
through `function_column.sequence_col` in `properties` and continue using
`merge`. The bare `sequen [...]
The following example performs a full-row upsert by `order_id`:
@@ -376,7 +416,7 @@ where updated_at >= (
{% endif %}
```
-To overwrite named Doris partitions:
+To overwrite named Internal Catalog Doris partitions:
```sql
{{
@@ -421,7 +461,9 @@ The first run creates the target through CTAS and does not
execute `INSERT OVERW
- List of partition names: overwrite only those partitions.
- `'*'`: let Doris dynamically overwrite partitions touched by the batch.
-A dynamic overwrite cannot infer which partition to clear from an empty batch.
Specify partition names when an empty source batch must clear target data.
+A dynamic OLAP overwrite cannot infer which partition to clear from an empty
batch. Specify partition names when an empty source batch must clear target
data.
+
+For Iceberg targets, omit `overwrite_partitions` to use native dynamic
overwrite on partitioned tables. Set a non-empty column/value mapping, such as
`overwrite_partitions={'dt': '2025-01-25'}`, to replace one static scope. The
model must return all rows in that scope. On an unpartitioned target, the model
must return the complete replacement result; an empty result clears the table.
An empty result for a static scope also clears that scope. A mapping containing
`NULL` uses native MERGE [...]
### Microbatch
@@ -461,9 +503,9 @@ select
from {{ source('raw', 'orders') }}
```
-`event_time` must be an unquoted column name and must refer to the same single
column as `partition_by`. `batch_size` supports `hour`, `day`, `month`, and
`year`. The target is a Duplicate Key table: `duplicate_key` configures the
Doris table key and is not dbt `unique_key`. Do not set `unique_key`,
`overwrite_partitions`, or `partition_by_init`; the adapter derives the
partition boundary and overwrite target from the current batch. If an upstream
`ref()` or `source()` also configures `e [...]
+For an Internal Catalog OLAP target, `event_time` must be an unquoted column
name and must refer to the same single column as `partition_by`. `batch_size`
supports `hour`, `day`, `month`, and `year`. The target is a Duplicate Key
table: `duplicate_key` configures the Doris table key and is not dbt
`unique_key`. Do not set `unique_key`, `overwrite_partitions`, or
`partition_by_init`; the adapter derives the partition boundary and overwrite
target from the current batch. If an upstream `re [...]
-Microbatch can also use Doris Dynamic Partition when the following properties
are configured and pass adapter validation:
+Internal Catalog Microbatch can also use Doris Dynamic Partition when the
following properties are configured and pass adapter validation:
- `dynamic_partition.enable='true'`.
- `dynamic_partition.time_unit` matches `batch_size`, and
`dynamic_partition.time_zone` is `UTC`, `Etc/UTC`, or `+00:00`.
@@ -475,7 +517,7 @@ Doris also requires a positive `dynamic_partition.buckets`
value when creating a
:::caution
-For `append`, `merge`, and partition-scoped `insert_overwrite`, the adapter
does not discover new or changed source rows automatically. The model SQL must
return the intended batch. Whole-table `insert_overwrite` must return the
complete target data. Each `merge` batch must contain unique `unique_key`
values. dbt Core supplies Microbatch filtering from upstream resources
configured with `event_time`, so Microbatch models do not need
`is_incremental()`. Run `dbt run --full-refresh` after [...]
+For `append`, `merge`, and partition-scoped `insert_overwrite`, the adapter
does not discover new or changed source rows automatically. The model SQL must
return the intended batch. Whole-table `insert_overwrite` must return the
complete target data. Each `merge` batch must contain unique `unique_key`
values. dbt Core supplies Microbatch filtering from upstream resources
configured with `event_time`, so Microbatch models do not need
`is_incremental()`. Run `dbt run --full-refresh` after [...]
:::
@@ -567,15 +609,15 @@ When model SQL or DDL configuration changes, the default
`on_configuration_chang
| Capability | Support |
| --- | --- |
| Seed | Put small, version-controlled CSV files in `seeds/`, then run `dbt
seed`; supports type inference, `column_types`, and `ref` |
-| Snapshot | Define `check` or `timestamp` strategies in `snapshots/`, then
run `dbt snapshot`; supports hard deletes, schema evolution, and atomic
replacement |
+| Snapshot | Define `check` or `timestamp` strategies in `snapshots/`, then
run `dbt snapshot`; supports hard deletes and schema evolution. Internal
Catalog replacement is atomic; Iceberg replacement uses backup and rename |
| Data tests | Configure Generic Tests in YAML or write Singular Tests in
`tests/`; supports ephemeral and `store_failures` paths |
| Unit tests | Provide inline-row or CSV fixtures in model YAML, then run `dbt
test --select test_type:unit` |
-| Model contracts | Set `contract.enforced: true` and declare column names and
types for Table, View, or Incremental models; does not create database PK or
NOT NULL constraints |
+| Model contracts | Set `contract.enforced: true` and declare column names and
types for Table, View, or Incremental models; `constraints` declarations are
rejected, and data tests do not create database constraints |
| Persisted docs | Enable `persist_docs` for relation and column comments on
primary relation types |
| Source freshness | Configure `loaded_at_field` or `loaded_at_query` with
`freshness`, then run `dbt source freshness` |
| Grants | Use standard `grants` configuration for Doris `user` and
`user@host` table privileges; role principals are currently not supported |
| Hooks | Use `pre_hook` and `post_hook` to execute Doris SQL around
materializations; hook side effects have no transactional rollback |
-| Documentation and lineage | Write descriptions and run `dbt docs generate`
for metadata about Doris databases, tables, views, columns, comments, and
asynchronous materialized views |
+| Documentation and lineage | Write descriptions and run `dbt docs generate`
for Internal and External Catalog relation and column metadata, including
Internal Catalog asynchronous materialized views |
Seeds are suitable for small, version-controlled data such as country codes or
status mappings, not bulk business-data ingestion. Use Doris Stream Load,
Broker Load, or a Doris Connector for large files.
@@ -692,12 +734,12 @@ dbt run --select fct_orders --full-refresh
## Current limitations
- The adapter currently does not support Aggregate Key table modeling or
secondary-index configuration.
-- The adapter currently does not support a complete External Catalog namespace.
+- External Catalog write behavior depends on the Doris connector. Catalogs V2
and connector-specific write guarantees are not supported.
- The adapter currently does not implement SSL configuration, timeout/retry,
multi-FE failover, server-side cancellation, or complete query telemetry.
- Some Table/View/MV type changes have a short canonical-name availability
window rather than a zero-downtime switch.
<!-- Knowledge Type: Troubleshooting -->
-<!-- Use Case: Connection errors / Profile database and schema mismatch -->
+<!-- Use Case: Connection errors / Catalog and database lookup -->
## Troubleshooting
@@ -705,6 +747,6 @@ dbt run --select fct_orders --full-refresh
Verify that `host` and `port` point to a reachable FE MySQL query port, and
check the username, password, and network policy. The default query port is
`9030`; `8030` is normally the FE HTTP port.
-### `database` and `schema` differ
+### Catalog or database cannot be found
-Profile `schema` is the target Doris database. Remove `database`; it is not a
Doris catalog and does not add another namespace level. If you retain this
compatibility field, set it to exactly the same value as `schema`.
+In v1.1.1, profile and source `database` values identify a Doris Catalog, and
`schema` identifies a database inside it. Omit `database` for Internal Catalog.
If an old project used source `database` for a Doris database, move that value
to `schema`. Check that external Catalogs already exist and are visible to the
dbt account.
diff --git a/docs/connection-integration/data-integration/intro.mdx
b/docs/connection-integration/data-integration/intro.mdx
index c024a0cc039..7707b88b92d 100644
--- a/docs/connection-integration/data-integration/intro.mdx
+++ b/docs/connection-integration/data-integration/intro.mdx
@@ -239,9 +239,9 @@ Apache Doris supports integration with a wide range of data
ecosystem tools. You
<div className="cards-grid">
<GettingStartedCard
- title="DBT Doris Adapter"
+ title="dbt Adapter"
icon="/images/next/connection-integration/data-integration/logos/dbt.svg"
- description="Use the dbt-doris adapter to run dbt models in Doris for ELT
transformations and data modeling"
+ description="Use dbt-for-apache-doris to run dbt models in Doris for ELT
transformations and data modeling"
link="./dbt-doris-adapter"
/>
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/connection-integration/data-integration/dbt-doris-adapter.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/connection-integration/data-integration/dbt-doris-adapter.md
index 90b2021bef9..cfe88da5df9 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/connection-integration/data-integration/dbt-doris-adapter.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/connection-integration/data-integration/dbt-doris-adapter.md
@@ -1,10 +1,11 @@
---
{
- "title": "Apache Doris dbt Adapter",
+ "title": "dbt Adapter",
+ "sidebar_label": "dbt Adapter",
"language": "zh-CN",
"toc_min_heading_level": 2,
"toc_max_heading_level": 3,
- "description": "安装和配置 dbt-for-apache-doris，连接 Apache Doris，并构建增量模型、Source
Freshness、异步物化视图及其他 dbt 数据转换功能。",
+ "description": "安装 dbt-for-apache-doris 1.1.1，配置模型、External Catalog
数据源、Iceberg 目标表和异步物化视图，并了解版本限制与常见故障处理。",
"keywords": [
"Apache Doris dbt",
"dbt Doris Adapter",
@@ -44,11 +45,13 @@
| 项目 | 要求 |
| --- | --- |
| Python | 3.10 或更高版本 |
+| dbt Core | 1.12.x（安装 Adapter 时自动安装） |
## 已发布版本
| 版本 |
| --- |
+| [1.1.1](https://github.com/velodb/dbt-for-apache-doris/releases/tag/v1.1.1) |
| [1.1.0](https://github.com/velodb/dbt-for-apache-doris/releases/tag/v1.1.0) |
<!-- 知识类型: 操作步骤 -->
@@ -56,7 +59,7 @@
## 安装 dbt-for-apache-doris
-以下命令以已发布的
[`v1.1.0`](https://github.com/velodb/dbt-for-apache-doris/releases/tag/v1.1.0)
为安装示例。安装其他版本时，请替换 `pip` 依赖中的版本号。
+以下命令以已发布的
[`v1.1.1`](https://github.com/velodb/dbt-for-apache-doris/releases/tag/v1.1.1)
为安装示例。安装其他版本时，请替换 `pip` 依赖中的版本号。
建议在独立的 Python 虚拟环境中从 PyPI 安装 Adapter：
@@ -64,7 +67,7 @@
python3 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
-python -m pip install "dbt-for-apache-doris==1.1.0"
+python -m pip install "dbt-for-apache-doris==1.1.1"
dbt --version
```
@@ -74,11 +77,11 @@ Windows PowerShell 使用以下命令：
py -m venv .venv
.\.venv\Scripts\Activate.ps1
python -m pip install --upgrade pip
-python -m pip install "dbt-for-apache-doris==1.1.0"
+python -m pip install "dbt-for-apache-doris==1.1.1"
dbt --version
```
-在本示例中，输出的 `Plugins` 列表中应包含 `doris: 1.1.0`。dbt Core 和 MySQL Connector/Python
会作为依赖项自动安装。
+在本示例中，输出的 `Plugins` 列表中应包含 `doris: 1.1.1`。dbt Core 和 MySQL Connector/Python
会作为依赖项自动安装。
<!-- 知识类型: 配置参数 / 操作步骤 -->
<!-- 适用场景: Doris Profile 配置 / 连接验证 / 跨 Database 读取 -->
@@ -119,24 +122,9 @@ doris_demo:
| `password` | 建议通过环境变量传入 | 省略时为空字符串 |
| `schema` | 必须填写目标 Doris Database 名称 | `dbt init` 的提示默认值为 `dbt`；底层凭据本身的默认值为
`None` |
| `threads` | 按 FE 和任务负载设置 | `dbt init` 的提示默认值为 `1` |
-| `database` | 不要在 Profile 中配置 | 这是 dbt 通用凭据字段；如果提供，当前实现要求它与 `schema` 完全相同 |
+| `database` | Internal Catalog 省略；External Catalog 目标填写已有 Doris Catalog 名称 |
在 v1.1.1 中映射为 Doris Catalog |
-Adapter 将 dbt Schema 映射为 Doris Database。目标 Database 不存在时，
-Adapter 会尝试创建它，因此执行用户需要相应权限。执行用户还需要读取源表，以及在
-目标 Database 中创建、修改、删除和写入对象的权限。
-
-这里的 `database` 不是 Doris Catalog。它只是 dbt 通用的
-`database.schema.identifier` 命名模型遗留到 Adapter 的字段；Doris Internal
-Catalog 中只需要 `database.table` 两层，因此 Profile 使用 `schema` 表示 Doris
-Database，并省略 `database`。Source 资源中的 `database` 是另一种用法：当前
-Adapter 会把它作为 Doris Database 覆盖 `schema`，用于跨 Database 读取。Profile
-和标准 `source()` 当前不能同时表示 Doris Catalog 与 Database 两个层级。
-
-:::caution
-
-当前不支持 External Catalog 的 Catalog、Database、Table 三段式命名空间。
-
-:::
+v1.1.1 将 dbt 的 `database.schema.identifier` 映射为 Doris 的
`catalog.database.table`。使用 Internal Catalog 时，省略 `database`，并将 `schema` 设置为
Doris Database。写入 External Catalog 时，将 `database` 设置为已有 Doris Catalog，将
`schema` 设置为该 Catalog 中的 Database。如果目标 Database 不存在且 Catalog 支持创建，Adapter
会尝试创建。执行 dbt 的用户需要读取源数据，以及创建、修改、删除和写入目标对象的权限。External Catalog 的写入能力还取决于 Doris
Connector。
确保 `dbt_project.yml` 中的 `profile` 与 `profiles.yml` 顶层名称一致：
@@ -267,10 +255,52 @@ sources:
- name: payments
```
-为兼容已有 dbt 项目，也可以只设置 `database: finance_raw`，或同时把 `database` 和
-`schema` 设置为相同值。对于 Source，若两者不同，当前 Adapter 会采用 `database`
-作为 Doris Database；这与 Profile 中必须保持两者相同的连接校验不同。为了避免
-歧义，新项目建议统一使用 `schema`。
+在 v1.1.1 中，Source 的 `database` 表示 Doris Catalog。读取 Internal Catalog 中其他
Database 的数据时，只需按上例设置 `schema`。如果旧项目在 v1.1.0 中使用 `database: finance_raw` 表示跨
Database Source，升级前应将该值移到 `schema`。
+
+## v1.1.1 中的 External Catalog
+
+Adapter 可以读取 External Catalog 中的 Source，并通过 `dbt docs generate` 获取表和字段元数据。先在
Doris 中创建 Catalog，再把 Catalog 名称写入 Source 的 `database` 字段：
+
+```yaml
+sources:
+ - name: lakehouse
+ database: hive_catalog
+ schema: ods
+ tables:
+ - name: orders
+```
+
+`{{ source('lakehouse', 'orders') }}` 对应 `hive_catalog.ods.orders`。其中 `schema`
表示该 Catalog 中的 Database。Adapter 支持三段式对象名，具体 DDL 和写入能力仍由 Doris Catalog Connector
决定。
+
+要创建 Iceberg 目标表，先[配置可写的 Iceberg
Catalog](../../lakehouse/catalogs/iceberg-catalog)，并在现有 Profile Output 中设置：
+
+```yaml
+database: iceberg_catalog
+schema: analytics
+```
+
+创建 `models/iceberg_sample.sql`：
+
+```sql
+{{ config(materialized='table', engine='iceberg') }}
+
+select 1 as id
+```
+
+执行 `dbt run --select iceberg_sample`，创建
`iceberg_catalog.analytics.iceberg_sample`。Catalog 必须已存在；Database 必须已存在或允许 dbt
创建，目标需要可写。`engine='iceberg'` 会生成显式 Engine 子句；省略 `engine` 时，Doris 会根据目标 Catalog
推断。Catalog 推断已在 Doris 4.1.3 验证；更早的 Doris 版本可能需要显式设置 Engine。
+
+| Iceberg 目标 | v1.1.1 行为 |
+| --- | --- |
+| `table` | 创建 Iceberg 表；再次运行时通过备份和重命名替换目标 |
+| `incremental` | 支持下文介绍的 `append`、`merge`、`insert_overwrite` 和 `microbatch` |
+| Seed 和 Snapshot | 支持；Snapshot 发布和 Seed 完整刷新使用备份和重命名。Iceberg Seed 常规重载使用覆盖或
MERGE |
+| `view` 和 `materialized_view` | 目标必须位于 Internal Catalog；Internal Catalog 中的
View 和 MV 可以查询 External Catalog Source |
+
+Iceberg 的 `merge` 和 `microbatch` 需要支持原生 Iceberg `MERGE INTO` 的 Doris 版本，以及
Iceberg V2 或更高版本的表。这些路径已在 Doris 4.1.3 验证；[Doris 文档将 Iceberg MERGE 标记为自 4.1.0
起的实验功能](../../lakehouse/catalogs/iceberg-catalog#merge-into)。External Catalog
表的替换不是原子操作：两次重命名之间，目标表名会短暂不可用；失败时旧表可能保留在备份名称下。生产使用前，应在隔离项目中验证 Doris 版本、Catalog
Connector 和故障恢复流程。
+
+Adapter 会在写入 Table、View、Incremental 或 MV 目标前拒绝 Model 或字段上的 `constraints`
声明。Model Contract 仍会校验字段名称和类型。`not_null` 等 `data_tests` 可在写入后校验数据，但不会创建数据库约束。
+
+[五个可运行的 Doris
Demo](https://github.com/velodb/dbt-for-apache-doris/tree/v1.1.1/examples/doris-demos)
展示了 Model、测试、跨 Database Source、增量处理、Snapshot 和异步物化视图。
<!-- 知识类型: 能力定义 / 配置参数 / 架构选型决策 -->
<!-- 适用场景: Materialization 选型 / 增量加工 / 物化视图 -->
@@ -281,10 +311,10 @@ sources:
| Materialization | Doris 对象或行为 | 适用场景 |
| --- | --- | --- |
-| `view` | Doris View | 轻量转换、始终读取最新源数据 |
-| `table` | 每次运行完整重建的 Doris Duplicate Key Table | 结果规模可控、需要稳定查询性能 |
+| `view` | Internal Catalog 中的 Doris View | 轻量转换、始终读取最新源数据 |
+| `table` | 每次运行完整重建的 Internal Catalog Duplicate Key 表或可写 Iceberg 表 |
结果规模可控、需要稳定查询性能 |
| `incremental` | 按策略追加、Upsert 或覆盖本批数据 | 大表增量加工 |
-| `materialized_view` | Doris 异步物化视图 | 由 Doris 管理刷新和透明改写 |
+| `materialized_view` | Internal Catalog 中的 Doris 异步物化视图 | 由 Doris 管理刷新和透明改写 |
| `ephemeral` | 由 dbt 编译为下游 Model 中的公共表表达式（CTE） | 只复用 SQL、不创建 Doris 对象 |
Seed 和 Snapshot 也可创建 Doris Table。Adapter 当前不包含自定义 `partition`
@@ -331,7 +361,7 @@ select
from {{ source('raw', 'orders') }}
```
-Table Materialization 创建 Duplicate Key 表，后续运行会完整替换目标。Table Model 可配置以下建表参数：
+对于 Internal Catalog 目标，Table Materialization 创建 Duplicate Key
表，后续运行会完整替换目标。Iceberg 目标不使用 Doris OLAP Key 或默认分桶。Table Model 可配置以下建表参数：
| 配置项 | 类型和默认值 | 说明 |
| --- | --- | --- |
@@ -346,7 +376,7 @@ Table Materialization 创建 Duplicate Key 表，后续运行会完整替换目
`duplicate_key`、分区列和分桶列必须出现在 Model 输出中，并满足 Doris 建表规则。
Adapter 会将 `partition_by_init` 和 `properties` 生成到 `CREATE TABLE` 语句中；分区定义、
-属性名称和取值是否合法，由 Doris 校验。
+属性名称和取值是否合法，由 Doris 校验。`duplicate_key`、`distributed_by` 等 OLAP 专用配置不适用于
Iceberg 目标。
普通 Table Model 当前不支持配置 Aggregate Key 或独立 Unique Key；Unique Key Upsert 使用
Incremental `merge`。
@@ -394,23 +424,22 @@ where updated_at >= (
| `incremental_strategy` | 目标表和写入语义 | 主要配置 |
| --- | --- | --- |
-| `append` | 向 Duplicate Key 表追加数据，不去重 | 不需要 `unique_key` |
-| `merge` | 向 Merge-on-Write 或 Merge-on-Read Unique Key 表执行全行 `INSERT INTO`
Upsert | 必须设置 `unique_key` |
-| `insert_overwrite` | 使用 Doris `INSERT OVERWRITE` 覆盖整表、指定分区或本批涉及的分区 | 不能设置
`unique_key`；可设置 `overwrite_partitions` |
-| `microbatch` | 按 dbt Core 的 UTC 时间窗口逐批覆盖一个精确的 Doris RANGE 分区 | 设置
`event_time`、`batch_size` 和 `begin`；不能设置 `unique_key` |
+| `append` | 对 OLAP Duplicate Key 表或可写 Iceberg 表执行 `INSERT INTO` | 不设置
`unique_key` |
+| `merge` | OLAP Unique Key 表使用全行 `INSERT INTO` Upsert；Iceberg 表使用原生 `MERGE
INTO` | 必须设置 `unique_key`；Iceberg 还要求原生 MERGE 能力和 V2+ 表 |
+| `insert_overwrite` | OLAP 可覆盖整表、指定分区或动态分区；Iceberg 可覆盖整表、动态分区或静态分区 | 不允许
`unique_key`；Iceberg 静态范围使用字段和值的映射 |
+| `microbatch` | OLAP 覆盖精确 RANGE 分区；Iceberg 使用原生 MERGE 替换一个 UTC 时间窗口 | 设置
`event_time`、`batch_size` 和 `begin`；Iceberg 要求原生 MERGE 和 V2+ |
未设置 `incremental_strategy` 时，有 `unique_key` 使用 `merge`，否则使用
`append`。Adapter 支持 `ignore`、`fail`、`append_new_columns` 和
`sync_all_columns` 四种 `on_schema_change` 策略。
`delete+insert` 和 `delete_insert` 已被当前实现明确拒绝；Doris Unique Key Upsert
-应使用 `merge`。`merge` 不生成原生 `MERGE INTO`，而是依赖 Doris Unique Key
-表的全行 `INSERT INTO` 语义。每批数据中的 `unique_key` 必须唯一。需要 Sequence
+应使用 `merge`。对 OLAP 目标，`merge` 依赖 Doris Unique Key 表的全行 `INSERT INTO` 语义。对
Iceberg 目标，`merge` 会生成原生 `MERGE INTO`，并在写入前拒绝源数据中重复的 Key。每批数据中的 `unique_key`
必须唯一。需要 Sequence
Column 时，在 `properties` 中设置可见列
`function_column.sequence_col`，并继续使用 `merge`。裸 `sequence_col` 配置和使用隐藏
`__DORIS_SEQUENCE_COL__` 的 `function_column.sequence_type` 当前均不受支持。
-覆盖指定 Doris 分区：
+覆盖指定的 Internal Catalog Doris 分区：
```sql
{{
@@ -452,8 +481,9 @@ where order_time >= '2026-07-01'
首次运行走 CTAS 建表路径，不执行 `INSERT OVERWRITE`，所以示例必须用
`partition_by_init` 创建目标分区。`overwrite_partitions` 只决定目标表已存在时的覆盖
范围。省略 `overwrite_partitions` 会覆盖整张表；设置分区名列表会只覆盖这些分区；
-设置 `'*'` 时，Doris 根据本批数据动态覆盖涉及的分区。空批次
-无法识别需要清空的分区；需要清空分区时应显式指定分区名。
+设置 `'*'` 时，Doris 根据本批数据动态覆盖涉及的分区。OLAP 动态覆盖无法从空批次识别需要清空的分区；需要清空分区时应显式指定分区名。
+
+对于 Iceberg 目标，省略 `overwrite_partitions` 可动态覆盖本批涉及的分区。设置非空字段和值映射，例如
`overwrite_partitions={'dt': '2025-01-25'}`，可替换一个静态范围。Model
必须返回该范围内的完整数据。对于非分区表，Model 必须返回完整替换结果；空结果会清空整表。静态范围的空结果也会清空该范围。映射中包含 `NULL`
时使用原生 MERGE，因此需要 Iceberg V2+。
### Microbatch
@@ -499,7 +529,7 @@ select
from {{ source('raw', 'orders') }}
```
-`event_time` 必须是未加引号的列名，并与 `partition_by` 指向同一个单列；
+对于 Internal Catalog OLAP 目标，`event_time` 必须是未加引号的列名，并与 `partition_by` 指向同一个单列；
`batch_size` 支持 `hour`、
`day`、`month` 和 `year`。目标是 Duplicate Key 表；`duplicate_key` 是 Doris 表的
Key 配置，不是 dbt 的 `unique_key`。不要为 Microbatch 设置 `unique_key`、
@@ -508,7 +538,7 @@ Key 配置，不是 dbt 的 `unique_key`。不要为 Microbatch 设置 `unique_k
当前批次窗口过滤输入。Microbatch 批次串行执行；即使遇到空批次，也会覆盖对应的
精确分区，因此可以清除该窗口中已不存在的数据。
-Microbatch 也可使用 Doris Dynamic Partition，但必须在 `properties` 中显式启用并
+Internal Catalog 中的 Microbatch 也可使用 Doris Dynamic Partition，但必须在 `properties`
中显式启用并
满足当前 Adapter 的校验：
- `dynamic_partition.enable='true'`；
@@ -530,7 +560,7 @@ Doris 创建 Dynamic Partition 表时还要求
`dynamic_partition.buckets` 为
目标表所需的完整数据。同一批 `merge` 数据中的 `unique_key` 必须唯一。Microbatch
的时间过滤由 dbt Core 根据已配置 `event_time` 的上游资源生成，不需要使用
`is_incremental()`。修改 Key、分区方式或其他不兼容表结构后，应执行
-`dbt run --full-refresh`。
+`dbt run --full-refresh`。对于 Iceberg Microbatch，不要配置 OLAP Dynamic Partition
属性、具名 `overwrite_partitions` 或 `partition_by_init`；每个 UTC 时间窗口由原生 MERGE
替换，包括空窗口。
:::
### 异步物化视图
@@ -635,15 +665,15 @@ Model SQL 或 DDL 配置变化时，`on_configuration_change=apply` 默认通过
| 功能 | 用法 |
| --- | --- |
| Seed | 将小型、版本受控的 CSV 放入 `seeds/`，执行 `dbt seed`；支持类型推断、`column_types` 和 `ref`
|
-| Snapshot | 在 `snapshots/` 中定义 `check` 或 `timestamp` 策略，执行 `dbt snapshot`；支持
Hard Delete、Schema 演进和原子替换 |
+| Snapshot | 在 `snapshots/` 中定义 `check` 或 `timestamp` 策略，执行 `dbt snapshot`；支持
Hard Delete 和 Schema 演进。Internal Catalog 原子替换，Iceberg 使用备份和重命名 |
| Data Test | 在 YAML 中配置 Generic Test，或在 `tests/` 中编写 Singular Test；支持
Ephemeral 和 `store_failures` 路径 |
| Unit Test | 在 Model YAML 中提供 Inline Row 或 CSV Fixture，执行 `dbt test --select
test_type:unit` |
-| Model Contract | 对 Table、View 或 Incremental Model 设置 `contract.enforced:
true` 并声明字段名称和类型；不会创建数据库 PK 或 NOT NULL 约束 |
+| Model Contract | 对 Table、View 或 Incremental Model 设置 `contract.enforced:
true` 并声明字段名称和类型；Adapter 拒绝 `constraints` 声明，Data Test 不会创建数据库约束 |
| Persisted Docs | 启用 `persist_docs`，为主要 Relation 类型写入 Relation 和字段注释 |
| Source Freshness | 配置 `loaded_at_field` 或 `loaded_at_query` 以及
`freshness`，执行 `dbt source freshness` |
| Grants | 使用标准 `grants` 配置管理 Doris `user` 和 `user@host` 表权限；当前不支持 Role 授权主体 |
| Hooks | 使用 `pre_hook` 和 `post_hook` 在 Materialization 前后执行 Doris SQL；Hook
副作用没有事务回滚 |
-| 文档与血缘 | 编写 Description 并执行 `dbt docs generate`，生成 Doris
Database、Table、View、字段、注释和异步物化视图元数据 |
+| 文档与血缘 | 编写 Description 并执行 `dbt docs generate`，获取 Internal 和 External
Catalog 的 Relation 与字段元数据，包括 Internal Catalog 异步物化视图 |
Seed 适合国家代码、状态映射等小型静态数据，不适合批量导入业务数据。大文件应使用
Doris Stream Load、Broker Load 或 Doris Connector。
@@ -768,12 +798,12 @@ dbt run --select fct_orders --full-refresh
## 当前限制
- 当前不支持 Aggregate Key 表建模和 Secondary Index 配置。
-- 当前不支持完整的 External Catalog 命名空间。
+- External Catalog 的写入行为取决于 Doris Connector。当前不支持 Catalogs V2 或针对各 Connector
的写入保证。
- 当前未实现 SSL 配置、超时与重试、多 FE 故障转移、服务端取消和完整的查询遥测。
- 部分 Table、View 和 MV 类型切换存在短暂的标准对象名不可用窗口，无法做到零停机切换。
<!-- 知识类型: 故障排查 -->
-<!-- 适用场景: 连接失败 / Profile database 与 schema 不一致 -->
+<!-- 适用场景: 连接失败 / Catalog 与 Database 查找 -->
## 故障处理
@@ -782,8 +812,6 @@ dbt run --select fct_orders --full-refresh
确认 `host` 和 `port` 指向可访问的 Doris FE Query Port，并检查用户名、密码和
网络策略。`9030` 是默认 Query Port，`8030` 通常是 FE HTTP Port。
-### `database` 与 `schema` 不一致
+### 找不到 Catalog 或 Database
-Profile 中的 `schema` 就是目标 Doris Database。删除 `database`；它不是 Doris
-Catalog，而且在 Profile 中没有提供额外命名层级。如果保留该兼容字段，当前实现
-要求它与 `schema` 完全相同。
+在 v1.1.1 中，Profile 和 Source 的 `database` 表示 Doris Catalog，`schema` 表示其中的
Database。使用 Internal Catalog 时省略 `database`。如果旧项目曾用 Source 的 `database` 表示
Doris Database，升级前应将该值移到 `schema`。检查 External Catalog 是否已创建，以及 dbt 用户是否有权限查看。
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/connection-integration/data-integration/intro.mdx
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/connection-integration/data-integration/intro.mdx
index 11a98a1118a..822af4e2e3c 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/connection-integration/data-integration/intro.mdx
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/connection-integration/data-integration/intro.mdx
@@ -239,9 +239,9 @@ Apache Doris 支持与多类数据生态工具集成。您可以按照使用场
<div className="cards-grid">
<GettingStartedCard
- title="DBT Doris Adapter"
+ title="dbt Adapter"
icon="/images/next/connection-integration/data-integration/logos/dbt.svg"
- description="使用 dbt-doris adapter 在 Doris 中运行 dbt 模型，完成 ELT 转换和数据建模"
+ description="使用 dbt-for-apache-doris 在 Doris 中运行 dbt 模型，完成 ELT 转换和数据建模"
link="./dbt-doris-adapter"
/>
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/connection-integration/data-integration/dbt-doris-adapter.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/connection-integration/data-integration/dbt-doris-adapter.md
index 90b2021bef9..cfe88da5df9 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/connection-integration/data-integration/dbt-doris-adapter.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/connection-integration/data-integration/dbt-doris-adapter.md
@@ -1,10 +1,11 @@
---
{
- "title": "Apache Doris dbt Adapter",
+ "title": "dbt Adapter",
+ "sidebar_label": "dbt Adapter",
"language": "zh-CN",
"toc_min_heading_level": 2,
"toc_max_heading_level": 3,
- "description": "安装和配置 dbt-for-apache-doris，连接 Apache Doris，并构建增量模型、Source
Freshness、异步物化视图及其他 dbt 数据转换功能。",
+ "description": "安装 dbt-for-apache-doris 1.1.1，配置模型、External Catalog
数据源、Iceberg 目标表和异步物化视图，并了解版本限制与常见故障处理。",
"keywords": [
"Apache Doris dbt",
"dbt Doris Adapter",
@@ -44,11 +45,13 @@
| 项目 | 要求 |
| --- | --- |
| Python | 3.10 或更高版本 |
+| dbt Core | 1.12.x（安装 Adapter 时自动安装） |
## 已发布版本
| 版本 |
| --- |
+| [1.1.1](https://github.com/velodb/dbt-for-apache-doris/releases/tag/v1.1.1) |
| [1.1.0](https://github.com/velodb/dbt-for-apache-doris/releases/tag/v1.1.0) |
<!-- 知识类型: 操作步骤 -->
@@ -56,7 +59,7 @@
## 安装 dbt-for-apache-doris
-以下命令以已发布的
[`v1.1.0`](https://github.com/velodb/dbt-for-apache-doris/releases/tag/v1.1.0)
为安装示例。安装其他版本时，请替换 `pip` 依赖中的版本号。
+以下命令以已发布的
[`v1.1.1`](https://github.com/velodb/dbt-for-apache-doris/releases/tag/v1.1.1)
为安装示例。安装其他版本时，请替换 `pip` 依赖中的版本号。
建议在独立的 Python 虚拟环境中从 PyPI 安装 Adapter：
@@ -64,7 +67,7 @@
python3 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
-python -m pip install "dbt-for-apache-doris==1.1.0"
+python -m pip install "dbt-for-apache-doris==1.1.1"
dbt --version
```
@@ -74,11 +77,11 @@ Windows PowerShell 使用以下命令：
py -m venv .venv
.\.venv\Scripts\Activate.ps1
python -m pip install --upgrade pip
-python -m pip install "dbt-for-apache-doris==1.1.0"
+python -m pip install "dbt-for-apache-doris==1.1.1"
dbt --version
```
-在本示例中，输出的 `Plugins` 列表中应包含 `doris: 1.1.0`。dbt Core 和 MySQL Connector/Python
会作为依赖项自动安装。
+在本示例中，输出的 `Plugins` 列表中应包含 `doris: 1.1.1`。dbt Core 和 MySQL Connector/Python
会作为依赖项自动安装。
<!-- 知识类型: 配置参数 / 操作步骤 -->
<!-- 适用场景: Doris Profile 配置 / 连接验证 / 跨 Database 读取 -->
@@ -119,24 +122,9 @@ doris_demo:
| `password` | 建议通过环境变量传入 | 省略时为空字符串 |
| `schema` | 必须填写目标 Doris Database 名称 | `dbt init` 的提示默认值为 `dbt`；底层凭据本身的默认值为
`None` |
| `threads` | 按 FE 和任务负载设置 | `dbt init` 的提示默认值为 `1` |
-| `database` | 不要在 Profile 中配置 | 这是 dbt 通用凭据字段；如果提供，当前实现要求它与 `schema` 完全相同 |
+| `database` | Internal Catalog 省略；External Catalog 目标填写已有 Doris Catalog 名称 |
在 v1.1.1 中映射为 Doris Catalog |
-Adapter 将 dbt Schema 映射为 Doris Database。目标 Database 不存在时，
-Adapter 会尝试创建它，因此执行用户需要相应权限。执行用户还需要读取源表，以及在
-目标 Database 中创建、修改、删除和写入对象的权限。
-
-这里的 `database` 不是 Doris Catalog。它只是 dbt 通用的
-`database.schema.identifier` 命名模型遗留到 Adapter 的字段；Doris Internal
-Catalog 中只需要 `database.table` 两层，因此 Profile 使用 `schema` 表示 Doris
-Database，并省略 `database`。Source 资源中的 `database` 是另一种用法：当前
-Adapter 会把它作为 Doris Database 覆盖 `schema`，用于跨 Database 读取。Profile
-和标准 `source()` 当前不能同时表示 Doris Catalog 与 Database 两个层级。
-
-:::caution
-
-当前不支持 External Catalog 的 Catalog、Database、Table 三段式命名空间。
-
-:::
+v1.1.1 将 dbt 的 `database.schema.identifier` 映射为 Doris 的
`catalog.database.table`。使用 Internal Catalog 时，省略 `database`，并将 `schema` 设置为
Doris Database。写入 External Catalog 时，将 `database` 设置为已有 Doris Catalog，将
`schema` 设置为该 Catalog 中的 Database。如果目标 Database 不存在且 Catalog 支持创建，Adapter
会尝试创建。执行 dbt 的用户需要读取源数据，以及创建、修改、删除和写入目标对象的权限。External Catalog 的写入能力还取决于 Doris
Connector。
确保 `dbt_project.yml` 中的 `profile` 与 `profiles.yml` 顶层名称一致：
@@ -267,10 +255,52 @@ sources:
- name: payments
```
-为兼容已有 dbt 项目，也可以只设置 `database: finance_raw`，或同时把 `database` 和
-`schema` 设置为相同值。对于 Source，若两者不同，当前 Adapter 会采用 `database`
-作为 Doris Database；这与 Profile 中必须保持两者相同的连接校验不同。为了避免
-歧义，新项目建议统一使用 `schema`。
+在 v1.1.1 中，Source 的 `database` 表示 Doris Catalog。读取 Internal Catalog 中其他
Database 的数据时，只需按上例设置 `schema`。如果旧项目在 v1.1.0 中使用 `database: finance_raw` 表示跨
Database Source，升级前应将该值移到 `schema`。
+
+## v1.1.1 中的 External Catalog
+
+Adapter 可以读取 External Catalog 中的 Source，并通过 `dbt docs generate` 获取表和字段元数据。先在
Doris 中创建 Catalog，再把 Catalog 名称写入 Source 的 `database` 字段：
+
+```yaml
+sources:
+ - name: lakehouse
+ database: hive_catalog
+ schema: ods
+ tables:
+ - name: orders
+```
+
+`{{ source('lakehouse', 'orders') }}` 对应 `hive_catalog.ods.orders`。其中 `schema`
表示该 Catalog 中的 Database。Adapter 支持三段式对象名，具体 DDL 和写入能力仍由 Doris Catalog Connector
决定。
+
+要创建 Iceberg 目标表，先[配置可写的 Iceberg
Catalog](../../lakehouse/catalogs/iceberg-catalog)，并在现有 Profile Output 中设置：
+
+```yaml
+database: iceberg_catalog
+schema: analytics
+```
+
+创建 `models/iceberg_sample.sql`：
+
+```sql
+{{ config(materialized='table', engine='iceberg') }}
+
+select 1 as id
+```
+
+执行 `dbt run --select iceberg_sample`，创建
`iceberg_catalog.analytics.iceberg_sample`。Catalog 必须已存在；Database 必须已存在或允许 dbt
创建，目标需要可写。`engine='iceberg'` 会生成显式 Engine 子句；省略 `engine` 时，Doris 会根据目标 Catalog
推断。Catalog 推断已在 Doris 4.1.3 验证；更早的 Doris 版本可能需要显式设置 Engine。
+
+| Iceberg 目标 | v1.1.1 行为 |
+| --- | --- |
+| `table` | 创建 Iceberg 表；再次运行时通过备份和重命名替换目标 |
+| `incremental` | 支持下文介绍的 `append`、`merge`、`insert_overwrite` 和 `microbatch` |
+| Seed 和 Snapshot | 支持；Snapshot 发布和 Seed 完整刷新使用备份和重命名。Iceberg Seed 常规重载使用覆盖或
MERGE |
+| `view` 和 `materialized_view` | 目标必须位于 Internal Catalog；Internal Catalog 中的
View 和 MV 可以查询 External Catalog Source |
+
+Iceberg 的 `merge` 和 `microbatch` 需要支持原生 Iceberg `MERGE INTO` 的 Doris 版本，以及
Iceberg V2 或更高版本的表。这些路径已在 Doris 4.1.3 验证；[Doris 文档将 Iceberg MERGE 标记为自 4.1.0
起的实验功能](../../lakehouse/catalogs/iceberg-catalog#merge-into)。External Catalog
表的替换不是原子操作：两次重命名之间，目标表名会短暂不可用；失败时旧表可能保留在备份名称下。生产使用前，应在隔离项目中验证 Doris 版本、Catalog
Connector 和故障恢复流程。
+
+Adapter 会在写入 Table、View、Incremental 或 MV 目标前拒绝 Model 或字段上的 `constraints`
声明。Model Contract 仍会校验字段名称和类型。`not_null` 等 `data_tests` 可在写入后校验数据，但不会创建数据库约束。
+
+[五个可运行的 Doris
Demo](https://github.com/velodb/dbt-for-apache-doris/tree/v1.1.1/examples/doris-demos)
展示了 Model、测试、跨 Database Source、增量处理、Snapshot 和异步物化视图。
<!-- 知识类型: 能力定义 / 配置参数 / 架构选型决策 -->
<!-- 适用场景: Materialization 选型 / 增量加工 / 物化视图 -->
@@ -281,10 +311,10 @@ sources:
| Materialization | Doris 对象或行为 | 适用场景 |
| --- | --- | --- |
-| `view` | Doris View | 轻量转换、始终读取最新源数据 |
-| `table` | 每次运行完整重建的 Doris Duplicate Key Table | 结果规模可控、需要稳定查询性能 |
+| `view` | Internal Catalog 中的 Doris View | 轻量转换、始终读取最新源数据 |
+| `table` | 每次运行完整重建的 Internal Catalog Duplicate Key 表或可写 Iceberg 表 |
结果规模可控、需要稳定查询性能 |
| `incremental` | 按策略追加、Upsert 或覆盖本批数据 | 大表增量加工 |
-| `materialized_view` | Doris 异步物化视图 | 由 Doris 管理刷新和透明改写 |
+| `materialized_view` | Internal Catalog 中的 Doris 异步物化视图 | 由 Doris 管理刷新和透明改写 |
| `ephemeral` | 由 dbt 编译为下游 Model 中的公共表表达式（CTE） | 只复用 SQL、不创建 Doris 对象 |
Seed 和 Snapshot 也可创建 Doris Table。Adapter 当前不包含自定义 `partition`
@@ -331,7 +361,7 @@ select
from {{ source('raw', 'orders') }}
```
-Table Materialization 创建 Duplicate Key 表，后续运行会完整替换目标。Table Model 可配置以下建表参数：
+对于 Internal Catalog 目标，Table Materialization 创建 Duplicate Key
表，后续运行会完整替换目标。Iceberg 目标不使用 Doris OLAP Key 或默认分桶。Table Model 可配置以下建表参数：
| 配置项 | 类型和默认值 | 说明 |
| --- | --- | --- |
@@ -346,7 +376,7 @@ Table Materialization 创建 Duplicate Key 表，后续运行会完整替换目
`duplicate_key`、分区列和分桶列必须出现在 Model 输出中，并满足 Doris 建表规则。
Adapter 会将 `partition_by_init` 和 `properties` 生成到 `CREATE TABLE` 语句中；分区定义、
-属性名称和取值是否合法，由 Doris 校验。
+属性名称和取值是否合法，由 Doris 校验。`duplicate_key`、`distributed_by` 等 OLAP 专用配置不适用于
Iceberg 目标。
普通 Table Model 当前不支持配置 Aggregate Key 或独立 Unique Key；Unique Key Upsert 使用
Incremental `merge`。
@@ -394,23 +424,22 @@ where updated_at >= (
| `incremental_strategy` | 目标表和写入语义 | 主要配置 |
| --- | --- | --- |
-| `append` | 向 Duplicate Key 表追加数据，不去重 | 不需要 `unique_key` |
-| `merge` | 向 Merge-on-Write 或 Merge-on-Read Unique Key 表执行全行 `INSERT INTO`
Upsert | 必须设置 `unique_key` |
-| `insert_overwrite` | 使用 Doris `INSERT OVERWRITE` 覆盖整表、指定分区或本批涉及的分区 | 不能设置
`unique_key`；可设置 `overwrite_partitions` |
-| `microbatch` | 按 dbt Core 的 UTC 时间窗口逐批覆盖一个精确的 Doris RANGE 分区 | 设置
`event_time`、`batch_size` 和 `begin`；不能设置 `unique_key` |
+| `append` | 对 OLAP Duplicate Key 表或可写 Iceberg 表执行 `INSERT INTO` | 不设置
`unique_key` |
+| `merge` | OLAP Unique Key 表使用全行 `INSERT INTO` Upsert；Iceberg 表使用原生 `MERGE
INTO` | 必须设置 `unique_key`；Iceberg 还要求原生 MERGE 能力和 V2+ 表 |
+| `insert_overwrite` | OLAP 可覆盖整表、指定分区或动态分区；Iceberg 可覆盖整表、动态分区或静态分区 | 不允许
`unique_key`；Iceberg 静态范围使用字段和值的映射 |
+| `microbatch` | OLAP 覆盖精确 RANGE 分区；Iceberg 使用原生 MERGE 替换一个 UTC 时间窗口 | 设置
`event_time`、`batch_size` 和 `begin`；Iceberg 要求原生 MERGE 和 V2+ |
未设置 `incremental_strategy` 时，有 `unique_key` 使用 `merge`，否则使用
`append`。Adapter 支持 `ignore`、`fail`、`append_new_columns` 和
`sync_all_columns` 四种 `on_schema_change` 策略。
`delete+insert` 和 `delete_insert` 已被当前实现明确拒绝；Doris Unique Key Upsert
-应使用 `merge`。`merge` 不生成原生 `MERGE INTO`，而是依赖 Doris Unique Key
-表的全行 `INSERT INTO` 语义。每批数据中的 `unique_key` 必须唯一。需要 Sequence
+应使用 `merge`。对 OLAP 目标，`merge` 依赖 Doris Unique Key 表的全行 `INSERT INTO` 语义。对
Iceberg 目标，`merge` 会生成原生 `MERGE INTO`，并在写入前拒绝源数据中重复的 Key。每批数据中的 `unique_key`
必须唯一。需要 Sequence
Column 时，在 `properties` 中设置可见列
`function_column.sequence_col`，并继续使用 `merge`。裸 `sequence_col` 配置和使用隐藏
`__DORIS_SEQUENCE_COL__` 的 `function_column.sequence_type` 当前均不受支持。
-覆盖指定 Doris 分区：
+覆盖指定的 Internal Catalog Doris 分区：
```sql
{{
@@ -452,8 +481,9 @@ where order_time >= '2026-07-01'
首次运行走 CTAS 建表路径，不执行 `INSERT OVERWRITE`，所以示例必须用
`partition_by_init` 创建目标分区。`overwrite_partitions` 只决定目标表已存在时的覆盖
范围。省略 `overwrite_partitions` 会覆盖整张表；设置分区名列表会只覆盖这些分区；
-设置 `'*'` 时，Doris 根据本批数据动态覆盖涉及的分区。空批次
-无法识别需要清空的分区；需要清空分区时应显式指定分区名。
+设置 `'*'` 时，Doris 根据本批数据动态覆盖涉及的分区。OLAP 动态覆盖无法从空批次识别需要清空的分区；需要清空分区时应显式指定分区名。
+
+对于 Iceberg 目标，省略 `overwrite_partitions` 可动态覆盖本批涉及的分区。设置非空字段和值映射，例如
`overwrite_partitions={'dt': '2025-01-25'}`，可替换一个静态范围。Model
必须返回该范围内的完整数据。对于非分区表，Model 必须返回完整替换结果；空结果会清空整表。静态范围的空结果也会清空该范围。映射中包含 `NULL`
时使用原生 MERGE，因此需要 Iceberg V2+。
### Microbatch
@@ -499,7 +529,7 @@ select
from {{ source('raw', 'orders') }}
```
-`event_time` 必须是未加引号的列名，并与 `partition_by` 指向同一个单列；
+对于 Internal Catalog OLAP 目标，`event_time` 必须是未加引号的列名，并与 `partition_by` 指向同一个单列；
`batch_size` 支持 `hour`、
`day`、`month` 和 `year`。目标是 Duplicate Key 表；`duplicate_key` 是 Doris 表的
Key 配置，不是 dbt 的 `unique_key`。不要为 Microbatch 设置 `unique_key`、
@@ -508,7 +538,7 @@ Key 配置，不是 dbt 的 `unique_key`。不要为 Microbatch 设置 `unique_k
当前批次窗口过滤输入。Microbatch 批次串行执行；即使遇到空批次，也会覆盖对应的
精确分区，因此可以清除该窗口中已不存在的数据。
-Microbatch 也可使用 Doris Dynamic Partition，但必须在 `properties` 中显式启用并
+Internal Catalog 中的 Microbatch 也可使用 Doris Dynamic Partition，但必须在 `properties`
中显式启用并
满足当前 Adapter 的校验：
- `dynamic_partition.enable='true'`；
@@ -530,7 +560,7 @@ Doris 创建 Dynamic Partition 表时还要求
`dynamic_partition.buckets` 为
目标表所需的完整数据。同一批 `merge` 数据中的 `unique_key` 必须唯一。Microbatch
的时间过滤由 dbt Core 根据已配置 `event_time` 的上游资源生成，不需要使用
`is_incremental()`。修改 Key、分区方式或其他不兼容表结构后，应执行
-`dbt run --full-refresh`。
+`dbt run --full-refresh`。对于 Iceberg Microbatch，不要配置 OLAP Dynamic Partition
属性、具名 `overwrite_partitions` 或 `partition_by_init`；每个 UTC 时间窗口由原生 MERGE
替换，包括空窗口。
:::
### 异步物化视图
@@ -635,15 +665,15 @@ Model SQL 或 DDL 配置变化时，`on_configuration_change=apply` 默认通过
| 功能 | 用法 |
| --- | --- |
| Seed | 将小型、版本受控的 CSV 放入 `seeds/`，执行 `dbt seed`；支持类型推断、`column_types` 和 `ref`
|
-| Snapshot | 在 `snapshots/` 中定义 `check` 或 `timestamp` 策略，执行 `dbt snapshot`；支持
Hard Delete、Schema 演进和原子替换 |
+| Snapshot | 在 `snapshots/` 中定义 `check` 或 `timestamp` 策略，执行 `dbt snapshot`；支持
Hard Delete 和 Schema 演进。Internal Catalog 原子替换，Iceberg 使用备份和重命名 |
| Data Test | 在 YAML 中配置 Generic Test，或在 `tests/` 中编写 Singular Test；支持
Ephemeral 和 `store_failures` 路径 |
| Unit Test | 在 Model YAML 中提供 Inline Row 或 CSV Fixture，执行 `dbt test --select
test_type:unit` |
-| Model Contract | 对 Table、View 或 Incremental Model 设置 `contract.enforced:
true` 并声明字段名称和类型；不会创建数据库 PK 或 NOT NULL 约束 |
+| Model Contract | 对 Table、View 或 Incremental Model 设置 `contract.enforced:
true` 并声明字段名称和类型；Adapter 拒绝 `constraints` 声明，Data Test 不会创建数据库约束 |
| Persisted Docs | 启用 `persist_docs`，为主要 Relation 类型写入 Relation 和字段注释 |
| Source Freshness | 配置 `loaded_at_field` 或 `loaded_at_query` 以及
`freshness`，执行 `dbt source freshness` |
| Grants | 使用标准 `grants` 配置管理 Doris `user` 和 `user@host` 表权限；当前不支持 Role 授权主体 |
| Hooks | 使用 `pre_hook` 和 `post_hook` 在 Materialization 前后执行 Doris SQL；Hook
副作用没有事务回滚 |
-| 文档与血缘 | 编写 Description 并执行 `dbt docs generate`，生成 Doris
Database、Table、View、字段、注释和异步物化视图元数据 |
+| 文档与血缘 | 编写 Description 并执行 `dbt docs generate`，获取 Internal 和 External
Catalog 的 Relation 与字段元数据，包括 Internal Catalog 异步物化视图 |
Seed 适合国家代码、状态映射等小型静态数据，不适合批量导入业务数据。大文件应使用
Doris Stream Load、Broker Load 或 Doris Connector。
@@ -768,12 +798,12 @@ dbt run --select fct_orders --full-refresh
## 当前限制
- 当前不支持 Aggregate Key 表建模和 Secondary Index 配置。
-- 当前不支持完整的 External Catalog 命名空间。
+- External Catalog 的写入行为取决于 Doris Connector。当前不支持 Catalogs V2 或针对各 Connector
的写入保证。
- 当前未实现 SSL 配置、超时与重试、多 FE 故障转移、服务端取消和完整的查询遥测。
- 部分 Table、View 和 MV 类型切换存在短暂的标准对象名不可用窗口，无法做到零停机切换。
<!-- 知识类型: 故障排查 -->
-<!-- 适用场景: 连接失败 / Profile database 与 schema 不一致 -->
+<!-- 适用场景: 连接失败 / Catalog 与 Database 查找 -->
## 故障处理
@@ -782,8 +812,6 @@ dbt run --select fct_orders --full-refresh
确认 `host` 和 `port` 指向可访问的 Doris FE Query Port，并检查用户名、密码和
网络策略。`9030` 是默认 Query Port，`8030` 通常是 FE HTTP Port。
-### `database` 与 `schema` 不一致
+### 找不到 Catalog 或 Database
-Profile 中的 `schema` 就是目标 Doris Database。删除 `database`；它不是 Doris
-Catalog，而且在 Profile 中没有提供额外命名层级。如果保留该兼容字段，当前实现
-要求它与 `schema` 完全相同。
+在 v1.1.1 中，Profile 和 Source 的 `database` 表示 Doris Catalog，`schema` 表示其中的
Database。使用 Internal Catalog 时省略 `database`。如果旧项目曾用 Source 的 `database` 表示
Doris Database，升级前应将该值移到 `schema`。检查 External Catalog 是否已创建，以及 dbt 用户是否有权限查看。
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/connection-integration/data-integration/intro.mdx
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/connection-integration/data-integration/intro.mdx
index a81f17aaebb..1f7ef4fb6f4 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/connection-integration/data-integration/intro.mdx
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/connection-integration/data-integration/intro.mdx
@@ -239,9 +239,9 @@ Apache Doris 支持与多类数据生态工具集成。您可以按照使用场
<div className="cards-grid">
<GettingStartedCard
- title="DBT Doris Adapter"
+ title="dbt Adapter"
icon="/images/next/connection-integration/data-integration/logos/dbt.svg"
- description="使用 dbt-doris adapter 在 Doris 中运行 dbt 模型，完成 ELT 转换和数据建模"
+ description="使用 dbt-for-apache-doris 在 Doris 中运行 dbt 模型，完成 ELT 转换和数据建模"
link="./dbt-doris-adapter"
/>
diff --git
a/versioned_docs/version-4.x/connection-integration/data-integration/dbt-doris-adapter.md
b/versioned_docs/version-4.x/connection-integration/data-integration/dbt-doris-adapter.md
index 9e7414445a9..205cd1dc01a 100644
---
a/versioned_docs/version-4.x/connection-integration/data-integration/dbt-doris-adapter.md
+++
b/versioned_docs/version-4.x/connection-integration/data-integration/dbt-doris-adapter.md
@@ -1,8 +1,9 @@
---
{
- "title": "Apache Doris dbt Adapter",
+ "title": "dbt Adapter",
+ "sidebar_label": "dbt Adapter",
"language": "en",
- "description": "Install and configure dbt-for-apache-doris for Apache
Doris models, source freshness, and asynchronous materialized views.",
+ "description": "Install dbt-for-apache-doris 1.1.1 and configure models,
External Catalog sources, Iceberg targets, and asynchronous materialized
views.",
"keywords": [
"Apache Doris dbt",
"dbt Doris Adapter",
@@ -41,11 +42,13 @@ Source data must already be loaded into Doris. The adapter
performs transformati
| Component | Requirement |
| --- | --- |
| Python | 3.10 or later |
+| dbt Core | 1.12.x (installed with the adapter) |
## Released versions
| Version |
| --- |
+| [1.1.1](https://github.com/velodb/dbt-for-apache-doris/releases/tag/v1.1.1) |
| [1.1.0](https://github.com/velodb/dbt-for-apache-doris/releases/tag/v1.1.0) |
<!-- Knowledge Type: Operational Steps -->
@@ -53,7 +56,7 @@ Source data must already be loaded into Doris. The adapter
performs transformati
## Installation
-The following commands use the published [`v1.1.0`
release](https://github.com/velodb/dbt-for-apache-doris/releases/tag/v1.1.0) as
an installation example. To install another release, replace the version in the
`pip` requirement.
+The following commands use the published [`v1.1.1`
release](https://github.com/velodb/dbt-for-apache-doris/releases/tag/v1.1.1) as
an installation example. To install another release, replace the version in the
`pip` requirement.
Install the adapter from PyPI in an isolated Python environment:
@@ -61,7 +64,7 @@ Install the adapter from PyPI in an isolated Python
environment:
python3 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
-python -m pip install "dbt-for-apache-doris==1.1.0"
+python -m pip install "dbt-for-apache-doris==1.1.1"
dbt --version
```
@@ -71,11 +74,11 @@ On Windows PowerShell:
py -m venv .venv
.\.venv\Scripts\Activate.ps1
python -m pip install --upgrade pip
-python -m pip install "dbt-for-apache-doris==1.1.0"
+python -m pip install "dbt-for-apache-doris==1.1.1"
dbt --version
```
-For this example, the `Plugins` section of `dbt --version` should contain
`doris: 1.1.0`. dbt Core and MySQL Connector/Python are installed as
dependencies.
+For this example, the `Plugins` section of `dbt --version` should contain
`doris: 1.1.1`. dbt Core and MySQL Connector/Python are installed as
dependencies.
<!-- Knowledge Type: Operational Steps / Minimal Example -->
<!-- Use Case: Connect dbt to Doris / Validate an end-to-end model build -->
@@ -114,17 +117,9 @@ doris_demo:
| `password` | Read from an environment variable | Defaults to an empty string
|
| `schema` | Set the target Doris database | The `dbt init` prompt defaults to
`dbt`; the underlying credentials default to `None` |
| `threads` | Set according to FE and workload capacity | The `dbt init`
prompt defaults to `1` |
-| `database` | Omit it | If set, it must exactly equal `schema` |
+| `database` | Omit for Internal Catalog; set to an existing Doris Catalog for
an external target | Maps to the Doris Catalog in v1.1.1 |
-The adapter maps a dbt schema to a Doris database. If the target database does
not exist, the adapter attempts to create it. The Doris account used by dbt
therefore needs permission to read source tables and to create, alter, drop,
and write objects in the target database.
-
-Here, `database` is not a Doris catalog. It is a field inherited from dbt's
generic `database.schema.identifier` naming model. Doris Internal Catalog uses
only `database.table`, so set the Doris database in profile `schema` and omit
profile `database`. Source `database` has a different compatibility behavior:
it can override `schema` as the Doris database for cross-database reads.
Profiles and standard `source()` cannot represent both a Doris catalog and
database level.
-
-:::caution
-
-The adapter currently does not support External Catalog three-part namespaces
using Catalog, Database, and Table.
-
-:::
+In v1.1.1, dbt `database.schema.identifier` maps to Doris
`catalog.database.table`. For Internal Catalog, omit `database` and set
`schema` to the Doris database. For an external target, set `database` to an
existing Doris Catalog and `schema` to a database in that Catalog. The adapter
can attempt to create a missing target database if the Catalog supports it. The
dbt account needs permission to read sources and to create, alter, drop, and
write target objects. External Catalog write supp [...]
Ensure that `dbt_project.yml` references the same profile:
@@ -253,7 +248,52 @@ sources:
- name: payments
```
-For compatibility with existing dbt projects, a source may instead set only
`database: finance_raw`, or set `database` and `schema` to the same value. If
source `database` and `schema` differ, the adapter uses `database` as the Doris
database. This differs from profile validation, which requires both values to
match. New projects should consistently use source `schema` to avoid ambiguity.
+In v1.1.1, a source's `database` names a Doris Catalog. To read from another
database in Internal Catalog, set only `schema`, as above. Projects that used
`database: finance_raw` for cross-database sources in v1.1.0 must move that
value to `schema` before upgrading.
+
+## External Catalogs in v1.1.1
+
+The adapter can read sources in External Catalogs and discover their tables
and columns for `dbt docs generate`. Create the Catalog in Doris first, then
specify its name in the source's `database` field:
+
+```yaml
+sources:
+ - name: lakehouse
+ database: hive_catalog
+ schema: ods
+ tables:
+ - name: orders
+```
+
+`{{ source('lakehouse', 'orders') }}` resolves to `hive_catalog.ods.orders`.
The `schema` field names the database inside that Catalog. The adapter supports
catalog-qualified relation names; each Catalog connector still determines which
DDL and writes Doris can perform.
+
+To create an Iceberg target, first [configure a writable Iceberg
Catalog](../../lakehouse/catalogs/iceberg-catalog). Set these fields in the
existing profile output:
+
+```yaml
+database: iceberg_catalog
+schema: analytics
+```
+
+Create `models/iceberg_sample.sql`:
+
+```sql
+{{ config(materialized='table', engine='iceberg') }}
+
+select 1 as id
+```
+
+Run `dbt run --select iceberg_sample`. It creates
`iceberg_catalog.analytics.iceberg_sample`. The Catalog must exist; its
database must exist or be creatable by dbt, and the target must be writable.
`engine='iceberg'` emits an explicit engine clause; omitting `engine` lets
Doris infer it from the target Catalog. Catalog-based inference was verified
with Doris 4.1.3. Earlier Doris releases may require the explicit engine.
+
+| Iceberg target | v1.1.1 behavior |
+| --- | --- |
+| `table` | Creates an Iceberg table; reruns replace it through backup and
rename |
+| `incremental` | Supports `append`, `merge`, `insert_overwrite`, and
`microbatch` as described below |
+| Seed and Snapshot | Supported; Snapshot publication and Seed full refresh
use backup and rename. Ordinary Iceberg Seed reload uses overwrite or MERGE |
+| `view` and `materialized_view` | Targets must be in Internal Catalog;
Internal Catalog views and MVs can query External Catalog sources |
+
+Iceberg `merge` and `microbatch` require a Doris release with native Iceberg
`MERGE INTO` and an Iceberg V2 or later table. These paths were verified on
Doris 4.1.3; [Doris documents Iceberg MERGE as experimental since
4.1.0](../../lakehouse/catalogs/iceberg-catalog#merge-into). An external table
replacement is not atomic: its name is briefly unavailable between renames, and
a failure can leave the old table under a backup name. Use an isolated test
project to validate your Doris release [...]
+
+The adapter rejects model or column `constraints` declarations before writing
a Table, View, Incremental, or MV target. Model contracts still check column
names and types. Use `data_tests` such as `not_null` to check data after
writing; these do not create database-enforced constraints.
+
+The [five runnable Doris
demos](https://github.com/velodb/dbt-for-apache-doris/tree/v1.1.1/examples/doris-demos)
cover models, tests, cross-database sources, incremental processing,
snapshots, and asynchronous materialized views.
<!-- Knowledge Type: Capability Reference / Configuration Parameters /
Architecture Decision -->
<!-- Use Case: Materialization selection / Incremental processing /
Materialized views -->
@@ -264,10 +304,10 @@ For compatibility with existing dbt projects, a source
may instead set only `dat
| Materialization | Doris object or behavior | Use case |
| --- | --- | --- |
-| `view` | Doris view | Lightweight transformations that should always read
current source data |
-| `table` | A fully rebuilt Doris Duplicate Key table | Bounded results that
need stable query performance |
+| `view` | Internal Catalog Doris view | Lightweight transformations that
should always read current source data |
+| `table` | A fully rebuilt Internal Catalog Duplicate Key table or writable
Iceberg table | Bounded results that need stable query performance |
| `incremental` | Appends, upserts, or overwrites data according to a strategy
| Incremental processing for large tables |
-| `materialized_view` | Doris asynchronous materialized view | Doris-managed
refresh and transparent query rewrite |
+| `materialized_view` | Internal Catalog Doris asynchronous materialized view
| Doris-managed refresh and transparent query rewrite |
| `ephemeral` | Compiles into a CTE in downstream models | Reuse SQL without
creating a Doris object |
Seeds and snapshots can also create Doris tables. The adapter currently does
not include a custom `partition` materialization; use incremental
`insert_overwrite` for partition replacement. The examples below keep
`replication_num=1` so they also run on a single-BE development cluster; use a
production-appropriate replica count in real deployments.
@@ -311,7 +351,7 @@ select
from {{ source('raw', 'orders') }}
```
-The `table` materialization creates a Duplicate Key table and fully replaces
the target on subsequent runs. Table models accept the following table-creation
settings:
+For Internal Catalog targets, the `table` materialization creates a Duplicate
Key table and fully replaces it on subsequent runs. Iceberg targets do not use
Doris OLAP keys or default distribution. Table models accept the following
table-creation settings:
| Setting | Type and default | Description |
| --- | --- | --- |
@@ -324,7 +364,7 @@ The `table` materialization creates a Duplicate Key table
and fully replaces the
| `replication_num` | Positive integer or numeric string; optional | Replica
count; the top-level setting overrides the same key in `properties` |
| `properties` | Dictionary; optional | Key-value pairs rendered into Doris
`PROPERTIES` |
-Duplicate Key, partition, and distribution columns must be present in the
model output and comply with Doris table-creation rules. The adapter renders
`partition_by_init` and `properties` into the `CREATE TABLE` statement; Doris
validates the partition definitions, property names, and values. An ordinary
`table` model currently does not expose Aggregate Key or standalone Unique Key
configuration. Use incremental `merge` for Unique Key upserts.
+Duplicate Key, partition, and distribution columns must be present in the
model output and comply with Doris table-creation rules. The adapter renders
`partition_by_init` and `properties` into the `CREATE TABLE` statement; Doris
validates the partition definitions, property names, and values. OLAP-specific
options such as `duplicate_key` and `distributed_by` are not suitable for
Iceberg targets. An ordinary `table` model currently does not expose Aggregate
Key or standalone Unique Key co [...]
### Incremental
@@ -337,14 +377,14 @@ Microbatch does not use this filtering pattern. dbt Core
injects time filters fr
| Strategy | Doris target and behavior | Main requirements |
| --- | --- | --- |
-| `append` | Executes `INSERT INTO` against a Duplicate Key table | Do not set
`unique_key` |
-| `merge` | Performs full-row `INSERT INTO` upserts against a MOW or MOR
Unique Key table | `unique_key` is required |
-| `insert_overwrite` | Overwrites the whole table, named partitions, or
partitions touched by the batch | `unique_key` is rejected |
-| `microbatch` | Overwrites one exact RANGE partition for each dbt Core UTC
window | Configure `event_time`, `batch_size`, and `begin` |
+| `append` | `INSERT INTO` on an OLAP Duplicate Key or writable Iceberg table
| Do not set `unique_key` |
+| `merge` | Full-row `INSERT INTO` upsert on an OLAP Unique Key table; native
`MERGE INTO` on Iceberg | Requires `unique_key`; Iceberg also requires native
MERGE support and V2+ tables |
+| `insert_overwrite` | OLAP whole-table, named or dynamic partition overwrite;
Iceberg whole-table, dynamic or static partition overwrite | Rejects
`unique_key`; Iceberg static scope uses a column/value mapping |
+| `microbatch` | OLAP exact RANGE-partition overwrite; Iceberg replaces one
UTC window with native MERGE | Configure `event_time`, `batch_size`, and
`begin`; Iceberg requires native MERGE and V2+ |
When no strategy is set, `unique_key` selects `merge`; otherwise the adapter
uses `append`. The adapter supports all four `on_schema_change` modes:
`ignore`, `fail`, `append_new_columns`, and `sync_all_columns`.
-`delete+insert` and `delete_insert` are rejected. Use `merge` for Doris Unique
Key upserts. `merge` does not generate SQL `MERGE INTO`; it uses full-row
`INSERT INTO` with Doris Unique Key semantics. Each batch must contain at most
one row for each `unique_key`. To use a Sequence Column, set the visible column
through `function_column.sequence_col` in `properties` and continue using
`merge`. The bare `sequence_col` setting and `function_column.sequence_type`
with the hidden `__DORIS_SEQU [...]
+`delete+insert` and `delete_insert` are rejected. Use `merge` for Doris Unique
Key upserts. On OLAP targets, `merge` uses full-row `INSERT INTO` with Doris
Unique Key semantics. On Iceberg targets, it generates native `MERGE INTO` and
rejects duplicate source keys before writing. Each batch must contain at most
one row for each `unique_key`. To use a Sequence Column, set the visible column
through `function_column.sequence_col` in `properties` and continue using
`merge`. The bare `sequen [...]
The following example performs a full-row upsert by `order_id`:
@@ -376,7 +416,7 @@ where updated_at >= (
{% endif %}
```
-To overwrite named Doris partitions:
+To overwrite named Internal Catalog Doris partitions:
```sql
{{
@@ -421,7 +461,9 @@ The first run creates the target through CTAS and does not
execute `INSERT OVERW
- List of partition names: overwrite only those partitions.
- `'*'`: let Doris dynamically overwrite partitions touched by the batch.
-A dynamic overwrite cannot infer which partition to clear from an empty batch.
Specify partition names when an empty source batch must clear target data.
+A dynamic OLAP overwrite cannot infer which partition to clear from an empty
batch. Specify partition names when an empty source batch must clear target
data.
+
+For Iceberg targets, omit `overwrite_partitions` to use native dynamic
overwrite on partitioned tables. Set a non-empty column/value mapping, such as
`overwrite_partitions={'dt': '2025-01-25'}`, to replace one static scope. The
model must return all rows in that scope. On an unpartitioned target, the model
must return the complete replacement result; an empty result clears the table.
An empty result for a static scope also clears that scope. A mapping containing
`NULL` uses native MERGE [...]
### Microbatch
@@ -461,9 +503,9 @@ select
from {{ source('raw', 'orders') }}
```
-`event_time` must be an unquoted column name and must refer to the same single
column as `partition_by`. `batch_size` supports `hour`, `day`, `month`, and
`year`. The target is a Duplicate Key table: `duplicate_key` configures the
Doris table key and is not dbt `unique_key`. Do not set `unique_key`,
`overwrite_partitions`, or `partition_by_init`; the adapter derives the
partition boundary and overwrite target from the current batch. If an upstream
`ref()` or `source()` also configures `e [...]
+For an Internal Catalog OLAP target, `event_time` must be an unquoted column
name and must refer to the same single column as `partition_by`. `batch_size`
supports `hour`, `day`, `month`, and `year`. The target is a Duplicate Key
table: `duplicate_key` configures the Doris table key and is not dbt
`unique_key`. Do not set `unique_key`, `overwrite_partitions`, or
`partition_by_init`; the adapter derives the partition boundary and overwrite
target from the current batch. If an upstream `re [...]
-Microbatch can also use Doris Dynamic Partition when the following properties
are configured and pass adapter validation:
+Internal Catalog Microbatch can also use Doris Dynamic Partition when the
following properties are configured and pass adapter validation:
- `dynamic_partition.enable='true'`.
- `dynamic_partition.time_unit` matches `batch_size`, and
`dynamic_partition.time_zone` is `UTC`, `Etc/UTC`, or `+00:00`.
@@ -475,7 +517,7 @@ Doris also requires a positive `dynamic_partition.buckets`
value when creating a
:::caution
-For `append`, `merge`, and partition-scoped `insert_overwrite`, the adapter
does not discover new or changed source rows automatically. The model SQL must
return the intended batch. Whole-table `insert_overwrite` must return the
complete target data. Each `merge` batch must contain unique `unique_key`
values. dbt Core supplies Microbatch filtering from upstream resources
configured with `event_time`, so Microbatch models do not need
`is_incremental()`. Run `dbt run --full-refresh` after [...]
+For `append`, `merge`, and partition-scoped `insert_overwrite`, the adapter
does not discover new or changed source rows automatically. The model SQL must
return the intended batch. Whole-table `insert_overwrite` must return the
complete target data. Each `merge` batch must contain unique `unique_key`
values. dbt Core supplies Microbatch filtering from upstream resources
configured with `event_time`, so Microbatch models do not need
`is_incremental()`. Run `dbt run --full-refresh` after [...]
:::
@@ -567,15 +609,15 @@ When model SQL or DDL configuration changes, the default
`on_configuration_chang
| Capability | Support |
| --- | --- |
| Seed | Put small, version-controlled CSV files in `seeds/`, then run `dbt
seed`; supports type inference, `column_types`, and `ref` |
-| Snapshot | Define `check` or `timestamp` strategies in `snapshots/`, then
run `dbt snapshot`; supports hard deletes, schema evolution, and atomic
replacement |
+| Snapshot | Define `check` or `timestamp` strategies in `snapshots/`, then
run `dbt snapshot`; supports hard deletes and schema evolution. Internal
Catalog replacement is atomic; Iceberg replacement uses backup and rename |
| Data tests | Configure Generic Tests in YAML or write Singular Tests in
`tests/`; supports ephemeral and `store_failures` paths |
| Unit tests | Provide inline-row or CSV fixtures in model YAML, then run `dbt
test --select test_type:unit` |
-| Model contracts | Set `contract.enforced: true` and declare column names and
types for Table, View, or Incremental models; does not create database PK or
NOT NULL constraints |
+| Model contracts | Set `contract.enforced: true` and declare column names and
types for Table, View, or Incremental models; `constraints` declarations are
rejected, and data tests do not create database constraints |
| Persisted docs | Enable `persist_docs` for relation and column comments on
primary relation types |
| Source freshness | Configure `loaded_at_field` or `loaded_at_query` with
`freshness`, then run `dbt source freshness` |
| Grants | Use standard `grants` configuration for Doris `user` and
`user@host` table privileges; role principals are currently not supported |
| Hooks | Use `pre_hook` and `post_hook` to execute Doris SQL around
materializations; hook side effects have no transactional rollback |
-| Documentation and lineage | Write descriptions and run `dbt docs generate`
for metadata about Doris databases, tables, views, columns, comments, and
asynchronous materialized views |
+| Documentation and lineage | Write descriptions and run `dbt docs generate`
for Internal and External Catalog relation and column metadata, including
Internal Catalog asynchronous materialized views |
Seeds are suitable for small, version-controlled data such as country codes or
status mappings, not bulk business-data ingestion. Use Doris Stream Load,
Broker Load, or a Doris Connector for large files.
@@ -692,12 +734,12 @@ dbt run --select fct_orders --full-refresh
## Current limitations
- The adapter currently does not support Aggregate Key table modeling or
secondary-index configuration.
-- The adapter currently does not support a complete External Catalog namespace.
+- External Catalog write behavior depends on the Doris connector. Catalogs V2
and connector-specific write guarantees are not supported.
- The adapter currently does not implement SSL configuration, timeout/retry,
multi-FE failover, server-side cancellation, or complete query telemetry.
- Some Table/View/MV type changes have a short canonical-name availability
window rather than a zero-downtime switch.
<!-- Knowledge Type: Troubleshooting -->
-<!-- Use Case: Connection errors / Profile database and schema mismatch -->
+<!-- Use Case: Connection errors / Catalog and database lookup -->
## Troubleshooting
@@ -705,6 +747,6 @@ dbt run --select fct_orders --full-refresh
Verify that `host` and `port` point to a reachable FE MySQL query port, and
check the username, password, and network policy. The default query port is
`9030`; `8030` is normally the FE HTTP port.
-### `database` and `schema` differ
+### Catalog or database cannot be found
-Profile `schema` is the target Doris database. Remove `database`; it is not a
Doris catalog and does not add another namespace level. If you retain this
compatibility field, set it to exactly the same value as `schema`.
+In v1.1.1, profile and source `database` values identify a Doris Catalog, and
`schema` identifies a database inside it. Omit `database` for Internal Catalog.
If an old project used source `database` for a Doris database, move that value
to `schema`. Check that external Catalogs already exist and are visible to the
dbt account.
diff --git
a/versioned_docs/version-4.x/connection-integration/data-integration/intro.mdx
b/versioned_docs/version-4.x/connection-integration/data-integration/intro.mdx
index d2e2bf448e3..fe178e34d72 100644
---
a/versioned_docs/version-4.x/connection-integration/data-integration/intro.mdx
+++
b/versioned_docs/version-4.x/connection-integration/data-integration/intro.mdx
@@ -239,9 +239,9 @@ Apache Doris supports integration with a wide range of data
ecosystem tools. You
<div className="cards-grid">
<GettingStartedCard
- title="DBT Doris Adapter"
+ title="dbt Adapter"
icon="/images/next/connection-integration/data-integration/logos/dbt.svg"
- description="Use the dbt-doris adapter to run dbt models in Doris for ELT
transformations and data modeling"
+ description="Use dbt-for-apache-doris to run dbt models in Doris for ELT
transformations and data modeling"
link="./dbt-doris-adapter"
/>
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
