This is an automated email from the ASF dual-hosted git repository.
curth pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/arrow-adbc.git
The following commit(s) were added to refs/heads/main by this push:
new 3c83c1596 fix(csharp/src/Drivers/Databricks): Fix Databricks readme
(#3365)
3c83c1596 is described below
commit 3c83c1596687553da5a0f68b19585c3285e445cc
Author: eric-wang-1990 <[email protected]>
AuthorDate: Thu Aug 28 11:04:54 2025 -0700
fix(csharp/src/Drivers/Databricks): Fix Databricks readme (#3365)
Fixing the Databricks Driver readme to list all the available config
properties customer can tune
---
csharp/src/Drivers/Databricks/readme.md | 101 +++++++++++++++++++++++++++-----
1 file changed, 85 insertions(+), 16 deletions(-)
diff --git a/csharp/src/Drivers/Databricks/readme.md
b/csharp/src/Drivers/Databricks/readme.md
index d7fe4843a..1255bf817 100644
--- a/csharp/src/Drivers/Databricks/readme.md
+++ b/csharp/src/Drivers/Databricks/readme.md
@@ -17,33 +17,102 @@
-->
-# Databricks
+# Databricks Driver
-The Databricks ADBC driver is built on top of the Spark ADBC driver and
inherits many of it's [properties](../Apache/Spark/readme.md).
+The Databricks ADBC driver is built on top of the Spark ADBC driver and
inherits all of its [properties](../Apache/Spark/readme.md), plus additional
Databricks-specific functionality.
+
+## Database and Connection Properties
+
+**Note**: The Databricks driver inherits all properties from the [Spark
driver](../Apache/Spark/readme.md). The properties below are
Databricks-specific additions.
+
+### Authentication Properties
+
+| Property | Description | Default |
+| :--- | :--- | :--- |
+| `adbc.databricks.oauth.grant_type` | The OAuth grant type. Supported values:
`access_token` (personal access token), `client_credentials` (OAuth client
credentials flow) | `access_token` |
+| `adbc.databricks.oauth.client_id` | The OAuth client ID (when using
`client_credentials` grant type) | |
+| `adbc.databricks.oauth.client_secret` | The OAuth client secret (when using
`client_credentials` grant type) | |
+| `adbc.databricks.oauth.scope` | The OAuth scope (when using
`client_credentials` grant type) | `sql` |
+| `adbc.databricks.token_renew_limit` | Minutes before token expiration to
start renewing the token. Set to 0 to disable automatic renewal | `0` |
+| `adbc.databricks.identity_federation_client_id` | The client ID of the
service principal when using workload identity federation | |
+
+### CloudFetch Properties
+
+CloudFetch is Databricks' high-performance result retrieval system that
downloads result data directly from cloud storage.
+
+| Property | Description | Default |
+| :--- | :--- | :--- |
+| `adbc.databricks.cloudfetch.enabled` | Whether to use CloudFetch for
retrieving results | `true` |
+| `adbc.databricks.cloudfetch.lz4.enabled` | Whether the client can decompress
LZ4 compressed results | `true` |
+| `adbc.databricks.cloudfetch.max_bytes_per_file` | Maximum bytes per file for
CloudFetch (e.g., `20971520` for 20MB) | `20971520` |
+| `adbc.databricks.cloudfetch.parallel_downloads` | Maximum number of parallel
downloads | `3` |
+| `adbc.databricks.cloudfetch.prefetch_count` | Number of files to prefetch |
`2` |
+| `adbc.databricks.cloudfetch.memory_buffer_size_mb` | Maximum memory buffer
size in MB for prefetched files | `200` |
+| `adbc.databricks.cloudfetch.prefetch_enabled` | Whether CloudFetch prefetch
functionality is enabled | `true` |
+| `adbc.databricks.cloudfetch.max_retries` | Maximum number of retry attempts
for downloads | `3` |
+| `adbc.databricks.cloudfetch.retry_delay_ms` | Delay in milliseconds between
retry attempts | `500` |
+| `adbc.databricks.cloudfetch.timeout_minutes` | Timeout in minutes for HTTP
operations | `5` |
+| `adbc.databricks.cloudfetch.url_expiration_buffer_seconds` | Buffer time in
seconds before URL expiration to trigger refresh | `60` |
+| `adbc.databricks.cloudfetch.max_url_refresh_attempts` | Maximum number of
URL refresh attempts | `3` |
+
+### Databricks-Specific Properties
+
+| Property | Description | Default |
+| :--- | :--- | :--- |
+| `adbc.connection.catalog` | Optional default catalog for the session | |
+| `adbc.connection.db_schema` | Optional default schema for the session | |
+| `adbc.databricks.enable_direct_results` | Whether to enable the use of
direct results when executing queries | `true` |
+| `adbc.databricks.apply_ssp_with_queries` | Whether to apply server-side
properties (SSP) with queries. If false, SSP will be applied when opening the
session | `false` |
+| `adbc.databricks.ssp_*` | Server-side properties prefix. Properties with
this prefix will be passed to the server by executing "set key=value" queries |
|
+| `adbc.databricks.enable_multiple_catalog_support` | Whether to use multiple
catalogs | `true` |
+| `adbc.databricks.enable_pk_fk` | Whether to enable primary key foreign key
metadata calls | `true` |
+| `adbc.databricks.use_desc_table_extended` | Whether to use DESC TABLE
EXTENDED to get extended column metadata when supported by DBR | `true` |
+| `adbc.databricks.enable_run_async_thrift` | Whether to enable RunAsync flag
in Thrift operations | `true` |
+| `adbc.databricks.driver_config_take_precedence` | Whether driver
configuration overrides passed-in properties during configuration merging |
`false` |
+
+### Tracing Properties
+
+| Property | Description | Default |
+| :--- | :--- | :--- |
+| `adbc.databricks.trace_propagation.enabled` | Whether to propagate trace
parent headers in HTTP requests | `true` |
+| `adbc.databricks.trace_propagation.header_name` | The name of the HTTP
header to use for trace parent propagation | `traceparent` |
+| `adbc.databricks.trace_propagation.state_enabled` | Whether to include trace
state header in HTTP requests | `false` |
+
+## Authentication Methods
The Databricks ADBC driver supports the following authentication methods:
-1. **Token-based authentication** using a [Databricks personal access
token](https://docs.databricks.com/en/dev-tools/auth/pat.html)
- - Set `adbc.spark.auth_type` to `oauth`
- - Set `adbc.databricks.oauth.grant_type` to `access_token` (this is the
default if not specified)
- - Set `adbc.spark.oauth.access_token` to your Databricks personal access
token
+### 1. Token-based Authentication
+Using a [Databricks personal access
token](https://docs.databricks.com/en/dev-tools/auth/pat.html):
-2. **OAuth Client Credentials Flow** for m2m authentication
- - Set `adbc.spark.auth_type` to `oauth`
- - Set `adbc.databricks.oauth.grant_type` to `client_credentials`
- - Set `adbc.databricks.oauth.client_id` to your OAuth client ID
- - Set `adbc.databricks.oauth.client_secret` to your OAuth client secret
- - Set `adbc.databricks.oauth.scope` to your auth scope (defaults to `"sql"`)
- - The driver will automatically handle token acquisition, renewal, and
authentication with the Databricks service
+- Set `adbc.spark.auth_type` to `oauth`
+- Set `adbc.databricks.oauth.grant_type` to `access_token` (this is the
default if not specified)
+- Set `adbc.spark.oauth.access_token` to your Databricks personal access token
-Basic (username and password) authentication is not supported at this time.
+### 2. OAuth Client Credentials Flow
+For machine-to-machine (m2m) authentication:
-Optional default catalog and default schema can be set for the session with
`adbc.connection.catalog` and `adbc.connection.db_schema`.
-The default catalog and schema will be used for subsequent metadata calls
unless user specified different catalog/schema to use.
+- Set `adbc.spark.auth_type` to `oauth`
+- Set `adbc.databricks.oauth.grant_type` to `client_credentials`
+- Set `adbc.databricks.oauth.client_id` to your OAuth client ID
+- Set `adbc.databricks.oauth.client_secret` to your OAuth client secret
+- Set `adbc.databricks.oauth.scope` to your auth scope (defaults to `sql`)
+
+The driver will automatically handle token acquisition, renewal, and
authentication with the Databricks service.
+
+**Note**: Basic (username and password) authentication is not supported at
this time.
+
+## Server-Side Properties
+
+Server-side properties allow you to configure Databricks session settings. Any
property with the `adbc.databricks.ssp_` prefix will be passed to the server by
executing `SET key=value` queries.
+
+For example, setting `adbc.databricks.ssp_use_cached_result` to `true` will
result in executing `SET use_cached_result=true` on the server when the session
is opened.
+
+The property name after the `ssp_` prefix becomes the server-side setting name.
## Data Types
