milenkovicm commented on PR #1590:
URL:
https://github.com/apache/datafusion-ballista/pull/1590#issuecomment-4338476636
we need to make mapping between `DataFrameWriteOptions` and
`DataFrameWriteOptionsInternal` similar to
```python
options_internal = ParquetWriterOptionsInternal(
options.data_pagesize_limit,
options.write_batch_size,
options.writer_version,
options.skip_arrow_metadata,
options.compression,
options.dictionary_enabled,
options.dictionary_page_size_limit,
options.statistics_enabled,
options.max_row_group_size,
# trimmed ...
)
```
`DataFrameWriteOptions` is defined as
```python
class DataFrameWriteOptions:
"""Writer options for DataFrame.
There is no guarantee the table provider supports all writer options.
See the individual implementation and documentation for details.
"""
def __init__(
self,
insert_operation: InsertOp | None = None,
single_file_output: bool = False,
partition_by: str | Sequence[str] | None = None,
sort_by: Expr | SortExpr | Sequence[Expr] | Sequence[SortExpr] |
None = None,
) -> None:
# trimmed ...
```
the only problem might be `sort_by: Expr | SortExpr | Sequence[Expr] |
Sequence[SortExpr] | None = None,`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
