GitHub user avamingli added a comment to the discussion: Extend the gpfdist
tool to support SFTP/HDFS protocols for high-performance multi-source data
ingestion
Hi, interesting!
As far as I know, gpfdist is a protocol for external tables, and the gpfdist
tool is an implementation of that protocol. Other tools, like GPSS (Greenplum
Streaming Server), also implement the gpfdist protocol for external tables. The
data format is specified by the format option when defining the external table.
```sql
CREATE EXTERNAL TABLE ext1 (d varchar(20)) location ('gpfdist://9727/d.dat')
format 'csv' (DELIMITER '|');
```
Are you planning to support additional protocols(ex: SFTP) for external tables
:
```sql
CREATE EXTERNAL TABLE ext1 (d varchar(20)) location ('SFTP://9727/d.dat')
```
or provide more format options?
```sql
CREATE EXTERNAL TABLE ext1 (d varchar(20)) location ('gpfdist://9727/d.dat')
format 'SFTP' ;
```
GitHub link:
https://github.com/apache/cloudberry/discussions/1205#discussioncomment-13636225
----
This is an automatically sent email for dev@cloudberry.apache.org.
To unsubscribe, please send an email to: dev-unsubscr...@cloudberry.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@cloudberry.apache.org
For additional commands, e-mail: dev-h...@cloudberry.apache.org
