Re: Most effective and fast way to load few Tbyte of data from flat files into postgresql

2020-08-27 Thread Shaozhong SHI
On Tue, 25 Aug 2020 at 12:24, Peter J. Holzer wrote: > On 2020-08-24 21:17:36 +, Dirk Krautschick wrote: > > what would be the fastest or most effective way to load few (5-10) TB > > of data from flat files into a postgresql database, including some 1TB > > tables and blobs? > > > > There is

Re: [E] Re: Most effective and fast way to load few Tbyte of data from flat files into postgresql

2020-08-25 Thread Saha, Sushanta K
You can explore "pgloader" also. Sushanta On Tue, Aug 25, 2020 at 7:24 AM Peter J. Holzer wrote: > On 2020-08-24 21:17:36 +, Dirk Krautschick wrote: > > what would be the fastest or most effective way to load few (5-10) TB > > of data from flat files into a postgresql database, includ

Re: Most effective and fast way to load few Tbyte of data from flat files into postgresql

2020-08-25 Thread Peter J. Holzer
On 2020-08-24 21:17:36 +, Dirk Krautschick wrote: > what would be the fastest or most effective way to load few (5-10) TB > of data from flat files into a postgresql database, including some 1TB > tables and blobs? > > There is the copy command but there is no way for native parallelism, > rig

Most effective and fast way to load few Tbyte of data from flat files into postgresql

2020-08-24 Thread Dirk Krautschick
Hi, what would be the fastest or most effective way to load few (5-10) TB of data from flat files into a postgresql database, including some 1TB tables and blobs? There is the copy command but there is no way for native parallelism, right? I have found pg_bulkload but haven't tested it yet. As