Async client libraries - not worth it?

2019-06-16 Thread Rob Nikander
Hi, I’m writing a new web app, and I’ve been experimenting with some async DB access libraries [1]. I also see some discussion online about a future Java standard to replace or supplement JDBC with an async API. While I understand the benefits of async in some situations, it seems to me that t

Re: arrays of composite types, and client drivers like JDBC

2019-06-16 Thread Rob Nikander
> On Jun 15, 2019, at 1:47 PM, Dave Cramer wrote: > > Basically because java would have to create a type dynamically to parse the > data into. > There's nothing inherently difficult about parsing the data, the problem is > what do we put it into ? (I accidentally replied off-list, so resend

Re: checkpoints taking much longer than expected

2019-06-16 Thread Stephen Frost
Greetings, * Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: > On 2019-Jun-16, Stephen Frost wrote: > > > The issue being discussed here is writing out to the heap files during a > > checkpoint... > > We don't really know, as it was already established that the log line is > misattributing time

Re: checkpoints taking much longer than expected

2019-06-16 Thread Alvaro Herrera
On 2019-Jun-16, Stephen Frost wrote: > The issue being discussed here is writing out to the heap files during a > checkpoint... We don't really know, as it was already established that the log line is misattributing time spent ... -- Álvaro Herrerahttps://www.2ndQuadrant.com/ Po

Re: checkpoints taking much longer than expected

2019-06-16 Thread Stephen Frost
Greetings, * Tiemen Ruiten (t.rui...@tech-lab.io) wrote: > On Sun, Jun 16, 2019 at 7:30 PM Stephen Frost wrote: > > Ok, so you want fewer checkpoints because you expect to failover to a > > replica rather than recover the primary on a failure. If you're doing > > synchronous replication, then th

Re: checkpoints taking much longer than expected

2019-06-16 Thread Stephen Frost
Greetings, * Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: > On 2019-Jun-16, Stephen Frost wrote: > > Not likely to help with what you're experiencing anyway though... > > My gut feeling is that you're wrong, since (as I understand) the > symptoms are the same. The issue in the linked-to thre

Re: checkpoints taking much longer than expected

2019-06-16 Thread Tiemen Ruiten
On Sun, Jun 16, 2019 at 7:30 PM Stephen Frost wrote: > Ok, so you want fewer checkpoints because you expect to failover to a > replica rather than recover the primary on a failure. If you're doing > synchronous replication, then that certainly makes sense. If you > aren't, then you're deciding

Re: checkpoints taking much longer than expected

2019-06-16 Thread Alvaro Herrera
On 2019-Jun-16, Stephen Frost wrote: > Not likely to help with what you're experiencing anyway though... My gut feeling is that you're wrong, since (as I understand) the symptoms are the same. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Re

unexpected behavior with pglogical -- bug?

2019-06-16 Thread Torsten Förtsch
Hi, out of curiosity I created the following setup, all with 9.6 and pglogical. D1 is configured as provider with a replication set that contains only 1 table. Only inserts are replicated. D2 is configured as subscriber for that replication set. Replication works, all inserts on D2 arrive also o

Re: checkpoints taking much longer than expected

2019-06-16 Thread Stephen Frost
Greetings, * Tiemen Ruiten (t.rui...@tech-lab.io) wrote: > On Sun, Jun 16, 2019 at 8:57 PM Alvaro Herrera > wrote: > > Note that Joyent ended up proposing patches to fix their performance > > problem (and got them committed). Maybe it would be useful for Tiemen > > to try that code? (That commi

Re: checkpoints taking much longer than expected

2019-06-16 Thread Tiemen Ruiten
On Sun, Jun 16, 2019 at 8:57 PM Alvaro Herrera wrote: > > Note that Joyent ended up proposing patches to fix their performance > problem (and got them committed). Maybe it would be useful for Tiemen > to try that code? (That commit cherry-picks cleanly on REL_11_STABLE.) > Interesting! The per

Re: checkpoints taking much longer than expected

2019-06-16 Thread Alvaro Herrera
On 2019-Jun-14, Peter J. Holzer wrote: > There was a discussion about ZFS' COW behaviour and PostgreSQL reusing > WAL files not being a good combination about a year ago: > https://www.postgresql.org/message-id/flat/CACukRjO7DJvub8e2AijOayj8BfKK3XXBTwu3KKARiTr67M3E3w%40mail.gmail.com > > Maybe yo

Re: bug regclass::oid

2019-06-16 Thread Tom Lane
I wrote: > ... I'd suggest recasting this as a join between > pg_catalog and pg_attribute, which would make the join condition > just "where c.oid = a.attrelid". Sigh, that should be "between pg_class and pg_attribute" of course. I'm really only firing on one cylinder today :-(

Re: bug regclass::oid

2019-06-16 Thread Tom Lane
"Peter J. Holzer" writes: > On 2019-06-16 18:03:02 +0200, John Mikel wrote: >> here is my query  >>  select   A.table_name  as "table_name",A.domain_name as "domain", >>  format_type(c.atttypid, c.atttypmod)  AS data_type ,A.column_name as >> "column_name", >>   A.is_nullable as "nullable",A.colum

Re: bug regclass::oid

2019-06-16 Thread Peter J. Holzer
On 2019-06-16 18:03:02 +0200, John Mikel wrote: > hi again  > here is my query  >  select   A.table_name  as "table_name",A.domain_name as "domain", >  format_type(c.atttypid, c.atttypmod)  AS data_type ,A.column_name as > "column_name", >   A.is_nullable as "nullable",A.column_default as "default"

Re: checkpoints taking much longer than expected

2019-06-16 Thread Stephen Frost
Greetings, * Jeff Janes (jeff.ja...@gmail.com) wrote: > On Sat, Jun 15, 2019 at 4:50 AM Tiemen Ruiten wrote: > > On Fri, Jun 14, 2019 at 5:43 PM Stephen Frost wrote: > >> The time information is all there and it tells you what it's doing and > >> how much had to be done... If you're unhappy with

Re: checkpoints taking much longer than expected

2019-06-16 Thread Stephen Frost
Greetings, * Tiemen Ruiten (t.rui...@tech-lab.io) wrote: > On Fri, Jun 14, 2019 at 5:43 PM Stephen Frost wrote: > > * Tiemen Ruiten (t.rui...@tech-lab.io) wrote: > > > checkpoint_timeout = 60min > > > > That seems like a pretty long timeout. > > My reasoning was that a longer recovery time to av

Re: bug regclass::oid

2019-06-16 Thread John Mikel
hi again here is my query *select A.table_name as "table_name",A.domain_name as "domain",* * format_type(c.atttypid, c.atttypmod) AS data_type ,A.column_name as "column_name",* * A.is_nullable as "nullable",A.column_default as "default"* * from information_schema.columns A inner join pg_att

Re: checkpoints taking much longer than expected

2019-06-16 Thread Jeff Janes
On Sat, Jun 15, 2019 at 4:50 AM Tiemen Ruiten wrote: > > On Fri, Jun 14, 2019 at 5:43 PM Stephen Frost wrote: > >> >> The time information is all there and it tells you what it's doing and >> how much had to be done... If you're unhappy with how long it takes to >> write out gigabytes of data an

Re: checkpoints taking much longer than expected

2019-06-16 Thread Michael Loftis
On Fri, Jun 14, 2019 at 08:02 Tiemen Ruiten wrote: > Hello, > > I setup a new 3-node cluster with the following specifications: > > 2x Intel(R) Xeon(R) CPU E5-2630 v4 @ 2.20GHz (2*20 cores) > 128 GB RAM > 8x Crucial MX500 1TB SSD's > > FS is ZFS, the dataset with the PGDATA directory on it has th