Fast seralizable transactions starving slower ones

2018-06-06 Thread Viktor Fougstedt
Hello! We have a system with a lot of integrity constraints that are not easily expressed as SQL constraints. We therefore run all writing transactions in serializable isolation, so that our code can make SELECT:s to check the constraints. We’ve run in to a, well, it’s not a “problem”, becaus

Re: Code of Conduct plan

2018-06-06 Thread Jan Claeys
On Wed, 2018-06-06 at 07:27 +0200, Chris Travers wrote: > The real fear here is the code of conduct being co-opted as a weapon > of world-wide culture war and that's what is driving a lot of the > resistance here. This is particularly an American problem here and > it causes a lot of resistance a

Re: Slow planning time for simple query

2018-06-06 Thread Tom Lane
Jerry Sievers writes: > Tom Lane writes: >> Oh, hmm, yeah it could be ye olde get_actual_variable_range() issue. >> When this happens, are there perhaps a lot of recently-dead rows at either >> extreme of the range of table1.source_id or table2.id? > We noticed the cluster of interest had a rogu

Re: Which backend using which pg_temp_N schema?

2018-06-06 Thread Adrian Klaver
On 06/06/2018 02:00 PM, Jerry Sievers wrote: Adrian Klaver writes: On 06/06/2018 08:54 AM, Jerry Sievers wrote: Adrian Klaver writes: Yep thanks... but IMO something that simply exposes whatever internal registry of temp schemas/PIDs (which I presume must exist) to DBA SQL avoids any perh

Re: Slow planning time for simple query

2018-06-06 Thread Jerry Sievers
Tom Lane writes: > Pavel Stehule writes: > >> 2018-06-06 18:59 GMT+02:00 Jeremy Finzel : >>> We have an odd scenario on one of our OLTP systems, which behaves the same >>> way on a streamer, of a 700-1000ms planning time for a query like this: >>> SELECT * >>> FROM table1 >>> WHERE source_id IN

Re: Which backend using which pg_temp_N schema?

2018-06-06 Thread Jerry Sievers
Adrian Klaver writes: > On 06/06/2018 08:54 AM, Jerry Sievers wrote: > >> Adrian Klaver writes: >> >> >> Yep thanks... but IMO something that simply exposes whatever internal >> registry of temp schemas/PIDs (which I presume must exist) to DBA SQL >> avoids any perhaps unreliable hackery such as

Re: Doing a \set through perl DBI ?

2018-06-06 Thread David Gauthier
I think I found my own answer. I wanted to use the current linux user's uid as part of a query (again, this is a perl/DBI script). I was thinking I might be able to set a variable into the DB session somehow using \set through DBI to accomplish this. The solution that finally occurred to me was

Re: Which backend using which pg_temp_N schema?

2018-06-06 Thread Adrian Klaver
On 06/06/2018 08:54 AM, Jerry Sievers wrote: Adrian Klaver writes: On 06/05/2018 04:49 PM, Jerry Sievers wrote: Adrian Klaver writes: On 06/05/2018 02:53 PM, Jerry Sievers wrote: Was just studying a legacy DB to learn about temp table activity. Felt like being able to tie temp schemas

Fw: Re: Out of memory error with PG10.3, 10.4 but not 9.3.19

2018-06-06 Thread ChatPristi
Forgotten to CC the list too, sorry. again.. >1073741818 is a bit less than 1GB and 1073741818+32 is a bit more. So >you are obviously hitting a 1GB limit here. >Given that 1GB is the maximum length of a character type value in >PostgreSQL and the error message mentions a "string buffer", I sus

Re: Slow planning time for simple query

2018-06-06 Thread Adrian Klaver
On 06/06/2018 09:59 AM, Jeremy Finzel wrote: Hello - We have an odd scenario on one of our OLTP systems, which behaves the same way on a streamer, of a 700-1000ms planning time for a query like this: SELECT * FROM table1 WHERE  source_id IN (SELECT id FROM table2 WHERE customer_id = $1); The

Fw: Re: Out of memory error with PG10.3, 10.4 but not 9.3.19

2018-06-06 Thread ChatPristi
Forgotten to CC the list, sorry... >Well, instead of an explain output which takes 2.4MB compressed and >9.6MB uncompressed (take it as unreadable), could you produce a >self-contained test case with a glimpse of the schema you are using? >Where does the OOM happen, and how did you change your pa

Re: Using distinct in an aggregate prevents parallel execution?

2018-06-06 Thread Thomas Kellerer
Tom Lane schrieb am 06.06.2018 um 16:32: Thomas Kellerer writes: Is this a known limitation? Yes, unless somebody has done radical restructuring of the aggregation code while I wasn't looking. agg(DISTINCT ...) is currently implemented inside the Agg plan node, so it's an indivisible black b

Re: Slow planning time for simple query

2018-06-06 Thread Tom Lane
Pavel Stehule writes: > 2018-06-06 18:59 GMT+02:00 Jeremy Finzel : >> We have an odd scenario on one of our OLTP systems, which behaves the same >> way on a streamer, of a 700-1000ms planning time for a query like this: >> SELECT * >> FROM table1 >> WHERE source_id IN (SELECT id FROM table2 WHERE

Re: Slow planning time for simple query

2018-06-06 Thread Pavel Stehule
2018-06-06 18:59 GMT+02:00 Jeremy Finzel : > Hello - > > We have an odd scenario on one of our OLTP systems, which behaves the same > way on a streamer, of a 700-1000ms planning time for a query like this: > > SELECT * > FROM table1 > WHERE source_id IN (SELECT id FROM table2 WHERE customer_id =

Re: Code of Conduct plan

2018-06-06 Thread Tom Lane
"David G. Johnston" writes: > On the topic of privacy - who exactly, from an administrative aspect, has > access to the systems that house these kinds of confidential > communications? Do these emails end up in PostgreSQL.org servers long-term > or is it mainly transient distribution and only ind

Re: Code of Conduct plan

2018-06-06 Thread Joshua D. Drake
On 06/06/2018 11:22 AM, Tom Lane wrote: I wrote: Yeah, somebody else made a similar point upthread. I guess we felt that the proper procedure was obvious given the structure, but maybe not. I could support adding text to clarify this, perhaps along the line of Hmm ... actually, there's anothe

Re: Code of Conduct plan

2018-06-06 Thread Alvaro Herrera
On 2018-Jun-06, David G. Johnston wrote: > On the topic of privacy - who exactly, from an administrative aspect, has > access to the systems that house these kinds of confidential > communications? Do these emails end up in PostgreSQL.org servers long-term > or is it mainly transient distribution

Re: Code of Conduct plan

2018-06-06 Thread David G. Johnston
On Wednesday, June 6, 2018, Tom Lane wrote: > Jeremy Schneider writes: > > My main feedback on the CoC is that it doesn't really say anything about > > what to do if the complaint is against a core team member. This was > > mentioned elsewhere in the email thread and I'm a bit surprised there's

Re: Code of Conduct plan

2018-06-06 Thread Tom Lane
I wrote: > Yeah, somebody else made a similar point upthread. I guess we felt that > the proper procedure was obvious given the structure, but maybe not. > I could support adding text to clarify this, perhaps along the line of Hmm ... actually, there's another special case that's not discussed, w

Re: Slow planning time for simple query

2018-06-06 Thread Jeremy Finzel
On Wed, Jun 6, 2018 at 1:13 PM, Jeremy Finzel wrote: > > > On Wed, Jun 6, 2018 at 12:12 PM, Tom Lane wrote: > >> Jeremy Finzel writes: >> > We have an odd scenario on one of our OLTP systems, which behaves the >> same >> > way on a streamer, of a 700-1000ms planning time for a query like this:

Re: Slow planning time for simple query

2018-06-06 Thread Jeremy Finzel
On Wed, Jun 6, 2018 at 12:12 PM, Tom Lane wrote: > Jeremy Finzel writes: > > We have an odd scenario on one of our OLTP systems, which behaves the > same > > way on a streamer, of a 700-1000ms planning time for a query like this: > > > SELECT * > > FROM table1 > > WHERE source_id IN (SELECT id

Re: Code of Conduct plan

2018-06-06 Thread Tom Lane
Jeremy Schneider writes: > My main feedback on the CoC is that it doesn't really say anything about > what to do if the complaint is against a core team member. This was > mentioned elsewhere in the email thread and I'm a bit surprised there's > nothing explicit in the CoC. If someone feels they h

Re: Which backend using which pg_temp_N schema?

2018-06-06 Thread Jerry Sievers
Tom Lane writes: > Jerry Sievers writes: > >> Yep thanks... but IMO something that simply exposes whatever internal >> registry of temp schemas/PIDs (which I presume must exist) > > Not really. There are a couple of ways that one could identify a > session's "BackendId", which is the "N" in the

Re: Slow planning time for simple query

2018-06-06 Thread Tom Lane
Jeremy Finzel writes: > We have an odd scenario on one of our OLTP systems, which behaves the same > way on a streamer, of a 700-1000ms planning time for a query like this: > SELECT * > FROM table1 > WHERE source_id IN (SELECT id FROM table2 WHERE customer_id = $1); Hm. Is this the first query

Slow planning time for simple query

2018-06-06 Thread Jeremy Finzel
Hello - We have an odd scenario on one of our OLTP systems, which behaves the same way on a streamer, of a 700-1000ms planning time for a query like this: SELECT * FROM table1 WHERE source_id IN (SELECT id FROM table2 WHERE customer_id = $1); The actual execution time is sub-ms. We initially t

Re: Which backend using which pg_temp_N schema?

2018-06-06 Thread Tom Lane
Jerry Sievers writes: > Yep thanks... but IMO something that simply exposes whatever internal > registry of temp schemas/PIDs (which I presume must exist) Not really. There are a couple of ways that one could identify a session's "BackendId", which is the "N" in the "pg_temp_N" name of the temp

Re: Which backend using which pg_temp_N schema?

2018-06-06 Thread Jerry Sievers
Adrian Klaver writes: > On 06/05/2018 04:49 PM, Jerry Sievers wrote: > >> Adrian Klaver writes: >> >>> On 06/05/2018 02:53 PM, Jerry Sievers wrote: >>> Was just studying a legacy DB to learn about temp table activity. Felt like being able to tie temp schemas to live backends s/b u

Re: Doing a \set through perl DBI ?

2018-06-06 Thread David G. Johnston
On Wednesday, June 6, 2018, David Gauthier wrote: > Hi: > > Is there a way to do the equivalent of a "\set foo 1" through perl dbi ? > I tried... > $dbh->do("\\set foo 1"); > and got a syntax error > > Of course, I'd also have to be able to access the value of foo once its > set. I'm guessi

Doing a \set through perl DBI ?

2018-06-06 Thread David Gauthier
Hi: Is there a way to do the equivalent of a "\set foo 1" through perl dbi ? I tried... $dbh->do("\\set foo 1"); and got a syntax error Of course, I'd also have to be able to access the value of foo once its set. I'm guessing the usual way ??? (select :foo) Thanks for any help !

Re: Code of Conduct plan

2018-06-06 Thread Joshua D. Drake
On 06/05/2018 08:55 PM, Tom Lane wrote: Adrian Klaver writes: On 06/05/2018 04:41 PM, Tom Lane wrote: I'm getting a little tired of people raising hypothetical harms and ignoring the real harms that we're hoping to fix. Yes, this is an experiment and it may not work, but we can't find out wit

Re: Using distinct in an aggregate prevents parallel execution?

2018-06-06 Thread Tom Lane
Thomas Kellerer writes: > Is this a known limitation? Yes, unless somebody has done radical restructuring of the aggregation code while I wasn't looking. agg(DISTINCT ...) is currently implemented inside the Agg plan node, so it's an indivisible black box to everything else. That was a simple,

Re: Failover replication building a new master

2018-06-06 Thread Adrian Klaver
On 06/05/2018 05:43 AM, Tom Loder wrote: I am using Postgres 10.4, and using replication, I have managed to set up four servers, with one running as a master and the other three running with streaming replication from the master. I have used the command: psql -c "ALTER SYSTEM SET synchronous_

Re: Which backend using which pg_temp_N schema?

2018-06-06 Thread Adrian Klaver
On 06/05/2018 04:49 PM, Jerry Sievers wrote: Adrian Klaver writes: On 06/05/2018 02:53 PM, Jerry Sievers wrote: Was just studying a legacy DB to learn about temp table activity. Felt like being able to tie temp schemas to live backends s/b useful but then didn't find a function/view for doi

Using distinct in an aggregate prevents parallel execution?

2018-06-06 Thread Thomas Kellerer
Consider this simplified example: select c.id, count(*) as total_orders, sum(p.price) as total_value from customer c join orders o ON c.id = o.customer_id join order_line ol ON o.id = ol.order_id join product p ON ol.product_id = p.id group by

Re: Setting up replication from 9.4 to 10.4

2018-06-06 Thread Lionel Tressens
Thanks Andreas, pglogical seems really great. My knowledge of replication was frozen at the time of Slony II and PG 8.4, I didn't have the chance to use replication since that time. I'll give a look at pglogical ! Best regards Lionel 2018-06-06 8:40 GMT+02:00 Andreas Kretschmer : > > > Am 06.0

Re: VBA to connect to postgresql from MS Access

2018-06-06 Thread Łukasz Jarych
Hi Adrian and Mike, All is working fine, thank you ! Problem was with connection string and 32 bit computer. My access is 32 bit so i should have odbc 32 bit driver for postgresql... Best, Jacek 2018-06-04 15:31 GMT+02:00 Adrian Klaver : > On 06/03/2018 09:30 PM, Łukasz Jarych wrote: > >> Th