PostgreSQL Rule does not work with deferred constraint.

2022-09-21 Thread Louis Tian
Hi all,

Encountered a possible bug today. In short, it appears that the rule system is 
not deferring the constraint checking as specified. Here is an example.
create table "parent" (
  "id" uuid primary key
);

create table "children" (
  "id" uuid primary key
  default gen_random_uuid()
   references parent("id")
deferrable
initially deferred
);

-- this works as expected.
begin;
  insert into "children" values ('74c247be-2fc3-43e5-94f8-d5e1af147a9b');
  insert into "parent" values ('74c247be-2fc3-43e5-94f8-d5e1af147a9b');
commit;

-- doing the same with a rule
create rule on_insert as on insert to "children" do also (
  insert into "parent" (id) values (new."id");
);

-- this fails with:
--    ERROR:  insert or update on table "children" violates foreign key 
constraint "children_id_fkey"
--    DETAIL:  Key (id)=(40513132-f9fa-4e5a-aa46-20c8185b3e58) is not present 
in table "parent".
--    SQL state: 23503
insert into "children" values (default);
The rules system supposedly transforms the insert statement and executed the 
rule action in the same transaction. So I think it should work.
But I got the same error on both pg13 and pg14.

Is there something I missed here? or is my understanding of the rule system 
just simply wrong?

Regards,

Louis Tian
louis.t...@aquamonix.com.au



Re: PostgreSQL Rule does not work with deferred constraint.

2022-09-21 Thread Laurenz Albe
On Wed, 2022-09-21 at 01:19 +, Louis Tian wrote:
> Encountered a possible bug today. In short, it appears that the rule system 
> is not deferring the constraint checking as specified. Here is an example.
> > create table "parent" (
> >   "id" uuid primary key
> > );
> > 
> > create table "children" (
> >   "id" uuid primary key
> >   default gen_random_uuid()
> >    references parent("id")
> > deferrable
> > initially deferred
> > );
> > 
> > -- this works as expected.
> > begin;
> >   insert into "children" values 
> > ('74c247be-2fc3-43e5-94f8-d5e1af147a9b');
> >   insert into "parent" values ('74c247be-2fc3-43e5-94f8-d5e1af147a9b');
> > commit;
> > 
> > -- doing the same with a rule
> > create rule on_insert as on insert to "children" do also (
> >   insert into "parent" (id) values (new."id");
> > );
> > 
> > -- this fails with:
> > --    ERROR:  insert or update on table "children" violates foreign key 
> > constraint "children_id_fkey"
> > --    DETAIL:  Key (id)=(40513132-f9fa-4e5a-aa46-20c8185b3e58) is not 
> > present in table "parent".
> > --    SQL state: 23503
> > insert into "children" values (default);
> The rules system supposedly transforms the insert statement and executed the 
> rule action in the same transaction. So I think it should work. 
> But I got the same error on both pg13 and pg14.
> 
> Is there something I missed here? or is my understanding of the rule system 
> just simply wrong? 

I just tried your commands, and it works as you expect on my PostgreSQL v15 
database.

Yours,
Laurenz Albe




Re: I slipped up so that no existing role allows connection. Is rescue possible?

2022-09-21 Thread Mladen Gogala

On 9/20/22 18:54, Theodore M Rolle, Jr. wrote:

And Tom’s English is excellent!


As opposed to mine?

--
Mladen Gogala
Database Consultant
Tel: (347) 321-1217
https://dbwhisperer.wordpress.com


10.22 Windows binaries download? (zip "invalid" on Enterprisedb)

2022-09-21 Thread Thomas, Richard
I'm trying to download the Windows x64 binaries only for PostgreSQL 10.22 in 
order to perform a minor upgrade. However, the download 
(postgresql-10.22-1-windows-x64-binaries.zip) from my usual go-to place: 
(https://www.enterprisedb.com/download-postgresql-binaries) gives me a file 
that Windows zip declares invalid. Other zip files (e.g. the 32-bit version) 
download and unzip quite happily. I've dropped Enterprise an email, but in the 
meantime wonder if there are any other (reputable!) binaries download sites I 
could use?

Richard


At Atkins - member of the SNC-Lavalin Group, we work flexible hours around the 
world. Although I have sent this email at a time convenient for me, I don't 
expect you to respond until it works for you.

NOTICE - This email message and any attachments may contain information or 
material that is confidential, privileged, and/or subject to copyright or other 
rights. Any unauthorized viewing, disclosure, retransmission, dissemination, or 
other use of or reliance on this message or anything contained therein is 
strictly prohibited and may be unlawful. If you believe you may have received 
this message in error, kindly inform the sender by return email and delete this 
message from your system. Thank you.


Re: 10.22 Windows binaries download? (zip "invalid" on Enterprisedb)

2022-09-21 Thread Hillary Masha
Hi Richard,

I ran into this issue as well, the issue is with the length of the paths
for the subdirectories,windows is unable to open them, they can be unzipped
in powershell and 7-zip  see
https://www.postgresql.org/message-id/CA+RCjaw=GH-Lyxe5=5-un6sqjo-cqjhtebdomixowqdrvvy...@mail.gmail.com

On Wed, Sep 21, 2022 at 9:34 AM Thomas, Richard <
richard.tho...@atkinsglobal.com> wrote:

> I’m trying to download the Windows x64 binaries only for PostgreSQL 10.22
> in order to perform a minor upgrade. However, the download
> (postgresql-10.22-1-windows-x64-binaries.zip) from my usual go-to place: (
> https://www.enterprisedb.com/download-postgresql-binaries) gives me a
> file that Windows zip declares invalid. Other zip files (e.g. the 32-bit
> version) download and unzip quite happily. I’ve dropped Enterprise an
> email, but in the meantime wonder if there are any other (reputable!)
> binaries download sites I could use?
>
>
>
> Richard
>
>
>
> *At Atkins - member of the SNC-Lavalin Group, we work flexible hours
> around the world. Although I have sent this email at a time convenient for
> me, I don't expect you to respond until it works for you. *
>
> NOTICE – This email message and any attachments may contain information or
> material that is confidential, privileged, and/or subject to copyright or
> other rights. Any unauthorized viewing, disclosure, retransmission,
> dissemination, or other use of or reliance on this message or anything
> contained therein is strictly prohibited and may be unlawful. If you
> believe you may have received this message in error, kindly inform the
> sender by return email and delete this message from your system. Thank you.
>


Re: 10.22 Windows binaries download? (zip "invalid" on Enterprisedb)

2022-09-21 Thread Erik Wienhold
> On 21/09/2022 15:34 CEST Thomas, Richard  
> wrote:
> 
> I’m trying to download the Windows x64 binaries only for PostgreSQL 10.22 in 
> order to perform a minor upgrade. However, the download 
> (postgresql-10.22-1-windows-x64-binaries.zip) from my usual go-to place: 
> (https://www.enterprisedb.com/download-postgresql-binaries) gives me a file 
> that Windows zip declares invalid.

There's already a thread on that issue:

https://www.postgresql.org/message-id/flat/CA%2BRCjaw%3DGH-Lyxe5%3D5-un6sQjO-cQjhTeBDomiXoWQDrvVYwxA%40mail.gmail.com

--
Erik




pg_stat_activity.backend_xmin

2022-09-21 Thread Dirschel, Steve
We are troubleshooting an issue where autovacuum is not cleaning up a table.  
The application using this database runs with autocommit turned off.  We can 
see in pg_stat_activity lots of sessions "idle in transaction" even though 
those sessions have not executed any DML-  they have executed selects but no 
DML.  The database's isolation level is set to read committed.

In a test database if I login through psql and set autocommit off and issue a 
select I can see my session in pg_stat_activity has xact_start populated but 
backend_xmin is null.  If I run vacuum against the table I selected from (that 
has garbage rows that need to be cleaned up) it will clean them up.  But if I 
do a "set transaction isolation level repeatable read" and then do the select 
pg_stat_activity xact_start is populated and backend_xmin is also populated.  
In a different session if I delete/insert into the table I selected from and 
then run vacuum against the table those rows will not get cleaned up because 
the xmin is of the rows that need to get cleaned up are higher (or is it lower) 
than the backend_xmin of my select session.

That scenario is the scenario we are seeing through the application.

According to the app team they are not aware of their code changing the 
isolation level to repeatable read.  Are there other scenarios where the 
transaction isolation is set to read committed, the session has autocommit off, 
and a select will populate backend_xmin in pg_stat_activity for the session 
that issued the select?

In session #1:

\set AUTOCOMMIT off
Set transaction isolation level repeatable read;
Select * from test1;


In session #2:
Delete from test1;
Insert into test1 values (1);
... do 10 more inserts, delete the rows, do 10 more inserts, delete the rows.
...

vacuum(verbose) test1;
INFO:  vacuuming "public.test1"
INFO:  "test1": found 0 removable, 55 nonremovable row versions in 1 out of 1 
pages
DETAIL:  44 dead row versions cannot be removed yet, oldest xmin: 32708199
There were 172 unused item identifiers.
Skipped 0 pages due to buffer pins, 0 frozen pages.
0 pages are entirely empty.
CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s.
INFO:  vacuuming "pg_toast.pg_toast_329295"
INFO:  "pg_toast_329295": found 0 removable, 0 nonremovable row versions in 0 
out of 0 pages
DETAIL:  0 dead row versions cannot be removed yet, oldest xmin: 32708199
There were 0 unused item identifiers.
Skipped 0 pages due to buffer pins, 0 frozen pages.
0 pages are entirely empty.
CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s.
VACUUM

The xmin value of 32708199 is the value in pg_stat_activity.backend_xmin for 
the session who issued the select.  Once I issue a commit in that session and 
then try autovacuum again:

vacuum(verbose) test1;
INFO:  vacuuming "public.test1"
INFO:  "test1": removed 44 row versions in 1 pages
INFO:  "test1": found 44 removable, 11 nonremovable row versions in 1 out of 1 
pages
DETAIL:  0 dead row versions cannot be removed yet, oldest xmin: 32708957
There were 172 unused item identifiers.
Skipped 0 pages due to buffer pins, 0 frozen pages.
0 pages are entirely empty.
CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s.
INFO:  vacuuming "pg_toast.pg_toast_329295"
INFO:  "pg_toast_329295": found 0 removable, 0 nonremovable row versions in 0 
out of 0 pages
DETAIL:  0 dead row versions cannot be removed yet, oldest xmin: 32708957
There were 0 unused item identifiers.
Skipped 0 pages due to buffer pins, 0 frozen pages.
0 pages are entirely empty.
CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s.
VACUUM


Thanks



This e-mail is for the sole use of the intended recipient and contains 
information that may be privileged and/or confidential. If you are not an 
intended recipient, please notify the sender by return e-mail and delete this 
e-mail and any attachments. Certain required legal entity disclosures can be 
accessed on our website: 
https://www.thomsonreuters.com/en/resources/disclosures.html


Re: pg_stat_activity.backend_xmin

2022-09-21 Thread Laurenz Albe
On Wed, 2022-09-21 at 14:11 +, Dirschel, Steve wrote:
> We are troubleshooting an issue where autovacuum is not cleaning up a table.
> The application using this database runs with autocommit turned off.
> We can see in pg_stat_activity lots of sessions “idle in transaction” even
> though those sessions have not executed any DML-  they have executed selects
> but no DML.  The database’s isolation level is set to read committed.
>  
> In a test database if I login through psql and set autocommit off and issue a
> select I can see my session in pg_stat_activity has xact_start populated but
> backend_xmin is null.  If I run vacuum against the table I selected from
> (that has garbage rows that need to be cleaned up) it will clean them up.
> But if I do a “set transaction isolation level repeatable read” and then do
> the select pg_stat_activity xact_start is populated and backend_xmin is also
> populated.  In a different session if I delete/insert into the table I
> selected from and then run vacuum against the table those rows will not get
> cleaned up because the xmin is of the rows that need to get cleaned up are
> higher (or is it lower) than the backend_xmin of my select session.
>  
> That scenario is the scenario we are seeing through the application.
>  
> According to the app team they are not aware of their code changing the
> isolation level to repeatable read.  Are there other scenarios where the
> transaction isolation is set to read committed, the session has autocommit
> off, and a select will populate backend_xmin in pg_stat_activity for the
> session that issued the select?

"backend_xmin" is set when the session has an active snapshot.  Such a
snapshot is held for the whole duration of a transaction in the REPEATABLE
READ isolation level, but there are cases where you can see that in READ
COMMITTED isolation level as well:

- if there is a long running query

- if there is a cursor open

Perhaps you could ask your developers if they have long running read-only
transactions with cursors.

Yours,
Laurenz Albe
-- 
Cybertec | https://www.cybertec-postgresql.com




Re: PostgreSQL Rule does not work with deferred constraint.

2022-09-21 Thread Tom Lane
Laurenz Albe  writes:
> I just tried your commands, and it works as you expect on my PostgreSQL v15 
> database.

It does fail for me, but I think it's a well-known trap rather than
a bug (or at least, it's not something that anyone wishes to redesign
the rule system to change).  The problem is that *a rule is a macro*
and therefore it's subject to multiple-evaluation hazards.  Your
volatile default expression does not play nice with that.

Initially you have:

insert into "children" values (default);

Replacement of the "default" produces:

insert into "children" values (gen_random_uuid());

Now the DO ALSO rule produces:

insert into "parent" (id) values (gen_random_uuid());

The two insertions will compute different random UUIDs,
and kaboom.

We tend to recommend using triggers not rules to implement this
sort of behavior; they are less prone to surprises.

regards, tom lane




RE: 10.22 Windows binaries download? (zip "invalid" on Enterprisedb)

2022-09-21 Thread Thomas, Richard
Thanks very much - 7-zip (from https://www.7-zip.org/) was just the job to 
unpack the archive where Windows "Extract all" failed.

Richard

> -Original Message-
> From: Erik Wienhold 
> Sent: 21 September 2022 14:44
> To: Thomas, Richard ; pgsql-
> gene...@lists.postgresql.org
> Subject: Re: 10.22 Windows binaries download? (zip "invalid" on Enterprisedb)
>
> > On 21/09/2022 15:34 CEST Thomas, Richard
>  wrote:
> >
> > I’m trying to download the Windows x64 binaries only for PostgreSQL 10.22 in
> order to perform a minor upgrade. However, the download
> (https://urldefense.com/v3/__http://postgresql-10.22-1-windows-x64-
> binaries.zip__;!!OepYZ6Q!8OcLEO8KpfZpItfLWpOliejGRWOyHm9zuZBvTxc6ifxC
> c7xA5i3TOGADp-wurt7EIz-8Z3QQj5_0fmh2l21mMw$  ) from my usual go-to
> place: (https://urldefense.com/v3/__https://www.enterprisedb.com/download-
> postgresql-
> binaries__;!!OepYZ6Q!8OcLEO8KpfZpItfLWpOliejGRWOyHm9zuZBvTxc6ifxCc7x
> A5i3TOGADp-wurt7EIz-8Z3QQj5_0fmjyao_6kg$  ) gives me a file that Windows
> zip declares invalid.
>
> There's already a thread on that issue:
>
> https://urldefense.com/v3/__https://www.postgresql.org/message-
> id/flat/CA*2BRCjaw*3DGH-Lyxe5*3D5-un6sQjO-
> cQjhTeBDomiXoWQDrvVYwxA*40mail.gmail.com__;JSUlJQ!!OepYZ6Q!8OcLEO
> 8KpfZpItfLWpOliejGRWOyHm9zuZBvTxc6ifxCc7xA5i3TOGADp-wurt7EIz-
> 8Z3QQj5_0fmhPwsTSbA$
>
> --
> Erik
At Atkins - member of the SNC-Lavalin Group, we work flexible hours around the 
world. Although I have sent this email at a time convenient for me, I don't 
expect you to respond until it works for you.
NOTICE – This email message and any attachments may contain information or 
material that is confidential, privileged, and/or subject to copyright or other 
rights. Any unauthorized viewing, disclosure, retransmission, dissemination, or 
other use of or reliance on this message or anything contained therein is 
strictly prohibited and may be unlawful. If you believe you may have received 
this message in error, kindly inform the sender by return email and delete this 
message from your system. Thank you.


RE: [EXT] Re: pg_stat_activity.backend_xmin

2022-09-21 Thread Dirschel, Steve
On Wed, 2022-09-21 at 14:11 +, Dirschel, Steve wrote:
>> We are troubleshooting an issue where autovacuum is not cleaning up a table.
>> The application using this database runs with autocommit turned off.
>> We can see in pg_stat_activity lots of sessions “idle in transaction” 
>> even though those sessions have not executed any DML-  they have 
>> executed selects but no DML.  The database’s isolation level is set to read 
>> committed.

> "backend_xmin" is set when the session has an active snapshot.  Such a 
> snapshot is held for the whole duration of a transaction in the REPEATABLE 
> READ isolation level, but there are cases where you can see that in READ 
> COMMITTED isolation level > as well:
>
> - if there is a long running query
>
> - if there is a cursor open
>
> Perhaps you could ask your developers if they have long running read-only 
> transactions with cursors.
>
> Yours,
> Laurenz Albe

Thanks for the reply Laurenz.  For an application session in this "state" 
pg_stat_activity shows the state of "idle in transaction" and backend_xmin is 
populated.  The query shows the last select query it ran.  It is not currently 
executing a query.  And dev has said they are not using a cursor for the query. 
 So it does not appear they have long running read-only transactions with 
cursors.

Outside that scenario can you think of any others where a session:
1. Login to the database
2. Set autocommit off
3. Run select query, query completes, session does nothing after that query 
completes.
4.  transaction isolation level is read committed

That session sitting there idle in transaction has backend_xmin populated.  
When I run that test backend_xmin does not get populated unless I set my 
transaction isolation level to repeatable read.  We have enabled statement 
logging so we can see if their sessions are changing that transaction isolation 
level behind the scenes that they are not aware of but so far we have not seen 
that type of command logged.

Regards
Steve


Re: [EXT] pg_stat_activity.backend_xmin

2022-09-21 Thread Rob Sargent



> On Sep 21, 2022, at 9:32 AM, Dirschel, Steve 
>  wrote:
> 
> On Wed, 2022-09-21 at 14:11 +, Dirschel, Steve wrote:
>>> We are troubleshooting an issue where autovacuum is not cleaning up a table.
>>> The application using this database runs with autocommit turned off.
>>> We can see in pg_stat_activity lots of sessions “idle in transaction” 
>>> even though those sessions have not executed any DML-  they have 
>>> executed selects but no DML.  The database’s isolation level is set to read 
>>> committed.
> 
>> "backend_xmin" is set when the session has an active snapshot.  Such a 
>> snapshot is held for the whole duration of a transaction in the REPEATABLE 
>> READ isolation level, but there are cases where you can see that in READ 
>> COMMITTED isolation level > as well:
>> 
>> - if there is a long running query
>> 
>> - if there is a cursor open
>> 
>> Perhaps you could ask your developers if they have long running read-only 
>> transactions with cursors.
>> 
>> Yours,
>> Laurenz Albe
> 
> Thanks for the reply Laurenz.  For an application session in this "state" 
> pg_stat_activity shows the state of "idle in transaction" and backend_xmin is 
> populated.  The query shows the last select query it ran.  It is not 
> currently executing a query.  And dev has said they are not using a cursor 
> for the query.  So it does not appear they have long running read-only 
> transactions with cursors.
> 
> Outside that scenario can you think of any others where a session:
> 1. Login to the database
> 2. Set autocommit off
> 3. Run select query, query completes, session does nothing after that query 
> completes.
> 4.  transaction isolation level is read committed
> 
> That session sitting there idle in transaction has backend_xmin populated.  
> When I run that test backend_xmin does not get populated unless I set my 
> transaction isolation level to repeatable read.  We have enabled statement 
> logging so we can see if their sessions are changing that transaction 
> isolation level behind the scenes that they are not aware of but so far we 
> have not seen that type of command logged.
> 
> Regards
> Steve

What stack is the application using?  Anything like Spring or Hibernate 
involved?





Re: tcp settings

2022-09-21 Thread Rob Sargent



> On Sep 20, 2022, at 10:03 PM, Tom Lane  wrote:
> 
> Rob Sargent  writes:
>>> then keepalives aren't necessarily the solution anyway.  When is
>>> this failure occurring ... is it while trying to establish the
>>> database connection in the first place?  Or does it only happen
>>> if you've left the psql session sit idle for a long while?
> 
>> Sitting idle for not even a long while. Under half hour I feel would do it.  
>> But I don’t think it times out while I’m “SQLing”
> 
> OK, that does sound like something that reducing the keepalive
> interval could help with.  The traditional keepalive timeout
> is a couple of hours, at least on my Linux box:
> 
> $ cat /proc/sys/net/ipv4/tcp_keepalive_time
> 7200
> 
> but it sounds like there's something between you and the database
> server that will forget connections a lot quicker than that.
> 
>   regards, tom lane

I was afraid you would conclude that.  But its just plain ol’ psql (inside 
emacs) on interactive node directly to db server node, all in a “protected 
environment”.  There could be network pieces in the middle that are getting 
uppity.  I’ll ask /them/.

Thanks  



Re: 10.22 Windows binaries download? (zip "invalid" on Enterprisedb)

2022-09-21 Thread JITEN KUMAR SHAH
Few days before one of the user having same issue and they find out it 
was some file name issue


Link of email:
https://www.postgresql.org/message-id/CA%2BRCjax7d3djH6YMRb9bnHzmtz6hFhexq9YsYbtYdeOVUwh5%2BQ%40mail.gmail.com


On 9/21/22 20:57, Thomas, Richard wrote:

Thanks very much - 7-zip (from https://www.7-zip.org/) was just the job to unpack the 
archive where Windows "Extract all" failed.

Richard


-Original Message-
From: Erik Wienhold 
Sent: 21 September 2022 14:44
To: Thomas, Richard ; pgsql-
gene...@lists.postgresql.org
Subject: Re: 10.22 Windows binaries download? (zip "invalid" on Enterprisedb)


On 21/09/2022 15:34 CEST Thomas, Richard

 wrote:

I’m trying to download the Windows x64 binaries only for PostgreSQL 10.22 in

order to perform a minor upgrade. However, the download
(https://urldefense.com/v3/__http://postgresql-10.22-1-windows-x64-
binaries.zip__;!!OepYZ6Q!8OcLEO8KpfZpItfLWpOliejGRWOyHm9zuZBvTxc6ifxC
c7xA5i3TOGADp-wurt7EIz-8Z3QQj5_0fmh2l21mMw$  ) from my usual go-to
place: (https://urldefense.com/v3/__https://www.enterprisedb.com/download-
postgresql-
binaries__;!!OepYZ6Q!8OcLEO8KpfZpItfLWpOliejGRWOyHm9zuZBvTxc6ifxCc7x
A5i3TOGADp-wurt7EIz-8Z3QQj5_0fmjyao_6kg$  ) gives me a file that Windows
zip declares invalid.

There's already a thread on that issue:

https://urldefense.com/v3/__https://www.postgresql.org/message-
id/flat/CA*2BRCjaw*3DGH-Lyxe5*3D5-un6sQjO-
cQjhTeBDomiXoWQDrvVYwxA*40mail.gmail.com__;JSUlJQ!!OepYZ6Q!8OcLEO
8KpfZpItfLWpOliejGRWOyHm9zuZBvTxc6ifxCc7xA5i3TOGADp-wurt7EIz-
8Z3QQj5_0fmhPwsTSbA$

--
Erik

At Atkins - member of the SNC-Lavalin Group, we work flexible hours around the 
world. Although I have sent this email at a time convenient for me, I don't 
expect you to respond until it works for you.
NOTICE – This email message and any attachments may contain information or 
material that is confidential, privileged, and/or subject to copyright or other 
rights. Any unauthorized viewing, disclosure, retransmission, dissemination, or 
other use of or reliance on this message or anything contained therein is 
strictly prohibited and may be unlawful. If you believe you may have received 
this message in error, kindly inform the sender by return email and delete this 
message from your system. Thank you.






RE: [EXT] pg_stat_activity.backend_xmin

2022-09-21 Thread Dirschel, Steve
>> On Sep 21, 2022, at 9:32 AM, Dirschel, Steve 
>>  wrote:
>> 
>> On Wed, 2022-09-21 at 14:11 +, Dirschel, Steve wrote:
 We are troubleshooting an issue where autovacuum is not cleaning up a 
 table.
 The application using this database runs with autocommit turned off.
 We can see in pg_stat_activity lots of sessions “idle in transaction” 
 even though those sessions have not executed any DML-  they have 
 executed selects but no DML.  The database’s isolation level is set to 
 read committed.
>> 
>>> "backend_xmin" is set when the session has an active snapshot.  Such a 
>>> snapshot is held for the whole duration of a transaction in the REPEATABLE 
>>> READ isolation level, but there are cases where you can see that in READ 
>>> COMMITTED isolation level > as well:
>>> 
>>> - if there is a long running query
>>> 
>>> - if there is a cursor open
>>> 
>>> Perhaps you could ask your developers if they have long running read-only 
>>> transactions with cursors.
>>> 
>>> Yours,
>>> Laurenz Albe
>> 
>> Thanks for the reply Laurenz.  For an application session in this "state" 
>> pg_stat_activity shows the state of "idle in transaction" and backend_xmin 
>> is populated.  The query shows the last select query it ran.  It is not 
>> currently executing a query.  And dev has said they are not using a cursor 
>> for the query.  So it does not appear they have long running read-only 
>> transactions with cursors.
>> 
>> Outside that scenario can you think of any others where a session:
>> 1. Login to the database
>> 2. Set autocommit off
>> 3. Run select query, query completes, session does nothing after that query 
>> completes.
>> 4.  transaction isolation level is read committed
>> 
>> That session sitting there idle in transaction has backend_xmin populated.  
>> When I run that test backend_xmin does not get populated unless I set my 
>> transaction isolation level to repeatable read.  We have enabled statement 
>> logging so we can see if their sessions are changing that transaction 
>> isolation level behind the scenes that they are not aware of but so far we 
>> have not seen that type of command logged.
>> 
>> Regards
>> Steve
>
>What stack is the application using?  Anything like Spring or Hibernate 
>involved?

Java is the stack.  Thanks


pgbackrest Help Required

2022-09-21 Thread Inzamam Shafiq
Hi Folks,

Hope you are doing well.

I am in process of configuring pgbackrest, I have followed documents and 
configured backup from backup server. I have a setup of Primary and Hot 
Standby, when I configured pgbackrest the standby DB got un-synced and now I am 
unable to recover it, can anyone help or share a good document where we can 
implement pgbackrest where streaming replication is already in place using 
traditional pg_basebackup method?

Note: I want to configure pgbackrest and to take backup from standby DB, 
however streaming replication is already there, where the archive and restore 
commands are configured using traditional zip method.

Thanks.

Regards,

Inzamam Shafiq
Sr. DBA


Re: PCI-DSS Requirements

2022-09-21 Thread Inzamam Shafiq
Hi Ron,

Thank you for the response.

Actually we are in a starting phase and I have done instance level encryption 
(CYBERTECH TDE Patch) but if someone take dump and restore it on another server 
the data get restored successfully. Also the problem is that the data is in 
plain text.

So I want to ask if disk or instance level encryption useful or we should focus 
on column level encryption?

Also if any error occurred during DML and a plain query will be written into 
the logs which may not be compliant with PCI. How to overcome that?

Thanks.

Regards,

Inzamam Shafiq
Sr. DBA

From: Ron 
Sent: Tuesday, September 20, 2022 10:44 PM
To: pgsql-general@lists.postgresql.org 
Subject: Re: PCI-DSS Requirements

On 9/20/22 04:27, Inzamam Shafiq wrote:

Hi Team,


Anyone on PCI-DSS requirements for PostgreSQL DB, need help for some of the 
points.

Can you be more specific?  (Typically. the auditors or the "audit pre-check" 
team will ask for a bunch of details on how your instance is configured.)

The usual questions I get are:
- What password hash algorithm is used?
- How frequently to passwords expire?
- Is SSL used when communicating with applications?

--
Angular momentum makes the world go 'round.