Re: Logical Replication 08P01 invalid memory alloc request size 1095736448

2025-07-06 Thread Soni M
Thanks Tom, really appreciate it. On Sun, Jul 6, 2025 at 10:12 PM Tom Lane wrote: > Soni M writes: > > We just do patch updates from 15.12 to 15.13, restart Logical Master and > > then Logical Replica, and suddenly Logical replication stops working. > > Streaming replica

Re: Logical Replication 08P01 invalid memory alloc request size 1095736448

2025-07-06 Thread Tom Lane
Soni M writes: > We just do patch updates from 15.12 to 15.13, restart Logical Master and > then Logical Replica, and suddenly Logical replication stops working. > Streaming replication of both are fine. This sounds like the same bug previously discussed in https://www.postgresql.org/m

Re: Logical Replication 08P01 invalid memory alloc request size 1095736448

2025-07-06 Thread Soni M
sr/pgsql-15/lib/libpqwalreceiver""",,,,,"","logical replication worker",,0 DEBUG,0,"find_in_dynamic_libpath: trying ""/usr/pgsql-15/lib/libpqwalreceiver.so""","","logical replication worker",,0 D

Logical Replication 08P01 invalid memory alloc request size 1095736448

2025-07-06 Thread Soni M
Hello All, we have this set up : Postgres 15.13 on RHEL 9.6 Logical Master also have 2 streaming replica Logical Replica also have 1 streaming replica We just do patch updates from 15.12 to 15.13, restart Logical Master and then Logical Replica, and suddenly Logical replication stops working

Re: Logical Replication Memory Allocation Error - "invalid memory alloc request size"

2025-06-17 Thread Masahiko Sawada
On Wed, Jun 11, 2025 at 7:36 PM Hayato Kuroda (Fujitsu) wrote: > > Dear Max, > > > We have rewritten as many of our transactions as possible to avoid using > > temporary tables, and so far, that seems to have resolved the problem. > > Good to know. We try to fix as soon as possible. > I pushed th

RE: Logical Replication Memory Allocation Error - "invalid memory alloc request size"

2025-06-11 Thread Hayato Kuroda (Fujitsu)
Dear Max, > We have rewritten as many of our transactions as possible to avoid using > temporary tables, and so far, that seems to have resolved the problem. Good to know. We try to fix as soon as possible. Sorry for inconvenience. Best regards, Hayato Kuroda FUJITSU LIMITED

Re: Logical Replication Memory Allocation Error - "invalid memory alloc request size"

2025-06-11 Thread Max Madden
s with memory allocation errors like: > > > > ``` > > 2025-06-10 14:14:56.800 UTC [299] ERROR: could not receive data from WAL > stream: ERROR: invalid memory alloc request size 1238451248 > > 2025-06-10 14:14:56.805 UTC [1] LOG: background worker "logical > replication worker

RE: Logical Replication Memory Allocation Error - "invalid memory alloc request size"

2025-06-10 Thread Hayato Kuroda (Fujitsu)
6-10 14:14:56.800 UTC [299] ERROR: could not receive data from WAL > stream: ERROR: invalid memory alloc request size 1238451248 > 2025-06-10 14:14:56.805 UTC [1] LOG: background worker "logical replication > worker" (PID 299) exited with exit code 1 > ``` I think this

Logical Replication Memory Allocation Error - "invalid memory alloc request size"

2025-06-10 Thread Max Madden
Hello, I'm encountering a consistent issue with PostgreSQL 15 logical replication and would appreciate any guidance on debugging or resolving this problem. *Setup:* - Source: PostgreSQL 15.x - Target: PostgreSQL 15.x - Replication: Logical replication using publication/subscription (pgo

Streaming Replication Disconnection Behavior under recovery_min_apply_delay Configuration

2025-05-27 Thread Riku Kashiwaki (Fujitsu)
I would like to know the behavior when disconnecting streaming replication. An attempt was made to disconnect streaming replication from a standby server during streaming replication by using the ALTER SET command and setting primary _conninfo to an empty string. However, we have observed that

Re: Custom logical replication client

2025-05-23 Thread Edson Carlos Ericksson Richter
On May 23, 2025, at 11:36, Edson Carlos Ericksson Richter wrote: Dear Community, I'm starting a project where I would like to write an application to decode the logical replication protocol to save data into an analytics streaming system. Is there a starting guide or similar project tha

Re: Custom logical replication client

2025-05-23 Thread Christophe Pettus
> On May 23, 2025, at 11:36, Edson Carlos Ericksson Richter > wrote: > > Dear Community, > I'm starting a project where I would like to write an application to decode > the logical replication protocol to save data into an analytics streaming > system. > Is ther

Custom logical replication client

2025-05-23 Thread Edson Carlos Ericksson Richter
Dear Community, I'm starting a project where I would like to write an application to decode the logical replication protocol to save data into an analytics streaming system. Is there a starting guide or similar project that can help me to educate myself on how to accomplish this

Re: Do stuck replication slots prevent autovacuum of running entirely?

2025-05-20 Thread Laurenz Albe
On Wed, 2025-05-21 at 16:34 +1200, Marcelo Fernandes wrote: > I am trying to understand if a stuck replication slot would be sufficient to > stop an autovacuum of even starting. > > Couldn't the autovacuum process start, but fail to remove dead tuples that > are still necessar

Do stuck replication slots prevent autovacuum of running entirely?

2025-05-20 Thread Marcelo Fernandes
Hi there, I am trying to understand if a stuck replication slot would be sufficient to stop an autovacuum of even starting. Couldn't the autovacuum process start, but fail to remove dead tuples that are still necessary by the replication slot? Why would it prevent autovacuum of even sta

Re: Logical replication, need to reclaim big disk space

2025-05-20 Thread Achilleas Mantzios
one server, I'll process a table a time, with a nightly script, and issue a VACUUM FULL to tables that have already been processed. But I'm in a logical replication architecture (we are using a multimaster system called pgEdge, but I don't think it will make big difference, si

Re: Logical replication, need to reclaim big disk space

2025-05-20 Thread Moreno Andreo
t server is almost immediate. If I had only one server, I'll process a table a time, with a nightly script, and issue a VACUUM FULL to tables that have already been processed. But I'm in a logical replication architecture (we are using a multimaster system called pgEdge, but I don&

Re: Logical replication, need to reclaim big disk space

2025-05-20 Thread Moreno Andreo
ou want to run on cloud. Even the VMs where PostgreSQL is running. The problem is: this is generating BIG table bloat, as you may imagine. Running a VACUUM FULL on an ex-22GB table on a standalone test server is almost immediate. If I had only one server, I'll process a table a time, with a ni

Re: Logical replication, need to reclaim big disk space

2025-05-19 Thread Achilleas Mantzios
with a nightly script, and issue a VACUUM FULL to tables that have already been processed. But I'm in a logical replication architecture (we are using a multimaster system called pgEdge, but I don't think it will make big difference, since it's based on logical replication), and

Re: Logical replication, need to reclaim big disk space

2025-05-19 Thread Moreno Andreo
ime, with a nightly script, and issue a VACUUM FULL to tables that have already been processed. But I'm in a logical replication architecture (we are using a multimaster system called pgEdge, but I don't think it will make big difference, since it's based on logical replication)

Re: Logical replication, need to reclaim big disk space

2025-05-19 Thread Achilleas Mantzios
ting BIG table bloat, as you may imagine. Running a VACUUM FULL on an ex-22GB table on a standalone test server is almost immediate. If I had only one server, I'll process a table a time, with a nightly script, and issue a VACUUM FULL to tables that have already been processed. But I'm

Re: Logical replication, need to reclaim big disk space

2025-05-19 Thread Moreno Andreo
pt, and issue a VACUUM FULL to tables that have already been processed. But I'm in a logical replication architecture (we are using a multimaster system called pgEdge, but I don't think it will make big difference, since it's based on logical replication), and I'm building a

Re: Logical replication, need to reclaim big disk space

2025-05-16 Thread Achilleas Mantzios
as you may imagine. Running a VACUUM FULL on an ex-22GB table on a standalone test server is almost immediate. If I had only one server, I'll process a table a time, with a nightly script, and issue a VACUUM FULL to tables that have already been processed. But I'm in a logical

Logical replication, need to reclaim big disk space

2025-05-16 Thread Moreno Andreo
nightly script, and issue a VACUUM FULL to tables that have already been processed. But I'm in a logical replication architecture (we are using a multimaster system called pgEdge, but I don't think it will make big difference, since it's based on logical replication), and I'

Re: replication

2025-04-25 Thread Achilleas Mantzios - cloud
On 4/25/25 14:50, Marc Millas wrote: Hello, One of my customers tells me he did a replication with the master on a debian 10 and the slave on a debian 12, both with a pg13.20, one on a pgdg110+1, the other a pgdg120.1 is this a good idea ??? (I don't think so...) Sorry I dont quit

replication

2025-04-25 Thread Marc Millas
Hello, One of my customers tells me he did a replication with the master on a debian 10 and the slave on a debian 12, both with a pg13.20, one on a pgdg110+1, the other a pgdg120.1 is this a good idea ??? (I don't think so...) thanks, Marc MILLAS Senior Architect +33607850334

Re: find replication slots that "belong" to a publication

2025-04-10 Thread Willy-Bas Loos
12:16 +0200, Willy-Bas Loos wrote: > > My question is not so much about "can i drop a certain replication > slot", > > more about "does this publication still have any replication slots?". > > Or, if you will: "what's the publication for this rep

Re: find replication slots that "belong" to a publication

2025-04-07 Thread Laurenz Albe
On Mon, 2025-04-07 at 22:32 +0200, Willy-Bas Loos wrote: > I find it very strange, because the publication is needed to make a > subscription, which makes the slot. Right, but that information is only on the subscriber. Yours, Laurenz Albe

Re: find replication slots that "belong" to a publication

2025-04-07 Thread Adrian Klaver
On 4/7/25 13:32, Willy-Bas Loos wrote: Hi Laurenz, Thanks for answering! I find it very strange, because the publication is needed to make a subscription, which makes the slot. From here: https://www.postgresql.org/docs/current/logical-replication-subscription.html "A subscription de

Re: find replication slots that "belong" to a publication

2025-04-07 Thread Justin
I can't think of a way to link publication to a replication slot I agree using pg_state_activity is the only way to do that however you don't know if the subscriber is momentary disconnected due network error or disconnected due to an error in replication such as duplicated key S

Re: find replication slots that "belong" to a publication

2025-04-07 Thread Laurenz Albe
On Mon, 2025-04-07 at 12:16 +0200, Willy-Bas Loos wrote: > My question is not so much about "can i drop a certain replication slot",  > more about "does this publication still have any replication slots?". > Or, if you will: "what's the publication for th

Re: find replication slots that "belong" to a publication

2025-04-07 Thread Willy-Bas Loos
Hi Justin, thanks for your anwer! My question is not so much about "can i drop a certain replication slot", more about "does this publication still have any replication slots?". Or, if you will: "what's the publication for this replication slot?". I've dou

Re: find replication slots that "belong" to a publication

2025-04-06 Thread Justin
On Fri, Apr 4, 2025 at 4:58 AM Willy-Bas Loos wrote: > Hi! > > I'm looking for a way to find out if there are still replication slots > active for a publication before dropping the publication in an automated > way. The idea is that the publication is thought not to be need

Create replication slot options

2025-04-05 Thread Phillip Diffley
I am testing out some streaming logical replication commands and am having trouble specifying options when calling CREATE_REPLICATION_SLOT <https://www.postgresql.org/docs/current/protocol-replication.html#PROTOCOL-REPLICATION-CREATE-REPLICATION-SLOT> . I connect to the database with psql &

Replication slot WAL reservation

2025-04-05 Thread Phillip Diffley
I am trying to understand how logical replication slots work, specifically in regard to how WAL is reserved and freed by a replication slot. My current understanding of the WAL (set to wal_level logical) is that: 1. Every DML operation (insert, update, delete, truncate) will have a row in the WAL

Re: find replication slots that "belong" to a publication

2025-04-04 Thread Willy-Bas Loos
postgres 13 BTW On Fri, Apr 4, 2025 at 10:58 AM Willy-Bas Loos wrote: > Hi! > > I'm looking for a way to find out if there are still replication slots > active for a publication before dropping the publication in an automated > way. The idea is that the publication is thou

find replication slots that "belong" to a publication

2025-04-04 Thread Willy-Bas Loos
Hi! I'm looking for a way to find out if there are still replication slots active for a publication before dropping the publication in an automated way. The idea is that the publication is thought not to be needed any longer, but we want to make sure. I'm having trouble finding a link

Re: Replication slot WAL reservation

2025-04-04 Thread Christophe Pettus
> On Mar 25, 2025, at 13:58, Phillip Diffley wrote: > > Oh I see! I was conflating the data I see coming out of a replication slot > with the internal organization of the WAL. I think the more specific question > I am trying to answer is, as a consumer of a replication

Re: Replication slot WAL reservation

2025-03-26 Thread Phillip Diffley
gt; successfully consumed? > > You're looking for Standby Status Update: > > > https://www.postgresql.org/docs/current/protocol-replication.html#PROTOCOL-REPLICATION-STANDBY-STATUS-UPDATE > > The logical replication protocol is a superset of the binary replication > protocol, s

Re: Replication slot WAL reservation

2025-03-26 Thread Christophe Pettus
a logical replica). For a logical replica, that can be confirmed_flush_lsn or any point after, but it can't be before. (Even if the WAL exists, the primary will return an error if the start point provided in START_REPLICATION is before confirmed_flush_lsn for a logical replication slot.) Of

Re: Replication slot WAL reservation

2025-03-25 Thread Christophe Pettus
> On Mar 25, 2025, at 20:56, Phillip Diffley wrote: > > Is there a message type that is used to confirm what logs have been > successfully consumed? You're looking for Standby Status Update: https://www.postgresql.org/docs/current/protocol-replication.html#PRO

Re: Replication slot WAL reservation

2025-03-25 Thread Phillip Diffley
> You shouldn't need to manually advance the replication slot. > The client is also expected to send back regular messages letting the publisher / primary know that it has successfully consumed up to a particular point I was thinking of these as the same thing, but it sounds li

Re: Replication slot WAL reservation

2025-03-25 Thread Phillip Diffley
Oh I see! I was conflating the data I see coming out of a replication slot with the internal organization of the WAL. I think the more specific question I am trying to answer is, as a consumer of a replication slot, how do I reason about what replication records will be made unavailable when I

Re: Replication slot WAL reservation

2025-03-25 Thread Christophe Pettus
Missed this question! > On Mar 25, 2025, at 09:56, Phillip Diffley wrote: > But when processing data from a replication slot, we confirm rows that have > been processed and can be deleted from the WAL based on the LSN (eg. with > pg_replication_slot_advance). How does postgres i

Re: Replication slot WAL reservation

2025-03-25 Thread Christophe Pettus
> On Mar 25, 2025, at 09:56, Phillip Diffley wrote: > 1. Every DML operation (insert, update, delete, truncate) will have a row in > the WAL and that row will have an LSN assigned to it. > 2. The LSNs are assigned when the operation happens. > 3. Operations within a transaction are written to

Re: Create replication slot options

2025-03-24 Thread Christophe Pettus
> On Mar 24, 2025, at 17:31, Phillip Diffley wrote: > > I am testing out some streaming logical replication commands and am having > trouble specifying options when calling CREATE_REPLICATION_SLOT. > > I connect to the database with > psql "dbname=replication_te

PG16 and replication, ensure a clean switchover after a stop of the primary server

2025-02-26 Thread François Lafont
Hi, First, I have put all details of this post in this gist https://gist.github.com/flaf/ccedf78d0290d231e79581077fd92dbc (with a little video/demo to see with your eyes ;)). I have 2 PostgreSQL servers version 16.8 on RedHat 8.10, with physical streaming async replication: * postgres-1 the

Re: Records count mismatch with logical replication

2025-01-23 Thread Adrian Klaver
On 1/23/25 10:21, Durgamahesh Manne wrote: At subscription side getting this error repeatedly  So Do we have any solution without resync full data again from scratch * |2025-01-23 18:11:46 UTC::@:[507]:DEBUG: logical replication did not find row to be updated in replication target

Re: Records count mismatch with logical replication

2025-01-23 Thread Adrian Klaver
On 1/23/25 09:54, Durgamahesh Manne wrote: See comments in line below. Source Publication Side: archiving=> select * from pg_stat_replication; There is missing information here. Am I right in assuming this is for slot cls_eva_msa? And that it going to same client_addr 10.80.0.168? cl

Re: Records count mismatch with logical replication

2025-01-23 Thread Durgamahesh Manne
> But records count varies with difference of more than 10 thousand >> > >> > Have you looked at the I/0 statistics between the Postgres instances? >> > >> > Seems everything looks good with pg replication slots >> >> Except the subscriber is lagging

Re: Records count mismatch with logical replication

2025-01-23 Thread Durgamahesh Manne
res instances? > > > > Seems everything looks good with pg replication slots > > Except the subscriber is lagging behind the publisher. > > '... everything looks good' is an opinion not actual data. > > > > > Does this pg logical slot get changes func

Re: Records count mismatch with logical replication

2025-01-23 Thread Adrian Klaver
On 1/22/25 18:53, Durgamahesh Manne wrote: > But records count varies with difference of more than 10 thousand Have you looked at the I/0 statistics between the Postgres instances? Seems everything looks good with pg replication slots Except the subscriber is lagging behind

Re: Records count mismatch with logical replication

2025-01-22 Thread Durgamahesh Manne
I/0 statistics between the Postgres instances? > > > > > How to mitigate this issue in simplest way ? > > Until it is determined what is causing the lag there is no way to deal > with it. > > > > > Regards > > Durga Mahesh > > -- > Adrian Klaver

Re: Records count mismatch with logical replication

2025-01-21 Thread Adrian Klaver
On 1/21/25 11:40, Durgamahesh Manne wrote: On Wed, 22 Jan, 2025, 00:22 Adrian Klaver, > wrote: On 1/21/25 10:06 AM, Durgamahesh Manne wrote: > > Hi Adrian Klaver > > 22,906,216 bytes/10,846 rows  works out to 2112 bytes per row.

Re: Records count mismatch with logical replication

2025-01-21 Thread Durgamahesh Manne
On Wed, 22 Jan, 2025, 00:22 Adrian Klaver, wrote: > > > On 1/21/25 10:06 AM, Durgamahesh Manne wrote: > > > > > Hi Adrian Klaver > > > > 22,906,216 bytes/10,846 rows works out to 2112 bytes per row. > > > > Is that a reasonable per row estimate? > > > > Yes sometimes would be vary > > If I am f

Re: Records count mismatch with logical replication

2025-01-21 Thread Adrian Klaver
On 1/21/25 10:06 AM, Durgamahesh Manne wrote: Hi Adrian Klaver 22,906,216 bytes/10,846 rows  works out to 2112 bytes per row. Is that a reasonable per row estimate? Yes  sometimes would be vary If I am following the lag went from 350GB behind to 22MB. Is the issue that the lag has stal

Re: Records count mismatch with logical replication

2025-01-21 Thread Durgamahesh Manne
On Tue, Jan 21, 2025 at 11:26 PM Adrian Klaver wrote: > On 1/21/25 09:38, Durgamahesh Manne wrote: > > > > > > > > > Hi Adrian Klaver > > > > Really Thanks for your quick response > > > > This happened during repack lag went to more than 350Gb then gradually > > decreased to minimal lag after run

Re: Records count mismatch with logical replication

2025-01-21 Thread Adrian Klaver
On 1/21/25 09:38, Durgamahesh Manne wrote: Hi Adrian Klaver Really Thanks for your quick response This happened during repack lag went to more than 350Gb then gradually decreased to minimal lag after running pg_repack I don't use pg_repack so I don't know what effect it would have on t

Re: Records count mismatch with logical replication

2025-01-21 Thread Durgamahesh Manne
On Tue, Jan 21, 2025 at 9:24 PM Adrian Klaver wrote: > On 1/21/25 04:08, Durgamahesh Manne wrote: > > Hi Team, > > > > I have publication and subscription servers .So seems data replication > > running with minimal lag but records count mismatch with more than 10 &

Re: Records count mismatch with logical replication

2025-01-21 Thread Adrian Klaver
On 1/21/25 04:08, Durgamahesh Manne wrote: Hi  Team, I have publication and subscription servers .So seems data replication running with minimal lag but records count mismatch with more than 10 thousand records between source and destination tables Could you please help in resolving this

Records count mismatch with logical replication

2025-01-21 Thread Durgamahesh Manne
Hi Team, I have publication and subscription servers .So seems data replication running with minimal lag but records count mismatch with more than 10 thousand records between source and destination tables Could you please help in resolving this issue? Regards, Durga Mahesh

Re: Different Autovacuum Settings on Master and Replica in Streaming Replication

2025-01-17 Thread Guillaume Lelarge
c a >> écrit : >> >>> Question regarding autovacuum settings in a PostgreSQL streaming >>> replication setup. Specifically, I am curious about whether it is possible >>> (and advisable) to have different values for the following settings on the >>> mas

Re: Different Autovacuum Settings on Master and Replica in Streaming Replication

2025-01-17 Thread Franjo Stipanovic
17, 2025 at 4:34 PM Guillaume Lelarge wrote: > Hi, > > Le ven. 17 janv. 2025 à 15:38, Franjo Stipanovic a > écrit : > >> Question regarding autovacuum settings in a PostgreSQL streaming >> replication setup. Specifically, I am curious about whether it is possible

Re: Different Autovacuum Settings on Master and Replica in Streaming Replication

2025-01-17 Thread Guillaume Lelarge
Hi, Le ven. 17 janv. 2025 à 15:38, Franjo Stipanovic a écrit : > Question regarding autovacuum settings in a PostgreSQL streaming > replication setup. Specifically, I am curious about whether it is possible > (and advisable) to have different values for the following settings on the

Re: Different Autovacuum Settings on Master and Replica in Streaming Replication

2025-01-17 Thread Tom Lane
Franjo Stipanovic writes: > Question regarding autovacuum settings in a PostgreSQL streaming > replication setup. Specifically, I am curious about whether it is possible > (and advisable) to have different values for the following settings on the > master and repli

Different Autovacuum Settings on Master and Replica in Streaming Replication

2025-01-17 Thread Franjo Stipanovic
Question regarding autovacuum settings in a PostgreSQL streaming replication setup. Specifically, I am curious about whether it is possible (and advisable) to have different values for the following settings on the master and replica databases: - autovacuum_vacuum_scale_factor

Re: Data Out of Sync with Physical Streaming Replication

2025-01-15 Thread Tim Gerber
e784e70250","\\x019344723f7778b6b34305157fb9a339","\\x01934472b0527ce29c7d35a809bcb3f6"} (5 rows) STANDBY: issuedb=# SELECT ctid, xmin, xmax, troubleshoot_issue_id as issue_id from troubleshoot_issue; ctid | xmin | xmax | troubleshoot_issue_id ---+-

Re: Data Out of Sync with Physical Streaming Replication

2025-01-15 Thread Tomas Vondra
On 1/15/25 19:50, Tim Gerber wrote: > Hi All, > >... > > Archive mode is on.   I ran pg_amcheck and everything came back clean. I > know synchronous replication is an option, but it feels like something > else is going on and would like to get to the bottom of it. > >

Data Out of Sync with Physical Streaming Replication

2025-01-15 Thread Tim Gerber
Hi All, We have run into an issue where the data for a primary and standby Postgres instances are out of sync. We are using streaming physical replication. We have encountered this issue several times over the last year on different Postgres Clusters. Postgres indicates everything is in sync

Re: Need help in logical replication

2025-01-13 Thread Divyansh Gupta JNsThMAudy
criber table empty?? > > Thank you, > > On Mon, Jan 13, 2025 at 7:30 AM Divyansh Gupta JNsThMAudy < > ag1567...@gmail.com> wrote: > >> Hii PostgreSQL Community, >> >> I am setting up logical replication between two clusters in the same >> subnet g

Re: Need help in logical replication

2025-01-13 Thread Justin
> Hii PostgreSQL Community, > > I am setting up logical replication between two clusters in the same > subnet group. I’ve created a publication on the primary and a subscription > on the secondary, and the replication slot has been created. However, the > slot remains inactive, and chan

Re: Need help in logical replication

2025-01-13 Thread Rob Sargent
> On Jan 13, 2025, at 5:30 AM, Divyansh Gupta JNsThMAudy > wrote: > >  > Hii PostgreSQL Community, > > I am setting up logical replication between two clusters in the same subnet > group. I’ve created a publication on the primary and a subscription on the > sec

Need help in logical replication

2025-01-13 Thread Divyansh Gupta JNsThMAudy
Hii PostgreSQL Community, I am setting up logical replication between two clusters in the same subnet group. I’ve created a publication on the primary and a subscription on the secondary, and the replication slot has been created. However, the slot remains inactive, and changes aren’t being

Re: Query related to Logical Replication using test_decoding and unchanged-toast-datum

2025-01-06 Thread David G. Johnston
On Mon, Jan 6, 2025 at 7:21 AM Kiran K V wrote: > could you please tell me whether PostgreSQL will truly log these values to > WAL or not ? If not, what will be entered into WAL for the JSON column if > it remains unchanged? > > Not quite sure what you are looking for but: https://www.postgresql

Query related to Logical Replication using test_decoding and unchanged-toast-datum

2025-01-06 Thread Kiran K V
Hi, I am currently using PostgreSQL version 16 and the test_decoding plugin to perform logical replication (using replication slots). I have a simple table with integer column and JSON column. When a non-JSON column is updated, the value "unchanged-toast-datum" for the JSON column i

Re: Starting logical replication at arbitrary point that's available in WAL

2024-12-27 Thread Alexander Uvizhev
On 12/27/24 13:20, Zhijie Hou (Fujitsu) wrote: On Friday, December 27, 2024 7:39 PM Alexander Uvizhev wrote: Hi, I'm doing a logical replication using streaming replication protocol and I'm trying to start a stream from a certain arbitrary point that's available in WAL

RE: Starting logical replication at arbitrary point that's available in WAL

2024-12-27 Thread Zhijie Hou (Fujitsu)
On Friday, December 27, 2024 7:39 PM Alexander Uvizhev wrote: Hi, > I'm doing a logical replication using streaming replication protocol and > I'm trying to start a stream from a certain arbitrary point that's > available in WAL. However, both CRE

Starting logical replication at arbitrary point that's available in WAL

2024-12-27 Thread Alexander Uvizhev
Hi, I'm doing a logical replication using streaming replication protocol and I'm trying to start a stream from a certain arbitrary point that's available in WAL. However, both CREATE_REPLICATION_SLOT and pg_create_logical_replication_slot() create slot with nearly last L

Re: Streaming replication problem with collation

2024-12-20 Thread Tom Lane
Ekaterina Amez Gonzalez writes: > I tried what was suggested: reindexing and running "refresh collation" > alter after that and everything seems to work ok so this looks like an easy > wat to migrate from one server to another. Plus I feel more comfortable > using stre

Streaming replication problem with collation

2024-12-20 Thread Ekaterina Amez Gonzalez
different glibc version between systems before I started my tests, but I tried it anyway. I first upgraded CentOS version to 15 and then made a streaming replication to the other server (Rocky). After that I encountered the next warning when connecting to new hot standby database: WARNING: datab

Continue Logical Replication After Master Became Slave and then Became Master Again

2024-12-15 Thread Avi Weinberg
Hi All, Postgres 15.2 We have Patroni cluster with one master and two replicas. The master is publisher (logical replication) to some subscriptions running on other clusters. When we have a failover, the master becomes replica and one of the replicas assume the role of master. In such a

Re: pg_upgrade vs. logical replication

2024-12-09 Thread Greg Sabino Mullane
On Mon, Dec 9, 2024 at 6:43 AM Joe Wildish wrote: Overall, your solution seems okay, but: > a fix has gone in to PG17 that sorts this problem. > > However, we can't go to 17 yet, so need a solution for 15 and 16. Honestly, this would seem like a really, really strong reason to push for v17.

pg_upgrade vs. logical replication

2024-12-09 Thread Joe Wildish
y, and the associated replication origin has no LSN information in pg_replication_origin_status. I found a thread on the hackers list that seems to call out this problem: https://www.postgresql.org/message-id/flat/20230217075433.u5mjly4d5cr4hcfe%40jrouhaud It is a very long thread. TLDR; Julien d

Re: Qsn on Setting replication using " set session_replication_role = "

2024-12-04 Thread David G. Johnston
On Wednesday, December 4, 2024, Bharani SV-forum wrote: > > > Team > Need help on clarification on using > " set session_replication_role " > > Assume i am loggging into > psql > and issue > \o output.lst > set session_replication_role=default > update > This seems like a bad idea. Why wou

Qsn on Setting replication using " set session_replication_role = "

2024-12-04 Thread Bharani SV-forum
  TeamNeed help on clarification on using" set  session_replication_role " Assume i am loggging intopsql and issue\o output.lstset session_replication_role=defaultupdate coming out of \psql command and verifying log fileand assume i come out of the psql session without doing set session_r

Re: Questions on Upgrading PostgreSQL from 15.0 to 15.9 and Setting Up Streaming Replication

2024-11-25 Thread Subhash Udata
primary server and a standby > >server. The database is currently running on PostgreSQL 15.0, and I > plan > >to upgrade both servers to 15.9. > > > >I have the following questions regarding the upgrade and replication > >process: > > > >

Re: Questions on Upgrading PostgreSQL from 15.0 to 15.9 and Setting Up Streaming Replication

2024-11-25 Thread Ilya Anfimov
servers to 15.9. > >I have the following questions regarding the upgrade and replication >process: > > 1. Upgrade and Replication Compatibility: > > * My plan is to perform a failover, promote the standby server > (currently 15.0) to pr

Re: Questions on Upgrading PostgreSQL from 15.0 to 15.9 and Setting Up Streaming Replication

2024-11-24 Thread Ron Johnson
<mailto:ronljohnso...@gmail.com>> wrote: > > > > > > Doesn't the existence of a replication slot force PG to retain > > WAL files when replication is broken? > > > > It does. I don't recall if the OP said that they were using a > >

Re: Questions on Upgrading PostgreSQL from 15.0 to 15.9 and Setting Up Streaming Replication

2024-11-24 Thread Adrian Klaver
On 11/24/24 13:00, Ron Johnson wrote: On Sun, Nov 24, 2024 at 2:55 PM Christophe Pettus <mailto:x...@thebuild.com>> wrote: > On Nov 24, 2024, at 09:15, Ron Johnson mailto:ronljohnso...@gmail.com>> wrote: > > Doesn't the existence of a replication sl

Re: Questions on Upgrading PostgreSQL from 15.0 to 15.9 and Setting Up Streaming Replication

2024-11-24 Thread Ron Johnson
On Sun, Nov 24, 2024 at 2:55 PM Christophe Pettus wrote: > > On Nov 24, 2024, at 09:15, Ron Johnson wrote: > > > > Doesn't the existence of a replication slot force PG to retain WAL files > when replication is broken? > > It does. I don't recall if the OP s

Re: Questions on Upgrading PostgreSQL from 15.0 to 15.9 and Setting Up Streaming Replication

2024-11-24 Thread Christophe Pettus
> On Nov 24, 2024, at 09:15, Ron Johnson wrote: > > Doesn't the existence of a replication slot force PG to retain WAL files when > replication is broken? It does. I don't recall if the OP said that they were using a persistent replication slot or not; it'

Re: Questions on Upgrading PostgreSQL from 15.0 to 15.9 and Setting Up Streaming Replication

2024-11-24 Thread Subhash Udata
Pettus wrote: > > > > On Nov 24, 2024, at 09:03, Subhash Udata wrote: > > When we shut down the standby, upgrade it, and then start it back up, > will the replication automatically resume from the primary to the standby? > > Assuming that the standby has access to a

Re: Questions on Upgrading PostgreSQL from 15.0 to 15.9 and Setting Up Streaming Replication

2024-11-24 Thread Christophe Pettus
> On Nov 24, 2024, at 09:03, Subhash Udata wrote: > When we shut down the standby, upgrade it, and then start it back up, will > the replication automatically resume from the primary to the standby? Assuming that the standby has access to any WAL generated during the shutdown (eit

Re: Questions on Upgrading PostgreSQL from 15.0 to 15.9 and Setting Up Streaming Replication

2024-11-24 Thread Ron Johnson
On Sun, Nov 24, 2024 at 12:06 PM Christophe Pettus wrote: > > On Nov 24, 2024, at 09:03, Subhash Udata wrote: > > When we shut down the standby, upgrade it, and then start it back up, > will the replication automatically resume from the primary to the standby? > > Assuming

Re: Questions on Upgrading PostgreSQL from 15.0 to 15.9 and Setting Up Streaming Replication

2024-11-24 Thread Christophe Pettus
> On Nov 24, 2024, at 08:51, Subhash Udata wrote: > However, my concern lies in the fact that we are working with production > servers, where downtime is not acceptable. There is no way to upgrade community PostgreSQL, either to a new minor version or a new major version, with absolute zero

Re: Questions on Upgrading PostgreSQL from 15.0 to 15.9 and Setting Up Streaming Replication

2024-11-24 Thread Subhash Udata
Thank you for your valuable suggestion! I have a question regarding the process: When we shut down the standby, upgrade it, and then start it back up, will the replication automatically resume from the primary to the standby? Looking forward to your clarification. 2) What do you mean by

Re: Questions on Upgrading PostgreSQL from 15.0 to 15.9 and Setting Up Streaming Replication

2024-11-24 Thread Ron Johnson
ng with production > servers, where downtime is not acceptable. > > Additionally, if a failover occurs due to a network issue or any other > disaster, setting up replication again requires running the pg_basebackup > command. For large databases, this process becomes a significant chal

Re: Questions on Upgrading PostgreSQL from 15.0 to 15.9 and Setting Up Streaming Replication

2024-11-24 Thread Subhash Udata
failover occurs due to a network issue or any other disaster, setting up replication again requires running the pg_basebackup command. For large databases, this process becomes a significant challenge, as running pg_basebackup for the entire cluster can be time-consuming and resource-intensive. On Sun

Re: Questions on Upgrading PostgreSQL from 15.0 to 15.9 and Setting Up Streaming Replication

2024-11-24 Thread Ron Johnson
On Sun, Nov 24, 2024 at 11:41 AM Adrian Klaver wrote: > On 11/24/24 08:36, Subhash Udata wrote: > > The reason to upgrade from 15.0 to 15.9 is this > > https://www.postgresql.org/support/security/CVE-2024-10979/ > > > > > > Here it is

Re: Questions on Upgrading PostgreSQL from 15.0 to 15.9 and Setting Up Streaming Replication

2024-11-24 Thread Ron Johnson
I have the following questions regarding the upgrade and replication > process: > >1. > >*Upgrade and Replication Compatibility*: >- My plan is to perform a failover, promote the standby server > (currently 15.0) to primary, and then upgrade the old primary

  1   2   3   4   5   6   7   8   9   10   >