Re: setting up streaming replication, part 2

2023-10-25 Thread Guillaume Lelarge
Hi, Le mer. 25 oct. 2023 à 02:29, Brad White a écrit : > I have the replication server set up and the streaming is working. > The latest data show up, as desired. > > 3 minor issues. > > 1) I also have the WAL files being copied to a common location from the > primary server with > archive

Re: setting up streaming replication, part 2

2023-10-25 Thread Brad White
From: Ron Sent: Wednesday, October 25, 2023 4:35:59 AM To: pgsql-generallists.postgresql.org Subject: Re: setting up streaming replication, part 2 On 10/24/23 23:47, Brad White wrote: On Tue, Oct 24, 2023, 9:02 PM Ron mailto:ronljohnso...@gmail.com>> wrote: On 10/24/23 19:29, Brad White

Re: setting up streaming replication, part 2

2023-10-25 Thread Ron
On 10/24/23 23:47, Brad White wrote: On Tue, Oct 24, 2023, 9:02 PM Ron wrote: On 10/24/23 19:29, Brad White wrote: > I have the replication server set up and the streaming is working. > The latest data show up, as desired. > > 3 minor issues. > > 2) I have the conn

Re: setting up streaming replication, part 2

2023-10-24 Thread Brad White
On Tue, Oct 24, 2023, 9:02 PM Ron wrote: > On 10/24/23 19:29, Brad White wrote: > > I have the replication server set up and the streaming is working. > > The latest data show up, as desired. > > > > 3 minor issues. > > > > > 2) I have the connection string set for the replication server to > con

Re: setting up streaming replication, part 2

2023-10-24 Thread Ron
On 10/24/23 19:29, Brad White wrote: I have the replication server set up and the streaming is working. The latest data show up, as desired. 3 minor issues. 1) I also have the WAL files being copied to a common location from the primary server with       archive_command = 'copy %p "DISKST

Re: setting up streaming replication

2023-10-24 Thread Ron
On 10/24/23 13:31, Brad White wrote: On Tue, Oct 24, 2023 at 8:56 AM Ron wrote: On 10/24/23 08:14, b55white wrote: Is all that stuff with slots necessary since the backup will automatically create a temporary slot for replication? I want a permanent slot so that replication auto

Re: setting up streaming replication

2023-10-24 Thread Christophe Pettus
> On Oct 24, 2023, at 11:31, Brad White wrote: > Are you saying that once I get streaming replication set up, it quits working > when I reboot the servers once a week? Not unless the downtime is sufficiently long that the replica can't find the WAL information it needs. You can avoid this w

Re: setting up streaming replication

2023-10-24 Thread Brad White
On Tue, Oct 24, 2023 at 8:56 AM Ron wrote: > On 10/24/23 08:14, b55white wrote: > > Is all that stuff with slots necessary since the backup will automatically > create a temporary slot for replication? > > > I want a permanent slot so that replication automatically resumes if the > secondary mus

Re: setting up streaming replication

2023-10-24 Thread Ron
On 10/24/23 08:14, b55white wrote: On Oct 23, 2023 at 8:30 PM, Ron wrote: On 10/23/23 18:16, Brad White wrote: > I'm stumped. > > Using this command to set up the slave and replication on PG v 15: > "C:\Program Files\PostgreSQL\15\bin\pg_basebackup" -h 192.168.1.126 -U > pgrep_user -p 5433 -D

Re: setting up streaming replication

2023-10-24 Thread David G. Johnston
On Monday, October 23, 2023, Brad White wrote: > I'm stumped. > > Using this command to set up the slave and replication on PG v 15: > > You must, by some means: Have PostgreSQL binaries installed on the machine that is to become the replica. Have the output of pg_basebackup present on the repli

Re: setting up streaming replication

2023-10-24 Thread b55white
> > On Oct 23, 2023 at 8:30 PM, Ronwrote: > > > On 10/23/23 18:16, Brad White wrote: > > I'm stumped. > > > > Using this command to set up the slave and replication on PG v 15: > > "C:\Program Files\PostgreSQL\15\bin\pg_basebackup" -h 192.168.1.126 -U > > pgrep_user -p 5433 -D

Re: setting up streaming replication

2023-10-23 Thread Ron
On 10/23/23 18:16, Brad White wrote: I'm stumped. Using this command to set up the slave and replication on PG v 15: "C:\Program Files\PostgreSQL\15\bin\pg_basebackup" -h 192.168.1.126 -U pgrep_user -p 5433 -D "C:\Program Files\PostgreSQL\15\data" -Fp -Xs -R If I have PG running on the remote

Re: [(catch-ext)] Re: [(catch-ext)] Re: Setting up streaming replication on large database (20+ TB) for the first time

2022-08-18 Thread Ivan N. Ivanov
Yes, pgbackrest seems the best solution for faster backup and restore. We can close the discussion here for now, replaying million WAL files is just slow and this is normal. On Thu, Aug 18, 2022 at 12:10 PM Vijaykumar Jain < vijaykumarjain.git...@gmail.com> wrote: > Sorry for top posting, from p

Re: [(catch-ext)] Re: Setting up streaming replication on large database (20+ TB) for the first time

2022-08-18 Thread Vijaykumar Jain
Sorry for top posting, from phone. But pgbackrest exactly helped with that. With compression and parallel process in backup, the backup and restore was quick. I used this, where I took a backup and immediately did a restore so less wals to replay, else wal replay is indeed slow. On Thu, Aug 18, 2

Re: [(catch-ext)] Re: Setting up streaming replication on large database (20+ TB) for the first time

2022-08-18 Thread Ivan N. Ivanov
Thank you, people. The big problem in my case, which I have not mentioned, is that I think the network is a bottleneck, because I am running pg_basebackup through internet from local country to Amazon instance in Germany and the speed in copying is around 50 MB/sec max, that is why it takes 2 days

Re: Setting up streaming replication on large database (20+ TB) for the first time

2022-08-17 Thread Vijaykumar Jain
I just did a backup and restore of a replica using pgbackrest. db size 28tb nvme/ssd storage 96 cpu, 380 gb mem zst compression, 24 workers (backup, 12 workers restore) 2.5 hours to backup 2 hours to restore. Wal replay is something I forgot to tune, but I could now use https://pgbackrest.org/co

Re: Setting up streaming replication on large database (20+ TB) for the first time

2022-08-17 Thread Ron
pg_backrest will certainly backup your data faster. It might be able to be used as a seed instead of pg_basebackup. On 8/17/22 15:06, Ivan N. Ivanov wrote: I have a large database (~25 TB) and I want to set up streaming replication for the first time. My problem is that after completion of th

Re: [(catch-ext)] Re: Setting up streaming replication on large database (20+ TB) for the first time

2022-08-17 Thread Ivan N. Ivanov
Thank you for your answer! I have found this tool and I will try it tomorrow to see if this "read-ahead" feature will speed up the process. On Wed, Aug 17, 2022 at 11:09 PM Christophe Pettus wrote: > > > > On Aug 17, 2022, at 13:06, Ivan N. Ivanov > wrote: > > > > How to speed up recovering of

Re: Setting up streaming replication on large database (20+ TB) for the first time

2022-08-17 Thread Christophe Pettus
> On Aug 17, 2022, at 13:06, Ivan N. Ivanov wrote: > > How to speed up recovering of WAL files? Since you are running on your own hardware, you might take a look at: https://github.com/TritonDataCenter/pg_prefaulter

Re: Setting up streaming replication problems

2018-01-21 Thread Andreas Kretschmer
Am 22.01.2018 um 07:39 schrieb Thiemo Kellner: Hi all I try to set up synchronous streaming replication as try-out. I use my laptop with Debian 9 and PostgreSQL package 10+189.pgdg90+1. And of this PostgreSQL installation I have two clusters main (master) and main2 (hot standby). I tried wi