Yes, pgbackrest seems the best solution for faster backup and restore.
We can close the discussion here for now, replaying million WAL files is
just slow and this is normal.
On Thu, Aug 18, 2022 at 12:10 PM Vijaykumar Jain <
vijaykumarjain.git...@gmail.com> wrote:
> Sorry for top posting, from p
Sorry for top posting, from phone.
But pgbackrest exactly helped with that. With compression and parallel
process in backup, the backup and restore was quick. I used this, where I
took a backup and immediately did a restore so less wals to replay, else
wal replay is indeed slow.
On Thu, Aug 18, 2
Thank you, people. The big problem in my case, which I have not mentioned,
is that I think the network is a bottleneck, because I am running
pg_basebackup through internet from local country to Amazon instance in
Germany and the speed in copying is around 50 MB/sec max, that is why it
takes 2 days
I just did a backup and restore of a replica using pgbackrest.
db size 28tb
nvme/ssd storage
96 cpu, 380 gb mem
zst compression, 24 workers (backup, 12 workers restore)
2.5 hours to backup
2 hours to restore.
Wal replay is something I forgot to tune, but I could now use
https://pgbackrest.org/co
pg_backrest will certainly backup your data faster. It might be able to be
used as a seed instead of pg_basebackup.
On 8/17/22 15:06, Ivan N. Ivanov wrote:
I have a large database (~25 TB) and I want to set up streaming
replication for the first time.
My problem is that after completion of th
Thank you for your answer! I have found this tool and I will try it
tomorrow to see if this "read-ahead" feature will speed up the process.
On Wed, Aug 17, 2022 at 11:09 PM Christophe Pettus wrote:
>
>
> > On Aug 17, 2022, at 13:06, Ivan N. Ivanov
> wrote:
> >
> > How to speed up recovering of
> On Aug 17, 2022, at 13:06, Ivan N. Ivanov wrote:
>
> How to speed up recovering of WAL files?
Since you are running on your own hardware, you might take a look at:
https://github.com/TritonDataCenter/pg_prefaulter
I have a large database (~25 TB) and I want to set up streaming replication
for the first time.
My problem is that after completion of the pg_basebackup (which completed
for 2 days with --wal-method=none) now PG is replaying the WAL files from
the WAL archive directory but it can not keep up. The
