Hi Kavya,

A few thoughts for your planning. Others on this list run larger
systems than I have, so take this as what I would personally watch
rather than the last word, and corrections are welcome.

My read is that at this size the risk is less about capacity and more
about operations. Roughly in the order I would expect things to bite:

> Backup, restore of such a large database
> Recovery of a broken database (A lost replica)
> Triggering a PITR with any backup tools

To me these are really one decision: a physical, parallel, incremental
tool -- pgBackRest (or Barman) -- rather than pg_dump, which is not
viable at this size. The same tool covers all three: incremental,
block-level backup; PITR; and rebuilding a lost replica by restoring
the delta from the backup repository instead of re-streaming it all
from the primary. I would derive the full-restore wall-clock from your
storage throughput and actually test it; at a few GB/s, 300 TB is
still many hours of moving bytes.

> Fixing a standby that lags a lot

This is the one I would watch most. WAL replay on a standby is
essentially single-threaded (the startup process), while the primary
generates WAL from many backends at once. At high write volume one
core replaying can fall behind, and no network change fixes that.
recovery_prefetch (PG15+) and fast standby storage help the read side,
but the single-threaded redo is a ceiling I would plan around up front.

> Maintenance at times

I would treat partitioning as non-optional: you cannot VACUUM or
REINDEX a monolithic multi-TB table inside a window. I would also watch
transaction-ID and multixact wraparound (age(datfrozenxid) and the
multixact age) -- the latter bites FK-heavy schemas and tends to
surprise people -- and keep real disk headroom, since REINDEX or
pg_repack of a large object needs roughly 2x its size free.

A few less obvious things that tend to cause outages at scale:

- Disk filling and a hard stop (PANIC): if archiving falls behind, or
  an orphaned replication slot retains WAL after a replica dies, pg_wal
  fills. I would set max_slot_wal_keep_size and watch the archiver.
- No optimizer hints: a plan flip on a huge table can hurt and you
  cannot pin it the Oracle way. Raising statistics targets on the big
  tables and using extended statistics helps.
- HA is assembled, not built in: Patroni or repmgr for automatic
  failover. Failover promotes a standby that may be lagging, so the
  real RTO/RPO ties back to the replay point above.
- Testing a full restore on a schedule; at this size a broken backup
  tends to be found during the actual incident.

> Finding equivalents of oracle items

A rough map, as I think of it: RAC -> no shared-everything; streaming
replication + read replicas (a different model, not a drop-in). RMAN ->
pgBackRest. Data Guard -> physical replication (+ Patroni for failover).
AWR/ASH -> pg_stat_statements plus your own monitoring. Flashback ->
PITR. ora2pg for the schema/data conversion itself.

> I dont see any limits in database/ objects

The page you linked is right -- there is no hard limit on database or
object size at 300 TB. The ceilings you will actually meet are
operational (replay throughput, maintenance windows, connection
scaling -- you will want PgBouncer), not catalog limits.

One thing I would settle early: is this one instance, or sharded or
heavily partitioned? Most of the pain above scales with the size of a
single instance, so that choice changes many of the answers.

That is my read; people here have run bigger clusters and may disagree
on specifics. Happy to go deeper on any of these.

Regards,
Manu


Reply via email to