Re: RPMs from postgresql.org break CentOS/RHEL RPMs

2020-03-19 Thread Michael Paquier
On Thu, Mar 19, 2020 at 01:41:55PM +0100, Peter Krefting wrote: > I have a system installed where one software is running a database using the > PostgreSQL 9.2 packages included in the CentOS/RHEL 7 release. On the side, > I am trying to install another software that is using the PostgreSQL 12 > pa

Re: Passwordcheck configuration

2020-03-22 Thread Michael Paquier
On Thu, Mar 19, 2020 at 07:19:06PM -0400, Tom Lane wrote: > passwordcheck hasn't got any out-of-the-box configurability. It's mainly > meant as sample code that people could modify if they have a mind to. Here is an example: https://github.com/michaelpq/pg_plugins/tree/master/passwordcheck_extra

Re: PG12 autovac issues

2020-03-23 Thread Michael Paquier
On Mon, Mar 23, 2020 at 01:00:51PM -0700, Andres Freund wrote: > On 2020-03-23 20:47:25 +0100, Julien Rouhaud wrote: >>> - relfrozenxid, age(relfrozenxid) for the oldest table in the oldest >>> database >>> SELECT oid::regclass, age(relfrozenxid), relfrozenxid FROM pg_class WHERE >>> relfrozenx

Re: PG12 autovac issues

2020-03-23 Thread Michael Paquier
On Mon, Mar 23, 2020 at 10:40:39PM -0700, Andres Freund wrote: > On 2020-03-24 14:26:06 +0900, Michael Paquier wrote: >> Nothing really fancy: >> - autovacuum_vacuum_cost_delay to 2ms (default of v12, but we used it >> in v11 as well). >> - autovacuum_naptime = 15s >

Re: PG12 autovac issues

2020-03-25 Thread Michael Paquier
On Wed, Mar 25, 2020 at 10:39:17AM -0500, Justin King wrote: > This started happening again. DEBUG1 is enabled: Thanks for enabling DEBUG1 logs while this happened. > Mar 25 14:48:26 cowtn postgres[39875]: [35298-1] 2020-03-25 > 14:48:26.329 GMT [39875] DEBUG: skipping redundant vacuum to preve

Re: PG12 autovac issues

2020-03-25 Thread Michael Paquier
On Wed, Mar 25, 2020 at 07:59:56PM -0700, Andres Freund wrote: > FWIW, this kind of thing is why I think the added skipping logic is a > bad idea. Silently skipping things like this (same with the "bogus" > logic in datfrozenxid computation) is dangerous. I think we should > seriously consider back

Re: PG12 autovac issues

2020-03-26 Thread Michael Paquier
On Thu, Mar 26, 2020 at 09:46:47AM -0500, Justin King wrote: > Nope, it was just these tables that were looping over and over while > nothing else was getting autovac'd. I'm happy to share the full log > if you'd like. Thanks, that could help. If that's very large, it could be a problem to send

Re: PG12 autovac issues

2020-03-28 Thread Michael Paquier
On Fri, Mar 27, 2020 at 05:10:03PM -0500, Justin King wrote: > Sounds great. I will email you directly with a link! Thanks. From the logs, the infinite loop on which autovacuum jobs are stuck is clear. We have a repetitive number of anti-wraparound and non-aggressive jobs happening for 7 shared

Re: PG12 autovac issues

2020-03-28 Thread Michael Paquier
On Fri, Mar 27, 2020 at 08:23:03PM +0100, Julien Rouhaud wrote: > FTR we reached the 200M transaxtion earlier, and I can see multiple logs of > the > form "automatic vacuum to prevent wraparound", so non-aggressive > antiwraparound > autovacuum, all on shared relations. Thanks Julien for sharing

Re: PG12 autovac issues

2020-03-29 Thread Michael Paquier
On Sat, Mar 28, 2020 at 11:29:41AM -0700, Andres Freund wrote: > I assume you're still trying to track the actual cause of the problem > further? That's the plan, and I'll try to spend some time on it next week. Any new information I have will be added to the thread you have begun on -hackers a c

Re: PG12 autovac issues

2020-03-30 Thread Michael Paquier
On Sat, Mar 28, 2020 at 05:53:59PM +0900, Michael Paquier wrote: > And I'll follow up there with anything new I find. Please let me know > if there are any objections with the revert though, this will address > the problem reported by Justin. Okay. Done with this part now as of

Re: How to prevent master server crash if hot standby stops

2020-04-06 Thread Michael Paquier
On Mon, Apr 06, 2020 at 11:03:20AM +0300, Andrus wrote: > If slave stops responing, master server will create files in pg_wal directory. > If disk becomes full, master server crashes also. > > How to avoid this ? > > If disk is nearly full, master should stop additional files creation > (and mayb

Re: Using of --data-checksums

2020-04-07 Thread Michael Paquier
On Tue, Apr 07, 2020 at 08:10:13AM -0700, BGoebel wrote: > initdb --data-checksums "... help to detect corruption by the I/O system" > There is an (negligible?) impact on performance, ok. > > Is there another reason NOT to use this feature ? > Has anyone had good or bad experience with the use o

Re: Using of --data-checksums

2020-04-11 Thread Michael Paquier
On Fri, Apr 10, 2020 at 04:37:46PM -0400, Stephen Frost wrote: > There's definitely a lot of reasons to want to have the ability to > change an existing cluster. Considering the complications around > running pg_upgrade already, I don't really think that changing the > default of initdb would be t

Re: Using of --data-checksums

2020-04-12 Thread Michael Paquier
On Sun, Apr 12, 2020 at 10:23:24AM -0400, Tom Lane wrote: > Magnus Hagander writes: >> And FWIW, I do think we should change the default. And maybe spend some >> extra effort on the message coming out of pg_upgrade in this case to make >> it clear to people what their options are and exactly what

Re: pg_restore: could not close data file: Success

2020-04-15 Thread Michael Paquier
On Thu, Apr 16, 2020 at 12:08:09PM +0900, Kyotaro Horiguchi wrote: > I'm surprised to find an old thread about the same issue. > > https://www.postgresql.org/message-id/20160307.174354.251049100.horiguchi.kyotaro%40lab.ntt.co.jp > > But I don't think it's not acceptable that use fake errno for gz

Re: Using of --data-checksums

2020-04-16 Thread Michael Paquier
On Thu, Apr 16, 2020 at 03:47:34PM -0700, Jeremy Schneider wrote: > Data checksums are a hard requirement across the entire RDS PostgreSQL > fleet - we do not allow it to be disabled in RDS. I've definitely seen a > lot of hard evidence (for example, customer cases I've personally been > involved i

Re: Can I tell libpq to connect to the primary?

2020-04-24 Thread Michael Paquier
On Thu, Apr 23, 2020 at 08:54:35AM +0200, Christian Ramseyer wrote: > Never mind, after RTFM'ing to the very end of > https://www.postgresql.org/docs/current/libpq-connect.html I have > discovered target_session_attrs=read-write|any which seems to do exactly > what I want. Yes, as long as you don'

Re: [GENERAL] import .sql file into PostgreSQL database

2020-05-09 Thread Michael Paquier
On Thu, May 07, 2020 at 11:46:02PM -0400, Gaurav wrote: > Just going thru old emails and came across this one. Thought to text you. > How have you been? What are you up to? How can I rejoin this group? Fine thanks. In order to subscribe to one or more mailing lists, please feel free to look here:

Re: [EXTERNAL] Re: PostgreSQL-12 replication failover, pg_rewind fails

2020-05-13 Thread Michael Paquier
On Wed, May 13, 2020 at 04:58:15AM +, Mariya Rampurawala wrote: > Thank you Kyotaro and Laurenz for your quick responses. > This helped me get my setup working. Please note that we have added in Postgres 13 the possibility to use a restore_command when using pg_rewind if the parameter is set i

Re: Reuse an existing slot with a new initdb

2020-05-13 Thread Michael Paquier
On Wed, May 13, 2020 at 02:12:45PM -0700, live-school support wrote: > I didn't recal that it was not possible to create a hot standby with a fresh > new install and pg_dumpall :(. > > only pg_basebackup or an exact copy of the data folder can do it right? is > the reason technical or else? When u

Re: Practical usage of large objects.

2020-05-13 Thread Michael Paquier
On Wed, May 13, 2020 at 01:55:48PM -0400, Tom Lane wrote: > Dmitry Igrishin writes: >> As you know, PostgreSQL has a large objects facility [1]. I'm curious >> are there real systems which are use this feature? > > We get questions about it regularly, so yeah people use it. I recall that some ap

Re: Check what has been done for a uncommitted prepared transaction

2020-05-14 Thread Michael Paquier
On Thu, May 14, 2020 at 03:38:24PM +0800, Andy Fan wrote: > I want to know what happens been done for an uncommitted prepared > transaction with pg_waldump, however I can't find it. > > demo=*# prepare transaction 's'; > PREPARE TRANSACTION > demo=# insert into mm select generate_series(1, 1000);

Re: How to recover from compressed wal archieve in windows

2020-05-20 Thread Michael Paquier
On Wed, May 20, 2020 at 11:36:09AM -0700, Adrian Klaver wrote: > The next problem is that I'm pretty sure a WAL file with *.gz extension will > not be able to be processed directly by the server. So you are going to have > to uncompress it at some point before it gets restored. The short answer to

Re: How to recover from compressed wal archieve in windows

2020-05-20 Thread Michael Paquier
On Thu, May 21, 2020 at 01:38:11PM +0900, Kyotaro Horiguchi wrote: > Difference from decompression by restore_command? > > A WAL (segment) file is filled with multiple WAL records. The "full > page image", which is described to be compressed by the parameter, is > a part of WAL record. A WAL file

Re: Query returns no rows in pg_basebackup cluster

2020-05-24 Thread Michael Paquier
On Fri, May 22, 2020 at 09:53:37AM +0300, Andrus wrote: > Backup in created in Windows from Linux server using pg_receivewal and > pg_basebackup . > Can this backup used for PITR in Linux ? No. Physical copies need to be based on the same platform. If you wish to replicate a cluster without any

Re: Query returns no rows in pg_basebackup cluster

2020-05-24 Thread Michael Paquier
On Mon, May 25, 2020 at 09:02:49AM +0300, Andrus wrote: > Will logical replication also allow two modes: > 1. PITR recovery can used if needed > 2. Hot standby: User databases in both clusters contain same data. > > How to set logical replication for all user databases in cluster so that > when

Re: lib/libecpg.so.6.11 && valgrind

2020-05-30 Thread Michael Paquier
On Fri, May 29, 2020 at 02:37:29PM +0200, Matthias Apitz wrote: > We're getting to the finish line in the port of our servers to > PostgreSQL 11.4. and started valgrind'ing the code, as the PostgreSQL is > new in the servers. Of course we have our own homework to do, but there > are also complaints

Re: Logical replication - ERROR: could not send data to WAL stream: cannot allocate memory for input buffer

2020-06-07 Thread Michael Paquier
On Fri, Jun 05, 2020 at 10:57:46PM +0200, Aleš Zelený wrote: > we are using logical replication for more than 2 years and today I've found > new not yet know error message from wal receiver. The replication was in > catchup mode (on publisher side some new tables were created and added to > publica

Re: Parallel safety of contrib extensions

2020-06-11 Thread Michael Paquier
On Wed, Jun 10, 2020 at 12:40:54PM -0400, Tom Lane wrote: > "Winfield, Steven" writes: >> There was a thread about this back in 2016[1], but I've just been >> bitten by it and wondered if any (more) extensions, particularly >> btree_gist, will have their operators/functions verified and marked as

Re: Parallel safety of contrib extensions

2020-06-18 Thread Michael Paquier
On Thu, Jun 18, 2020 at 02:26:04PM +, Winfield, Steven wrote: > Many thanks for the pointers - I've submitted a patch. Thanks Steve, I have noticed the patch. For the sake of the archives, it is here: https://www.postgresql.org/message-id/AM5PR0901MB1587E47B1ACF23C6089DFCA3FD9B0%40AM5PR0901MB

Re: how reliable is pg_rewind?

2020-08-02 Thread Michael Paquier
On Sat, Aug 01, 2020 at 10:35:37AM -0700, Curt Kolovson wrote: > When trying to resync an old primary to become a new standby, I have found > that pg_rewind only works occasionally. How reliable/robust is pg_rewind, > and what are its limitations? We have observed that approx half our FPIs in > the

Re: How to rebuild index efficiently

2020-08-03 Thread Michael Paquier
On Mon, Aug 03, 2020 at 01:04:45PM -0500, Ron wrote: > same definition, and when that is complete, drop the old index. The > locking that is required here is modest: CREATE INDEX CONCURRENTLY > needs to lock the table briefly at a couple of points in the > operation, and dropping the old index req

Re: [EXTERNAL] RE: PostgreSQL-12 replication. Check replication lag

2020-08-05 Thread Michael Paquier
On Wed, Aug 05, 2020 at 06:36:15PM +, Mariya Rampurawala wrote: > What I want to understand is that, in case of replication link > failure, there will still be inserts happening at the master > node. In that case, how will the slave know if it is up-to-date? It cannot do that by itself, which

Re: When are largobject records TOASTed into pg_toast_2613?

2020-08-21 Thread Michael Paquier
On Fri, Aug 21, 2020 at 03:10:30PM +0200, Laurenz Albe wrote: > Indeed, system tables have no TOAST tables in PostgreSQL, so I wonder > how your "pg_largeobject" table could have grown one. FWIW, src/include/catalog/toasting.h is giving me a list of 28 catalog tables with a toast relation as of HE

Re: Autovacuum of independent tables

2020-09-08 Thread Michael Paquier
On Tue, Sep 08, 2020 at 11:16:04AM +0300, Michael Holzman wrote: > Autovacuum does not clean dead tuples of closed transactions in tableB > while there is an open transaction on tableA. > But the tables have nothing in common. They are handled by separate > applications and there are no transaction

Re: PG 13 trusted extensions and pg_available_extensions

2020-09-23 Thread Michael Paquier
On Wed, Sep 23, 2020 at 03:28:45PM +, Daniel Westermann (DWE) wrote: > I was playing a bit with trusted extensions and wondered if there is > a reason that the "trusted" flag is not exposed in pg_available_extensions. > I believe that information would be quite useful so one can easily > iden

Re: Both type of replications from a single server?

2020-10-07 Thread Michael Paquier
On Thu, Oct 08, 2020 at 11:43:26AM +0530, Srinivasa T N wrote: >Is it possible to have both type of replications (streaming and logical) > from a single server? Yes. >If I have 3 servers A,B and C, then I want to have streaming replication > from A to B whereas logical replication from A

Re: Both type of replications from a single server?

2020-10-08 Thread Michael Paquier
On Thu, Oct 08, 2020 at 01:25:14PM +0530, Srinivasa T N wrote: > For streaming replication, I need to set wal_level to replica in A whereas > for logical_replication we need to set wal_level to replica in the same A > server. So, was wondering how to go about? A logical replica needs wal_level =

Re: rum index supported on pg13?

2020-10-11 Thread Michael Paquier
On Thu, Oct 08, 2020 at 09:29:31PM -0500, John the Scott wrote: > will rum index from postgrespro be supported in pg13? > numerous errors occur when compiling rum in pg13 and > no replies from github. the differences from pg12 > to pg13 seem to be significant > > https://github.com/postgresp

Re: rum index supported on pg13?

2020-10-12 Thread Michael Paquier
On Mon, Oct 12, 2020 at 12:17:04PM -0500, John the Scott wrote: > I am still new to github protocol, so i was not sure > if asking about longer term support of rum was appropriate for > the github issues posting. Most of the original developers of rum are registered on this mailing list so there w

Re: PG 9.2 slave restarted - cache not impacted

2020-10-22 Thread Michael Paquier
On Fri, Oct 23, 2020 at 11:23:20AM +1300, Lucas Possamai wrote: > I'm a bit confused about PG cache. > > I have a PostgreSQL 9.2 cluster (yes, we're planning on upgrading it to 12) > with a master and a slave database. > > The application is sending all read requests to the slave, where the maste

Re: PANIC: could not write to log file {} at offset {}, length {}: Invalid argument

2020-11-04 Thread Michael Paquier
On Wed, Nov 04, 2020 at 01:24:46PM +0100, Andreas Kretschmer wrote: >> Any ideas about what is the problem? or anything else I need to check? > > wild guess: Antivirus Software? Perhaps not. To bring more context in here, PostgreSQL opens any files on WIN32 with shared writes and reads allowed t

Re: PANIC: could not write to log file {} at offset {}, length {}: Invalid argument

2020-11-05 Thread Michael Paquier
On Thu, Nov 05, 2020 at 10:21:40AM +0100, Magnus Hagander wrote: > The problem with AVs generally doesn't come from them opening files in > non-share mode (I've, surprisingly enough, seen backup software that > causes that problem for example). It might happen on scheduled scans > for example, but

Re: PANIC: could not write to log file {} at offset {}, length {}: Invalid argument

2020-11-05 Thread Michael Paquier
On Wed, Nov 04, 2020 at 10:23:04PM -0500, Tom Lane wrote: > The latter case would result in a LOG message "unrecognized win32 error > code", so it would be good to know if any of those are showing up in > the postmaster log. Yeah. Not sure which one it could be here: https://docs.microsoft.com/en

Re: initdb --data-checksums

2020-11-09 Thread Michael Paquier
On Mon, Nov 09, 2020 at 06:03:43PM +0100, Paul Förster wrote: > indeed, it is. Have a look at: > > https://www.postgresql.org/docs/12/app-pgchecksums.html > > Make sure the database is cleanly shut down before doing it. This tool is really useful with upgrades after pg_upgrade. Please note that

Re: archive file "00000001000000000000006F" has wrong size: 67118648 instead of 16777216

2020-11-24 Thread Michael Paquier
On Wed, Nov 25, 2020 at 11:01:37AM +0900, 江川潔 wrote: > Hi, > > WAL log recovery was failed on wrong log record size. Could you please > advise me what is wrong in the setting ? Any suggestions will be highly > appreciated. > 2020-11-25 10:12:23.569 JST [7792] FATAL: archive file > "0001

Re: meaning of (to prevent wraparound) ..??

2020-11-25 Thread Michael Paquier
On Wed, Nov 25, 2020 at 11:10:50PM -0700, Jessica Sharp wrote: > On Wed, Nov 25, 2020 at 23:09 Atul Kumar wrote: >> Thanks Jessica. Could help me out by sharing documents that can help me >> understand “to prevent wraparound “ in simplest way, postgres doc is little >> bit harder for a newbee lik

Re: postgres-10 with FIPS

2020-12-03 Thread Michael Paquier
On Thu, Dec 03, 2020 at 05:57:04PM +0530, Aravindhan Krishnan wrote: > Since postgres is linked against openssl we wanted to make sure we build > postgres against the FIPS compliant openssl libraries. Does postgres > provide a FIPS debian package that can be used. If not it would be of great > help

Re: Postgres C-API: How to get the Oid for a custom type defined in a schema outside of the current search path

2020-12-03 Thread Michael Paquier
On Thu, Dec 03, 2020 at 01:45:05PM +0100, Pavel Stehule wrote: > When you write C extensions for Postgres, then PostgreSQL source code is > the best source of inspiration. One common source of inspiration for such cases is regproc.c. For a type, you can for example look at what to_regtype() uses

Re: Error messages on duplicate schema names

2021-01-14 Thread Michael Paquier
On Wed, Jan 06, 2021 at 07:15:24PM +0200, Andrus wrote: > Should duplicate schema names accepted or should their usage throw better > error messages. This means that we are one call of CommandCounterIncrement() short for such queries, and similar safeguards already exist in this area for GRANT/REV

Re: Error messages on duplicate schema names

2021-01-19 Thread Michael Paquier
On Tue, Jan 19, 2021 at 05:37:51PM -0300, Alvaro Herrera wrote: > I guess you could make the case that the CCI call should be in the > callers where we actually loop (SetDefaultACLsInSchemas, > RemoveRoleFromObjectACL), but it's hard to get excited about the added > clutter. Yeah, that matches my

Re: Does pg_ctl promote wait for pending WAL?

2021-01-28 Thread Michael Paquier
Hi Ishii-san, On Fri, Jan 29, 2021 at 07:59:26AM +0100, Paul Förster wrote: > On 29. Jan, 2021, at 03:51, Tatsuo Ishii wrote: >> >> Does anybody know whether a standby server waits for pending WAL >> records/files while promotion is requested? I assume that no data >> update is performed on the

Re: ransomware

2021-02-01 Thread Michael Paquier
On Mon, Feb 01, 2021 at 03:38:35PM +0100, Marc Millas wrote: > there are various ways to do those checks but I was wandering if any > ""standard''" solution exist within postgres ecosystem, or someone do have > any feedback on the topic. It seems to me that you should first write down on a sheet o

Re: Can you install/run postgresql on a FIPS enabled host?

2022-03-22 Thread Michael Paquier
On Mon, Mar 21, 2022 at 06:33:29PM -0400, Tom Lane wrote: > It sounds like something thinks that scram-sha-256 encryption is > disallowed by FIPS. That may or may not be accurate. If it's > supposed to be allowed, you'd need to poke a little harder to > narrow down where the problem is. > > (Dig

Re: What do you guys use for issue tracking, CI/CD and team management? any nice open source options?

2022-04-15 Thread Michael Paquier
On Thu, Apr 14, 2022 at 06:19:44PM +0300, Achilleas Mantzios wrote: > What issue/bug tracking is PostgreSQL itself using? > What continuous build system (CI/CD) is PostgreSQL itself using? > Any tool that you ppl or the PostgreSQL infrastructure use that > links people/committers with bugs/issues,

Re: multiple entries for synchronous_standby_names

2022-06-13 Thread Michael Paquier
On Fri, Jun 10, 2022 at 05:04:30PM +0100, Nitesh Nathani wrote: > Trying to achieve sync streaming to barman server and i need to add an > entry to postgresql.conf for this parameter, which already has an entry and > tried a few variations but does not work. Any ideas? Also tried '&&' but in > vain

Re: Postgresql error : PANIC: could not locate a valid checkpoint record

2022-06-26 Thread Michael Paquier
On Fri, Jun 24, 2022 at 01:03:57PM +, Mahendrakar, Prabhakar - Dell Team wrote: > Is it possible to explicitly issue a checkpoint before we move on to > the pg_upgrade command? > so that in the circumstances of the Upgrade issues (like PANIC: > could not locate a valid checkpoint record), we

Re: PostgreSQL 14.4 ERROR: out of memory issues

2022-07-12 Thread Michael Paquier
On Mon, Jul 11, 2022 at 10:50:23AM +0200, Aleš Zelený wrote: > So far, it has happened three times (during a single week) from the 14.3 -> > 14.4 upgrade, before 14.4 we haven't suffered from such an issue. > > Questions: > 1) Can we safely downgrade from 14.4 to 14.3 by shutting down the instanc

Re: Streaming wal from primiry terminating

2022-09-29 Thread Michael Paquier
On Thu, Sep 29, 2022 at 01:04:08PM +, Lahnov, Igor wrote: > Primary_conninfo is set in postgresql.conf > The reason is definitely something else, we do not know what. This is fatal > in this case. Which version of PostgreSQL are you using? There has been a few commits around continuation rec

Re: Incorrect resource manager data checksum in record with zfs and compression

2022-10-03 Thread Michael Paquier
On Mon, Oct 03, 2022 at 12:41:23PM -0700, John Bolliger wrote: > Our architecture is similar but all of the servers are now on ZFS now and > Postgres 13.8 with Ubuntu 18.04+ and still doing streaming replication, all > with ECC memory and 26-64 cores with 192gb ram+ on top of a ZPOOL made out > of

Re: Zheap Tech Problem

2022-10-14 Thread Michael Paquier
On Fri, Oct 14, 2022 at 07:53:17PM -0700, Adrian Klaver wrote: > On 10/14/22 18:59, jacktby wrote: >> What's Zheap tech? Can you give me some details or stuff to study? and >> which version will be realized in ? There are a few videos on youtube that can provide some insight about all that, mainly

Re: empty pg_stat_progress_vacuum

2022-10-31 Thread Michael Paquier
On Fri, Oct 21, 2022 at 10:21:23PM +, senor wrote: > I'm mainly wanting to understand why I'm not seeing processes in > pg_stat_progress_vacuum. If I rapidly refresh I occasionally see an > entry for a very small table. A manually started vacuum didn't show > up either. It may be possible that

Re: Q: pg_hba.conf separate database names file format

2022-11-08 Thread Michael Paquier
On Tue, Nov 08, 2022 at 02:16:03PM +0100, Albrecht Dreß wrote: > However, I could not find a specification of the format for this > file… It appears as if simply giving each database name on a > separate line does the job. Is this correct? May the file contain > comments (i.e. lines starting wit

Re: Q: pg_hba.conf separate database names file format

2022-11-09 Thread Michael Paquier
On Wed, Nov 09, 2022 at 04:02:43AM -0600, Ron wrote: > Are these "include" files supposed to solve the problem of having a *lot* of > databases (or users) that you want to allow access to? Yes, splitting the list of users and database eases the maintenance and readability of pg_hba.conf as each HB

Re: Libpq linked to LibreSSL

2022-12-04 Thread Michael Paquier
On Sun, Dec 04, 2022 at 09:02:07AM +0100, Marco Bambini wrote: > After several attempts (and runtime crashes), I am asking for help > with how to compile libpq with LibreSSL support (both dynamic and > static links would be OK to me). > I know how to compile libpq with OpenSSL support, but I need t

Re: Binding Postgres to port 0 for testing

2023-03-26 Thread Michael Paquier
On Sat, Mar 25, 2023 at 11:01:33AM -0600, Markus Pilman wrote: > Now the problem is that I need to find a TCP port for each running postgres > instance. There's multiple ways to do this, but by far the easiest one I > know is to bind to port 0. So my plan was to start postgres with "-p 0" and > the

Re: Binding Postgres to port 0 for testing

2023-03-26 Thread Michael Paquier
On Sun, Mar 26, 2023 at 10:49:33PM -0600, Markus Pilman wrote: > I somehow didn't consider looking at the postgres tests, though it makes > sense that they need to solve this problem. If I read the perl code > correctly though it seems that this could, in theory, cause a race? The > script checks f

Re: "PANIC: could not open critical system index 2662" - twice

2023-04-07 Thread Michael Paquier
On Fri, Apr 07, 2023 at 01:04:34PM +0200, Laurenz Albe wrote: > On Thu, 2023-04-06 at 16:41 +, Evgeny Morozov wrote: >> Could this be a PG bug? > > It could be, but data corruption caused by bad hardware is much more likely. There is no way to be completely sure here, except if we would be ab

Re: "PANIC: could not open critical system index 2662" - twice

2023-04-11 Thread Michael Paquier
On Tue, Apr 11, 2023 at 04:44:54PM +, Evgeny Morozov wrote: > We have data_checksums=on. (It must be on by default, since I cannot > find that in our config files anywhere.) initdb does not enable checksums by default, requiring a -k/--data-checksums, so likely this addition comes from from yo

Re: pg_basebackup / recovery

2023-04-12 Thread Michael Paquier
On Wed, Apr 12, 2023 at 01:45:56PM +0300, Achilleas Mantzios - cloud wrote: > On 4/12/23 12:32, Fabrice Chapuis wrote: >> During recovery process of a self contained backup, how postgres know to >> stop reading wal when consistency is reached? > > Because it knows the full packup info. It will obs

Re: Oracle vs PG

2018-10-23 Thread Michael Paquier
On Wed, Oct 24, 2018 at 07:31:57AM +0200, Laurenz Albe wrote: > I have seen people use savepoints in PostgreSQL to emulate Oracle's > "statement rollback" behavior: If a statement fails, only the statement > is undone, but the transaction continues. > > If you insert a savepoint before *every* sta

Re: Truncation of UNLOGGED tables upon restart.

2018-11-01 Thread Michael Paquier
On Thu, Nov 01, 2018 at 07:06:32PM -0400, Stephen Frost wrote: > No, we don't currently track that information but it's an interesting > idea, at least imv. What would be the use case for it? What you are looking for here is gathering information about all pages in a relation and just aggregate w

Re: Running pg_upgrade Version 11

2018-11-05 Thread Michael Paquier
On Tue, Nov 06, 2018 at 04:27:35PM +1100, rob stone wrote: > Logged in as user postgres and postgres owns the files created by > initdb, so is this a permissions problem or am I having a brain fade? Having 0600 as umask for those files is normal. Don't you have more logs about the error? You sho

Re: Reg: Query for DB growth size

2018-11-29 Thread Michael Paquier
On Thu, Nov 29, 2018 at 08:15:10AM +, R.R. PRAVEEN RAJA wrote: > I hope you did not understand my question. I am not asking query for > current database size. I am asking for database growth, i mean > predicting the increase in database size over a period like a month > just like its available

Re: Tables(s) that feed pg_controldata

2018-12-09 Thread Michael Paquier
On Sun, Dec 09, 2018 at 08:10:57PM -0600, Ron wrote: > On 12/09/2018 07:51 PM, Ian Barwick wrote: >> but you're out of luck for 9.2. The only option to query the desired >> values via >> SQL would be to write an extension which reads pg_controldata >> (possibly as a backport of the above-mentioned

Re: pg_stat_replication view

2018-12-10 Thread Michael Paquier
On Mon, Dec 10, 2018 at 02:24:43PM -0500, Jeff Janes wrote: > What does this mean? Is the standby "caught up" when it replays the LSN > which was current on the master as-of the time that the standby initiated > this connection? Or is it caught up when the master receives at least one > notificat

Re: Does idle sessions will consume more cpu and ram? If yes,how to control them

2018-12-19 Thread Michael Paquier
On Thu, Dec 20, 2018 at 11:32:22AM +1100, Joshua White wrote: >> In my application, the idle sessions are consuming cpu and ram. refer the >> ps command output. >> > > If you connect to the database, does select * from pg_stat_activity() show > a lot of idle connections? Each backend stores its o

Re: Question about unlogged to logged conversion

2018-12-27 Thread Michael Paquier
On Thu, Dec 27, 2018 at 05:52:14PM -0500, Ravi Krishna wrote: > Unlogged tables are not stored in the catalog like a regular table. When > it is converted to a logged table , the entire operation is logged (goes > into wal logs) in one single transaction, blocking the entire table > during the proc

Re: Get LSN at which a cluster was promoted on previous timeline

2019-01-04 Thread Michael Paquier
On Fri, Jan 04, 2019 at 08:41:35PM -0600, Jerry Sievers wrote: > Take a look at the *.history file in pg_xlog or pg_wal. These files are also archived. If you want to be able to see such contents at SQL level, you would need some parsing logic like this one for example which is a toy of mine (thi

Re: pg_rewind success even though getting error 'record with incorrect prev-link'

2019-01-29 Thread Michael Paquier
On Tue, Jan 29, 2019 at 07:13:11PM +0600, Abdullah Al Maruf wrote: > When I try to attach an old master with 'archiving set to on` as a new > standby, `pg_rewind` doesn't throw any error, But, when the database > starts, The following error appears: > > ``` > LOG: invalid record length at 0/B

Re: Server goes to Recovery Mode when run a SQL

2019-02-03 Thread Michael Paquier
On Sun, Feb 03, 2019 at 10:05:46AM -0800, Adrian Klaver wrote: > On 2/3/19 8:32 AM, PegoraroF10 wrote: >> I have a complex query which puts my server in recovery mode every time I >> run it. >> I don´t need to say that recovery mode is a situation you don´t want your >> server goes to. Do you mean

Re: Promote replica before being able to accept connections

2019-02-11 Thread Michael Paquier
On Mon, Feb 11, 2019 at 06:59:27AM -0800, Martín Fernández wrote: > That makes sense! > > Thanks for the quick reply Note that PostgreSQL 9.4 has introduced a new parameter in recovery.conf that allows recovery to finish exactly when a consistent state has been reached: recovery_target = 'immedia

Re: Server goes to Recovery Mode when run a SQL

2019-02-11 Thread Michael Paquier
On Sun, Feb 10, 2019 at 03:15:38PM +1100, rob stone wrote: > Down around line 87 onwards there are generate_series without any > parameters, and further dubious usage of EPOCH, as well as DOW. > > Not having the table definitions is obviously clouding the analysis. That seems like the root issue

Re: Channel binding not supported using scram-sha-256 passwords

2019-02-17 Thread Michael Paquier
On Fri, Feb 15, 2019 at 04:18:40PM -0500, Hugh Ranalli wrote: > I did see that. However, I'm not *trying* to use it. I set up accounts with > scram-sha-256 passwords, and when trying to connect I get this message. > Hence why I tried to disable it. tls-server-end-point is implemented as channel bi

Re: WSL (windows subsystem on linux) users will need to turn fsync off as of 11.2

2019-02-17 Thread Michael Paquier
On Sun, Feb 17, 2019 at 10:54:54AM -0800, Andres Freund wrote: > On 2019-02-17 23:29:09 +1300, Thomas Munro wrote: >> Hmm. Well, at least ENOSPC should be treated the same way as EIO. >> Here's an experiment that seems to confirm some speculations about NFS >> on Linux from the earlier threads: >

Re: Promoted slave tries to archive previously archived WAL file

2019-02-17 Thread Michael Paquier
On Fri, Feb 15, 2019 at 12:03:39PM -0800, Andre Piwoni wrote: > My slave nodes were created using pg_basebackup with --wal-method=stream. > If I understand right this option streams WAL files generated during backup > and this WAL file was 00010002 but its contents were > different

Re: Channel binding not supported using scram-sha-256 passwords

2019-02-20 Thread Michael Paquier
On Wed, Feb 20, 2019 at 04:53:32PM +0100, Peter Eisentraut wrote: > I think there is a bug in the frontend code. If the server offers > SCRAM-SHA-256-PLUS, the client will choose it if SSL is in use, but it > will later fail with this error message if not > HAVE_PGTLS_GET_PEER_CERTIFICATE_HASH. G

Re: Channel binding not supported using scram-sha-256 passwords

2019-02-21 Thread Michael Paquier
On Thu, Feb 21, 2019 at 08:32:01PM +0100, Peter Eisentraut wrote: > On 2019-02-21 05:47, Michael Paquier wrote: >> if (conn->ssl_in_use) >> +{ >> +/* >> + * The server

Re: How many billion rows of data I can store in PostgreSQL RDS.

2019-02-21 Thread Michael Paquier
On Thu, Feb 21, 2019 at 09:14:24PM -0800, Adrian Klaver wrote: > This would be a question for AWS RDS support. And this depends also a lot on your schema, your column alignment and the level of bloat of your relations.. -- Michael signature.asc Description: PGP signature

Re: Channel binding not supported using scram-sha-256 passwords

2019-02-26 Thread Michael Paquier
On Tue, Feb 26, 2019 at 10:04:35AM -0500, Tom Lane wrote: > Peter Eisentraut writes: >> I tried reproducing the issue locally, but the required OpenSSL version >> is too old to be easily available. > > Hm, I've got buildfarm hosts with quite old OpenSSLs handy. What > tests do you want done, exa

Re: Channel binding not supported using scram-sha-256 passwords

2019-02-27 Thread Michael Paquier
On Wed, Feb 27, 2019 at 10:21:00AM +0100, Peter Eisentraut wrote: > On 2019-02-26 23:35, Michael Paquier wrote: >> What I do in such cases is to compile OpenSSL by myself and link >> Postgres to it, here is a command to build shared libraries (all that >> is documented in

Re: cannot execute VACUUM during recovery

2019-02-27 Thread Michael Paquier
On Wed, Feb 27, 2019 at 10:39:10AM -0800, Stephen Eilert wrote: > Are you running Vacuum on the slave node? It has to run on the master. VACUUM performs an equivalent write activity so it has to be restricted. ANALYZE can work though. -- Michael signature.asc Description: PGP signature

Re: Future Non-server Windows support???

2019-03-03 Thread Michael Paquier
On Sun, Mar 03, 2019 at 08:30:48AM +0100, Laurenz Albe wrote: > Speaking as a semi-ignorant, I had the impressions that all Windows versions > are pretty similar under the hood (with occasional annoying behavior changes), > and most of the differences are on the GUI level, while the C API is pretty

Re: Notification or action when WAL archives fully restored and streaming replication started

2019-03-13 Thread Michael Paquier
On Thu, Mar 14, 2019 at 02:59:38PM +1100, Michael Cassaniti wrote: > I've got master/slave replication setup between a few hosts. At any > point a slave could become a master. I've got appropriate locking in > place using an external system so that only one master can exist at a > time. I'm having

Re: PostgreSQL-11 Streaming Replication Slave Recovering

2019-03-22 Thread Michael Paquier
On Fri, Mar 22, 2019 at 12:07:27PM +0300, Ali Mumcu wrote: > when on master create new wal_file(like *00010009)* and > then on slave this is updating on slave like "*startup recovering > 00010009* " > > Is that a problem or not i dont know? That's normal. Fr

Re: WAL Archive Cleanup?

2019-03-22 Thread Michael Paquier
On Fri, Mar 22, 2019 at 12:26:33PM -0400, Jeff Janes wrote: > archive_cleanup_command is pretty much obsolete. The modern way to do this > is with streaming replication, using either replication slots or > wal_keep_segments. If the only reason you want an archive is for > replication, then use st

Re: stale WAL files?

2019-03-28 Thread Michael Paquier
On Tue, Mar 26, 2019 at 09:50:37AM -0600, Rob Sargent wrote: > No, sorry I should have said that up front. We’re simple folk. What is the WAL position (LSN) where Postgres is writing to and what is the set of WAL segments in pg_wal (or pg_xlog if that's a server older than 10)? Please double-chec

Re: stale WAL files?

2019-03-29 Thread Michael Paquier
On Thu, Mar 28, 2019 at 09:53:16AM -0600, Rob Sargent wrote: > This is pg10 so it's pg_wal.  ls -ltr > > > -rw---. 1 postgres postgres 16777216 Mar 16 16:33 > 00010CEA00B1 > -rw---. 1 postgres postgres 16777216 Mar 16 16:33 > 00010CEA00B2 > >  ... 217 more on thro

Re: CVE-2019-9193 about COPY FROM/TO PROGRAM

2019-04-01 Thread Michael Paquier
On Mon, Apr 01, 2019 at 10:04:32AM -0400, Jonathan S. Katz wrote: > +1, though I’d want to see if people get noisier about it before we rule > out an official response. > > A blog post from a reputable author who can speak to security should > be good enough and we can make noise through our vario

<    1   2   3   4   >