Re: LDAP authenticated session terminated by signal 11: Segmentation fault, PostgresSQL server terminates other active server processes

2019-02-26 Thread Thomas Munro
On Wed, Feb 27, 2019 at 3:57 AM Tom Lane wrote: > Thomas Munro writes: > > Question > > for the list: other stuff in the server needs libpthread (SSL, LLVM, > > ...), so why are we insisting on using non-MT LDAP? > > The traditional reason for avoiding that is the

Re: Performance comparison between Pgsql 10.5 and Pgsql 11.2

2019-03-02 Thread Thomas Munro
"perf" (or something equivalent on your OS) to figure out where 11 is spending more time in the write test? -- Thomas Munro https://enterprisedb.com

Re: Performance comparison between Pgsql 10.5 and Pgsql 11.2

2019-03-04 Thread Thomas Munro
t it shows aggregated data (up until you ^C it) instead of every syscall? -- Thomas Munro https://enterprisedb.com

Re: Slave server sometimes locks up

2019-03-05 Thread Thomas Munro
I think you could also avoid this problem by loading a library that calls something like srandom(getpid()) in _PG_init() (so it runs in every parallel worker making ID collisions extremely unlikely), but that's not really a serious recommendation since it requires writing C code. -- Thomas Munro https://enterprisedb.com

Re: Performance comparison between Pgsql 10.5 and Pgsql 11.2

2019-03-05 Thread Thomas Munro
for your workload, or at least which systems calls they are, assuming you run the same transactions against both versions. -- Thomas Munro https://enterprisedb.com

Re: LDAP authenticated session terminated by signal 11: Segmentation fault, PostgresSQL server terminates other active server processes

2019-03-06 Thread Thomas Munro
Adding Noah to thread. On Wed, Feb 27, 2019 at 11:28 AM Tom Lane wrote: > Thomas Munro writes: > > I don't see pthread_is_threaded_np() on any non-Apple systems in my > > lab. > > Yeah, I thought that might be a Mac thing. I wonder if POSIX has any > usable equi

Re: Performance comparison between Pgsql 10.5 and Pgsql 11.2

2019-03-06 Thread Thomas Munro
On Wed, Mar 6, 2019 at 11:14 PM Nicola Contu wrote: > Here is the strace as requested for pg11 How does it compare to v10 running the same test? -- Thomas Munro https://enterprisedb.com

Re: Performance comparison between Pgsql 10.5 and Pgsql 11.2

2019-03-06 Thread Thomas Munro
h levels or something like that: we see 7 usec vs 1 usec in those two files (though I have no idea how reliable these times are) and if we're going to call it 21 million times at some point it might matter... -- Thomas Munro https://enterprisedb.com

Re: Performance comparison between Pgsql 10.5 and Pgsql 11.2

2019-03-12 Thread Thomas Munro
wer syscalls, one factor that is unexplained is why that system is calling lseek() more times per query (but you said there may be other work happening on the server, so who knows). -- Thomas Munro https://enterprisedb.com

Re: LDAP authenticated session terminated by signal 11: Segmentation fault, PostgresSQL server terminates other active server processes

2019-03-13 Thread Thomas Munro
a minor release; I don't see any problem with it since > > I expect both variants to be provided by the same package in every > > distro but we'd certainly want to highlight this to the package > > maintainers if we did it. > > It's not great to change

Re: ERROR: XX000: cannot update SecondarySnapshot during a parallel operation

2019-03-14 Thread Thomas Munro
PostgreSQL with that error changed to a PANIC, and examine the resulting smoldering core. (Someone had a proposal to make PostgreSQL errors optionally dump the function call stack with backtrace(3) even in regular production builds, which would make this kind of investigations go f

Re: LDAP authenticated session terminated by signal 11: Segmentation fault, PostgresSQL server terminates other active server processes

2019-03-19 Thread Thomas Munro
rwise this problem could just keep coming back until libldap.so is eventually entirely phased out by all distros. In 2023 I want to be working on quantum parallelism or something, not LDAP bug reports. Any objections? -- Thomas Munro https://enterprisedb.com

Re: FreeBSD 12 and Postgres build failures

2019-03-19 Thread Thomas Munro
e you need to re-run configure? Be sure to blow > away any autoconf cache file if you're using one. Builds OK for me on FreeBSD 12.0-RELEASE (using OpenSSL 1.1.1a-freebsd from base system), tested with PostgreSQL REL_11_STABLE. +1 for the autoconf cache theory. -- Thomas Munro https://enterprisedb.com

Re: FreeBSD 12 and Postgres build failures

2019-03-19 Thread Thomas Munro
re was getting > confused (in theory this shouldn't get you, but theory and practice sometimes > don't play nice. :-) Hmm. I guess it'd be good not to get confused between ports and base OpenSSL installations... is there something we're doing wrong that makes that happen? -- Thomas Munro https://enterprisedb.com

Re: FreeBSD 12 and Postgres build failures

2019-03-19 Thread Thomas Munro
d age. The silliest case is macOS, where gcc and g++ are found and used but are in fact... wrappers for clang, for maximum confusion. -- Thomas Munro https://enterprisedb.com

LDAP on AIX build farm animals

2019-03-20 Thread Thomas Munro
binaries it's running so that we can see which libraries it finished up linking? -- Thomas Munro https://enterprisedb.com

Re: LDAP authenticated session terminated by signal 11: Segmentation fault, PostgresSQL server terminates other active server processes

2019-03-20 Thread Thomas Munro
On Wed, Mar 20, 2019 at 10:51 AM Tom Lane wrote: > Thomas Munro writes: > > Even though I can't reproduce the problem myself, I'm quite keen to go > > ahead and push the patch I proposed for v12 anyway, and close this > > case. Otherwise this problem co

Re: LDAP authenticated session terminated by signal 11: Segmentation fault, PostgresSQL server terminates other active server processes

2019-03-21 Thread Thomas Munro
On Thu, Mar 21, 2019 at 5:07 PM Tom Lane wrote: > Thomas Munro writes: > > If someone out there is not enabling any of that stuff > > because their system doesn't like threads, they can use > > --disable-thread-safety to avoid the effects of this change. > > No, t

Re: New LLVM JIT Features

2019-04-02 Thread Thomas Munro
proposal section. Hi Praveen, FYI the final "commitfest" for PostgreSQL 12 is wrapping up right now and the code freeze begins in a few days, so I wouldn't expect an immediate reply. -- Thomas Munro https://enterprisedb.com

Re: 9.6.11- could not truncate directory "pg_serial": apparent wraparound

2019-04-08 Thread Thomas Munro
7;s just the index used to find entries -- the actual amount of data stored wouldn't change, you'd just start seeing wider filenames, and all the fragile modulo comparison truncation stuff would disappear from the tree. -- Thomas Munro https://enterprisedb.com

Re: 9.6.11- could not truncate directory "pg_serial": apparent wraparound

2019-04-08 Thread Thomas Munro
On Tue, Apr 9, 2019 at 12:14 PM Thomas Munro wrote: > It's more doable here than elsewhere because the data on disk isn't > persistent across server restart, let alone pg_upgrade. Let's see... > each segment file is 256kb and we need to be able to address 2^64 * > siz

Re: os upgrade 7.3 to 7.5 (postgres version 10.5)

2019-04-10 Thread Thomas Munro
glibc 2.28 corrupts indexes even for English language collations (because the sort order of ' ', '-' and some other symbols moved around), and other versions have affected individual particular languages (I forget which one corrupted German language indexes a few years back). It&#

Re: Upgrading locale issues

2019-05-01 Thread Thomas Munro
1] https://www.postgresql.org/message-id/flat/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com [2] https://reviews.freebsd.org/D17166 [3] https://www.postgresql.org/message-id/flat/3366.1498183854%40sss.pgh.pa.us -- Thomas Munro https://enterprisedb.com

Re: Centos : Load Average :OS Process Name : migration/1, migration/2 , migration/n

2019-05-22 Thread Thomas Munro
erfault.com/questions/674685/kernel-processes-periodically-eating-cpu-during-high-load https://bugs.gentoo.org/394487 https://bugzilla.kernel.org/show_bug.cgi?id=47341 -- Thomas Munro https://enterprisedb.com

Re: Cause: org.postgresql.util.PSQLException: ERROR: could not resize shared memory segment "/PostgreSQL.1946998112" to 8388608 bytes: No space left on device

2019-06-03 Thread Thomas Munro
ect to run * number of tables you expect them to join * number of parallel workers you expect to run. The amount of it that happens to be in /dev/shm on a Linux system (rather than private memory) is controlled by what fraction of your joins are parallel hash joins. Making our memory limits better is really hard. -- Thomas Munro https://enterprisedb.com

Re: LDAP Search failing when i provide Multiple entry in pg_hba.conf

2019-06-23 Thread Thomas Munro
ds lots of discussion of how to build a search filter that searches more than one OU. You can experiment with the ldapsearch command-line tool. [1] https://www.postgresql.org/docs/11/auth-pg-hba-conf.html [2] https://www.postgresql.org/docs/11/auth-ldap.html -- Thomas Munro https://enterprisedb.com

Re: "PANIC: could not open critical system index 2662" - twice

2023-05-04 Thread Thomas Munro
On Fri, May 5, 2023 at 6:11 AM Evgeny Morozov wrote: > Meanwhile, what do I do with the existing server, though? Just try to > drop the problematic DBs again manually? That earlier link to a FreeBSD thread is surely about bleeding edge new ZFS stuff that was briefly broken then fixed, being disco

Re: "PANIC: could not open critical system index 2662" - twice

2023-05-04 Thread Thomas Munro
On Fri, May 5, 2023 at 11:15 AM Thomas Munro wrote: > Now *that* is a piece of > logic that changed in PostgreSQL 15. It changed from sector-based > atomicity assumptions to a directory entry swizzling trick, in commit > d8cd0c6c95c0120168df93aae095df4e0682a08a. Hmm. I spoke to

Re: "PANIC: could not open critical system index 2662" - twice

2023-05-04 Thread Thomas Munro
On Fri, May 5, 2023 at 11:15 AM Thomas Munro wrote: > What does select > pg_relation_filepath('pg_class_oid_index') show in the corrupted > database, base/5/2662 or something else? Oh, you can't get that far, but perhaps you could share the pg_filenode.map file? Or

Re: "PANIC: could not open critical system index 2662" - twice

2023-05-05 Thread Thomas Munro
On Fri, May 5, 2023 at 7:50 PM Evgeny Morozov wrote: > The OID of the bad DB ('test_behavior_638186279733138190') is 1414389 and > I've uploaded base/1414389/pg_filenode.map and also base/5/2662 (in case > that's helpful) as https://objective.realityexists.net/temp/pgstuff1.zip Thanks. That pg

Re: "PANIC: could not open critical system index 2662" - twice

2023-05-06 Thread Thomas Munro
On Sat, May 6, 2023 at 9:58 PM Evgeny Morozov wrote: > Right - I should have realised that! base/1414389/2662 is indeed all > nulls, 32KB of them. I included the file anyway in > https://objective.realityexists.net/temp/pgstuff2.zip OK so it's not just page 0, you have 32KB or 4 pages of all zero

Re: "PANIC: could not open critical system index 2662" - twice

2023-05-06 Thread Thomas Munro
On Sun, May 7, 2023 at 12:29 AM Evgeny Morozov wrote: > On 6/05/2023 12:34 pm, Thomas Munro wrote: > > So it does indeed look like something unknown has replaced 32KB of > > data with 32KB of zeroes underneath us. Are there more non-empty > > files that are all-zeroes? Some

Re: "PANIC: could not open critical system index 2662" - twice

2023-05-06 Thread Thomas Munro
On Sun, May 7, 2023 at 10:23 AM Jeffrey Walton wrote: > This may be related... I seem to recall the GNUlib folks talking about > a cp bug on sparse files. It looks like it may be fixed in coreutils > release 9.2 (2023-03-20): > https://github.com/coreutils/coreutils/blob/master/NEWS#L233 > > If I

Re: "PANIC: could not open critical system index 2662" - twice

2023-05-07 Thread Thomas Munro
On Sun, May 7, 2023 at 1:21 PM Tom Lane wrote: > Thomas Munro writes: > > Did you previously run this same workload on versions < 15 and never > > see any problem? 15 gained a new feature CREATE DATABASE ... > > STRATEGY=WAL_LOG, which is also the default. I wond

Re: "PANIC: could not open critical system index 2662" - twice

2023-05-07 Thread Thomas Munro
On Mon, May 8, 2023 at 4:10 AM Evgeny Morozov wrote: > On 6/05/2023 11:13 pm, Thomas Munro wrote: > > Would you like to try requesting FILE_COPY for a while and see if it > > eventually happens like that too? > Sure, we can try that. Maybe you could do some one way and some

Re: "PANIC: could not open critical system index 2662" - twice

2023-05-07 Thread Thomas Munro
On Mon, May 8, 2023 at 2:24 PM Michael Paquier wrote: > I can reproduce the same backtrace here. That's just my usual laptop > with ext4, so this would be a Postgres bug. First, here are the four > things running in parallel so as I can get a failure in loading a > critical index when connecting

Re: "PANIC: could not open critical system index 2662" - twice

2023-05-08 Thread Thomas Munro
On Tue, May 9, 2023 at 10:04 AM Tom Lane wrote: > Michael Paquier writes: > > One thing I was wondering about to improve the odds of the hits is to > > be more aggressive with the number of relations created at once, so as > > we are much more aggressive with the number of pages extended in > > p

Re: fsync data directory after DB crash

2023-07-18 Thread Thomas Munro
On Wed, Jul 19, 2023 at 12:41 PM Michael Paquier wrote: > On Tue, Jul 18, 2023 at 04:50:25PM +0800, Pandora wrote: > > I found that starting from version 9.5, PostgreSQL will do fsync on > > the entire data directory after DB crash. Here's a question: if I > > have FPW = on, why is this step still

Re: fsync data directory after DB crash

2023-07-20 Thread Thomas Munro
On Wed, Jul 19, 2023 at 2:09 PM Pandora wrote: > Yes, I saw the usage of syncfs in PG14, but it is recommended to use it on > Linux 5.8 or higher. If my OS version is lower than 5.8, can I still enable > it? Nothing stops you from enabling it, it's fairly ancient and should work. It just doesn

Re: psql crash with custom build on RedHat 7

2023-12-19 Thread Thomas Munro
On Wed, Dec 20, 2023 at 1:39 AM Dominique Devienne wrote: > Program received signal SIGSEGV, Segmentation fault. > 0x004232b8 in slash_yylex () I think this might have to do with flex changing. Does it help if you "make maintainer-clean"?

Re: psql crash with custom build on RedHat 7

2023-12-19 Thread Thomas Munro
On Wed, Dec 20, 2023 at 4:41 AM Dominique Devienne wrote: > On Tue, Dec 19, 2023 at 2:02 PM Thomas Munro wrote: >> On Wed, Dec 20, 2023 at 1:39 AM Dominique Devienne >> wrote: >> > Program received signal SIGSEGV, Segmentation fault. >> > 0x004232b8 in

Re: How to generate random bigint

2023-12-20 Thread Thomas Munro
On Thu, Dec 21, 2023 at 7:21 PM Tom Lane wrote: > Phillip Diffley writes: > > Postgres's random() function generates a random double. That can be > > converted to a random int for smaller integers, but a double can't > > represent all of the values in a bigint. Is there a recommended way to > > g

Re: Query crash with 15.5 on debian bookworm/armv8

2023-12-26 Thread Thomas Munro
On Wed, Dec 27, 2023 at 5:17 AM Clemens Eisserer wrote: > > FWIW, since this crash is inside LLVM you could presumably dodge the bug > > by setting "jit" to off. > > Thanks, this indeed solved the crash. > Just to make sure this crash doesn't have anything to do with my > setup/config (I'd changed

Re: How do you optimize the disk IO when you cannot assume a file will start at a boundary then?

2024-02-22 Thread Thomas Munro
working on bringing to PostgreSQL in useful form, we had to do this: commit faeedbcefd40bfdf314e048c425b6d9208896d90 Author: Thomas Munro Date: Sat Apr 8 10:38:09 2023 +1200 Introduce PG_IO_ALIGN_SIZE and align all I/O buffers. ... to avoid EINVAL errors, falling back to buffered mode or pa

Re: could not open file "global/pg_filenode.map": Operation not permitted

2024-03-28 Thread Thomas Munro
On Fri, Mar 29, 2024 at 4:47 AM Nick Renders wrote: > Looking at the 2 machines that are having this issue (and the others that > don't), I think it is somehow related to the following setup: > - macOS Sonoma (14.4 and 14.4.1) > - data directory on an external drive > > That external drive (a Pro

Re: could not open file "global/pg_filenode.map": Operation not permitted

2024-04-03 Thread Thomas Munro
On Thu, Apr 4, 2024 at 3:11 AM Nick Renders wrote: > In the macOS Finder, when you show the Info (command+i) for an external drive > (or any partition that is not the boot drive), there is a checkbox "Ignore > ownership on this volume" in the Permissions section. I think it is by > default "on"

Re: could not open file "global/pg_filenode.map": Operation not permitted

2024-04-03 Thread Thomas Munro
On Sat, Mar 23, 2024 at 3:01 AM Nick Renders wrote: > We now have a second machine with this issue: it is an Intel Mac mini running > macOS Sonoma (14.4) and PostgreSQL 16.2. > This one only has a single Data directory, so there are no multiple instances > running. BTW if you're running databas

Re: Preallocation changes in Postgresql 16

2024-04-25 Thread Thomas Munro
On Fri, Apr 26, 2024 at 4:37 AM Riku Iki wrote: > I am wondering if there were preallocation related changes in PG16, and if it > is possible to disable preallocation in PostgreSQL 16? I have no opinion on the btrfs details, but I was wondering if someone might show up with a system that doesn't

Re: libpq v17 PQsocketPoll timeout is not granular enough

2024-06-10 Thread Thomas Munro
On Tue, Jun 11, 2024 at 2:36 AM Dominique Devienne wrote: > Hi. I've noticed [that libpq API in v17 beta1][1], and wanted to use > it to replace an existing Boost.ASIO-based async polling of the > connection's socket, waiting for notifications. The use case being > using PostgreSQL LISTEN/NOTIFY f

Re: Windows installation problem at post-install step

2024-07-21 Thread Thomas Munro
On Mon, Jul 22, 2024 at 7:29 AM Adrian Klaver wrote: > On 7/21/24 12:00, Ertan Küçükoglu wrote: > > My main purpose was and still is to reach EDB people using the forum and > > let them know about the problem. > > I believe it is something to be fixed for future installations. I would > > like to

Re: Windows installation problem at post-install step

2024-07-21 Thread Thomas Munro
On Mon, Jul 22, 2024 at 11:58 AM Ertan Küçükoglu wrote: > Thomas Munro , 21 Tem 2024 Paz, 23:27 tarihinde şunu > yazdı: >> 2. Some existing database clusters which had been installed with the >> name "Turkish_Turkey.1254" became unstartable when the OS upgra

Re: Windows installation problem at post-install step

2024-07-22 Thread Thomas Munro
On Mon, Jul 22, 2024 at 11:51 PM Sandeep Thakkar wrote: > EDB's windows installer gets the locales on the system using the > https://github.com/EnterpriseDB/edb-installers/blob/REL-16/server/scripts/windows/getlocales/getlocales.cpp > and then substitute some patterns > (https://github.com/Ente

Re: Windows installation problem at post-install step

2024-08-05 Thread Thomas Munro
less confusing than looking at script output via email (I don't even know how many onion layers of transcoding are involved...) From b97fe5a55e50a447d41a439412922ffe3f7e168b Mon Sep 17 00:00:00 2001 From: Thomas Munro Date: Tue, 6 Aug 2024 16:06:29 +1200 Subject: [PATCH 1/3] xxx debug --- src/

Re: Windows installation problem at post-install step

2024-08-06 Thread Thomas Munro
On Tue, Aug 6, 2024 at 10:38 PM Sandeep Thakkar wrote: > On Tue, Aug 6, 2024 at 4:06 PM Sandeep Thakkar > wrote: [v15] >>> XXX debug raw: setup_locale_encoding = "Turkish_Türkiye.1254" >>> XXX debug hex: setup_locale_encoding = { 54 75 72 6b 69 73 68 5f 54 fc 72 >>> 6b 69 79 65 2e 31 32 35

Re: Windows installation problem at post-install step

2024-08-06 Thread Thomas Munro
On Tue, Aug 6, 2024 at 11:44 PM Peter J. Holzer wrote: > I assume that "1254" here is the code page. > But you specified --encoding=UTF-8 above, so your default locale uses a > different encoding than the template databases. I would expect that to > cause problems if the template databases contain

Re: Windows installation problem at post-install step

2024-08-07 Thread Thomas Munro
Thanks. The log didn't offer any more clues, and my colleague David R has Windows and knows how to work its debugger so we sat down together and chased this down (thanks David!). 1. It is indeed calling abort(), but it's not a PANIC or Assert() in PostgreSQL, it's an assertion inside Windows' ow

Re: Error:could not extend file " with FileFallocate(): No space left on device

2024-09-11 Thread Thomas Munro
On Wed, Sep 11, 2024 at 9:56 PM Alvaro Herrera wrote: > On 2024-Sep-10, Pecsök Ján wrote: > > After upgrade of Posgres from version 13.5 to 16.2 we experience following > > error: > > could not extend file "pg_tblspc/16401/PG_16_202307071/17820/3968302971" > > with FileFallocate(): No space left

Re: Error:could not extend file " with FileFallocate(): No space left on device

2024-09-11 Thread Thomas Munro
On Thu, Sep 12, 2024 at 12:39 AM Alvaro Herrera wrote: >> On 2024-Sep-11, Pecsök Ján wrote: > > In our case: > > Kernel: Linux version 4.18.0-513.18.1.el8_9.ppc64le > > (mockbu...@ppc-hv-13.build.eng.rdu2.redhat.com) (gcc version 8.5.0 20210514 > > (Red Hat 8.5.0-20) (GCC)) #1 SMP Thu Feb 1 02:5

Re: Error:could not extend file " with FileFallocate(): No space left on device

2024-09-11 Thread Thomas Munro
I don't understand what ENOSPC has to do with the file descriptor limits, but this person reported: # touch test touch: cannot touch ‘test’: No space left on device https://serverfault.com/questions/746032/rsync-and-scp-failing-with-no-space-left-on-xfs-device ... with plenty of free space, and

Re: Error:could not extend file " with FileFallocate(): No space left on device

2024-09-12 Thread Thomas Munro
On Thu, Sep 12, 2024 at 8:54 PM Pecsök Ján wrote: > In link you provided there is mention, that in PostgreSQL 16 data is not being > compressed for PostgreSQL 16 server. Does it mean, that PosgreSQL 16 use much > more space while computing queries? > If that is the case, it can be our problem, be

Re: Log files polluted with permission denied error messages after every 10 seconds

2021-03-08 Thread Thomas Munro
On Sat, Mar 6, 2021 at 2:36 PM Michael Paquier wrote: > On Fri, Mar 05, 2021 at 07:36:37PM +0200, Andrus wrote: > > Then turned real-time protection off: > > > > Problem persists. New entry is written after every 10 seconds. > > On which files are those complaints? It seems to me that you may hav

Re: SV: Log files polluted with permission denied error messages after every 10 seconds

2021-03-09 Thread Thomas Munro
On Tue, Mar 9, 2021 at 9:43 PM Andrus wrote: > > Any hints in Windows event viewer? Events occurring at the same time > showing up there. > > Looked into Administrative Events/Custom views and few others. There are no > messages about this. Windowsi perfomance monitor and Filemon show files >

Re: SV: Log files polluted with permission denied error messages after every 10 seconds

2021-03-11 Thread Thomas Munro
Hi Andrus, On Thu, Mar 11, 2021 at 2:21 AM Andrus wrote: > Windows Resource manger shows that wal files are used by large number of > postgres processes: > > postgres.exe22656FileC:\Program > Files\PostgreSQL\13\data\pg_wal\0001000A0075 > postgres.exe30788File

Re: Postgres crashes at memcopy() after upgrade to PG 13.

2021-03-15 Thread Thomas Munro
On Mon, Mar 15, 2021 at 1:29 PM Avinash Kumar wrote: > Is this expected when replication is happening between PostgreSQL databases > hosted on different OS versions like Ubuntu 16 and Ubuntu 20 ? Or, do we > think this is some sort of corruption ? Is this index on a text datatype, and using a c

Re: -1/0 virtualtransaction

2021-04-30 Thread Thomas Munro
On Wed, Apr 28, 2021 at 2:25 AM Mike Beachy wrote: > On Tue, Apr 27, 2021 at 2:56 AM Laurenz Albe wrote: >> >> Not sure, but do you see prepared transactions in "pg_prepared_xacts"? > > No, the -1 in the virtualtransaction > (https://www.postgresql.org/docs/11/view-pg-locks.html) for pg_prepared

Re: Huge performance penalty with parallel queries in Windows x64 v. Linux x64

2021-05-03 Thread Thomas Munro
On Tue, May 4, 2021 at 4:05 AM Hans Buschmann wrote: > The main difference is the time shown for the Gather Merge step (65 ms vs. 7 > ms) No Windows here, but could it be super slow at launching workers? How does a trivial parallel query compare, something like? SET force_parallel_mode = on; E

Re: Huge performance penalty with parallel queries in Windows x64 v. Linux x64

2021-05-04 Thread Thomas Munro
On Tue, May 4, 2021 at 7:40 PM Hans Buschmann wrote: > The problem seems that this (probably inherent) performance disadvantage of > windows is not reflected in the cost model. https://www.postgresql.org/docs/13/runtime-config-query.html#GUC-PARALLEL-SETUP-COST is for that. It might be interest

Re: Huge performance penalty with parallel queries in Windows x64 v. Linux x64

2021-05-04 Thread Thomas Munro
On Wed, May 5, 2021 at 3:50 AM Hans Buschmann wrote: > (BTW: Is this cost multiplied by the real count of workers choosen > (max_parallel_workers_per_gather) or only a value independent of the number > of workers?. This would matter in windows-high-parallel scenarios) It's not multiplied: http

Re: Huge performance penalty with parallel queries in Windows x64 v. Linux x64

2021-05-04 Thread Thomas Munro
On Wed, May 5, 2021 at 2:12 PM Thomas Munro wrote: > It might be interesting to know how that 40ms time scales as you add > more workers. ... Another thought: I'd also try tests like that in large databases (ie large virtual memory) vs small ones, and with and without huge/locked m

Re: Chain Hashing

2021-05-06 Thread Thomas Munro
On Thu, May 6, 2021 at 9:48 PM Jian He wrote: > The following part is about the Chain Hashing. >> >> Maintain a linked list of buckets for each slot in the hash table. >> Resolve collisions by placing all elements with the same hash key into the >> same bucket. >> → To determine whether an elem

Re: CREATE/REFRESH MATERIALIZED VIEW planner difference?

2021-06-01 Thread Thomas Munro
On Wed, Jun 2, 2021 at 7:15 AM Vijaykumar Jain wrote: > i only get workers to create mv, but refresh mv plan does not use workers for > the same conf params. Yeah, this changed in v14: https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=9e7ccd9ef64d05e87ceb1985d459bef9031205c0

Re: order by

2021-06-10 Thread Thomas Munro
On Thu, Jun 10, 2021 at 8:40 PM Luca Ferrari wrote: > On Thu, Jun 10, 2021 at 10:15 AM Vijaykumar Jain > > when you provide a table in query in the order by clause, it is > > ordered by cols of that table in that order. > > Clever, thanks! > I also realized that this "table to tuples" expansion wo

Re: Is there something similar like flashback query from Oracle planned for PostgreSQL

2021-06-23 Thread Thomas Munro
On Thu, Jun 24, 2021 at 6:54 AM Dirk Krautschick wrote: > Is there something planned to get a behaviour like Oracle's flashback query > based on the old values > before deleted by vacuum? > > So a feature to recreate old versions of rows if still there? > > Or are there any related extensions or

Re: spiexceptions.UndefinedFile: could not open file "base/16482/681830": No such file or directory

2021-09-08 Thread Thomas Munro
On Thu, Sep 9, 2021 at 9:19 AM Celia McInnis wrote: > Note that the file does exist:! (How do I know if it is looking under the > correct directory? Other times I have done similar temporary table creations > with no problems!): PostgreSQL internally uses relative paths. It's probably not a ve

Re: To JIT (or not!) in Ubuntu packages

2021-09-14 Thread Thomas Munro
On Tue, Sep 14, 2021 at 10:11 AM Ben Chobot wrote: > We've noticed that the Ubuntu postgresql-12 package has --with-llvm > enabled on x86_64, but not on aarch64. Does anybody know if this was > intentional, or just an oversight? > > For what it's worth, it seems the beta postgresql-14 package for

Re: To JIT (or not!) in Ubuntu packages

2021-09-14 Thread Thomas Munro
On Wed, Sep 15, 2021 at 3:30 PM Ben Chobot wrote: > So I've installed > http://apt.postgresql.org/pub/repos/apt/pool/14/p/postgresql-14/postgresql-14_14~beta3-1.pgdg18.04+1_arm64.deb, > after which I see: Ahhh, so you're on 18.04, an old LTS. I remember now, there was this issue in LLVM 3.9 on

Re: Wrong sorting on docker image

2021-10-16 Thread Thomas Munro
On Sun, Oct 17, 2021 at 4:42 AM Tom Lane wrote: > Speaking of ICU, if you are using an ICU-enabled Postgres build, > maybe you could find an ICU collation that acts the way you want. > This wouldn't be a perfect solution, because we don't yet have > the ability to set an ICU collation as a databas

Re: Need to know more about pg_test_fsync utility

2021-12-09 Thread Thomas Munro
On Fri, Dec 10, 2021 at 3:20 PM PGSQL DBA wrote: > 1) How to interpret the output of pg_test_fsync? The main interesting area is probably the top section that compares the different wal_sync_method settings. For example, it's useful to verify the claim that fdatasync() is faster than fsync() (be

Re: Need to know more about pg_test_fsync utility

2021-12-12 Thread Thomas Munro
On Mon, Dec 13, 2021 at 3:04 PM PGSQL DBA wrote: > As you mentioned in question-8, "I'd investigate whether data is being cached > unexpectedly, perhaps indicating that committed transactions be lost in a > system crash event." So, I would like to know that if we configure the disk > for the WA

Re: create database hangs forever on WSL - autovacuum deadlock?

2022-01-08 Thread Thomas Munro
On Sat, Jan 8, 2022 at 8:11 PM Alicja Kucharczyk wrote: > this is postgres running on windows wsl: PostgreSQL 14.1 (Ubuntu > 14.1-1.pgdg18.04+1) on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu > 7.5.0-3ubuntu1~18.04) 7.5.0, 64-bit Is this WSL1 (some kind of Linux system call emulator running o

Re: create database hangs forever on WSL - autovacuum deadlock?

2022-01-09 Thread Thomas Munro
On Sun, Jan 9, 2022 at 2:15 PM Alicja Kucharczyk wrote: > sob., 8 sty 2022 o 22:40 Thomas Munro napisał(a): >> On Sat, Jan 8, 2022 at 8:11 PM Alicja Kucharczyk >> wrote: >> > this is postgres running on windows wsl: PostgreSQL 14.1 (Ubuntu >> > 14.1-1.pg

Re: could not accept SSL connection: Success

2022-01-19 Thread Thomas Munro
On Thu, Jan 20, 2022 at 12:06 AM Carla Iriberri wrote: > On Wed, Jan 19, 2022 at 5:42 AM Michael Paquier wrote: >> "On an unexpected EOF, versions before OpenSSL 3.0 returned >> SSL_ERROR_SYSCALL, nothing was added to the error stack, and errno was >> 0. Since OpenSSL 3.0 the returned error is SS

Re: sort order for UTF-8 char column with Japanese UTF-8

2022-02-03 Thread Thomas Munro
On Fri, Feb 4, 2022 at 8:11 AM Matthias Apitz wrote: > On my FreeBSD laptop the same file sorts as > > guru@c720-r368166:~ $ LANG=de_DE.UTF-8 sort swd > A > ゲアハルト・A・リッター > ゲルハルト・A・リッター > チャールズ・A・ビアード > A010STRUKTUR > A010STRUKTUR > A010STRUKTUR > A0150SUPRALEITER Wow, so it's one thing to have a

Re: Compile 14.1 in EL5.8

2022-02-09 Thread Thomas Munro
On Thu, Feb 10, 2022 at 3:38 AM Tom Lane wrote: > Gabriela Serventi writes: > > Hi! I'm trying to compile release 14.1 in a very old Linux Server (Red Hat > > Enterprise Linux Server release 5.8). I can run configure successfully, but > > when I try to run make, I get the following error: > > lat

Re: Compile 14.1 in EL5.8

2022-02-09 Thread Thomas Munro
On Thu, Feb 10, 2022 at 2:23 PM Tom Lane wrote: > Thomas Munro writes: > > ... I wondered about also removing the leftover comment > > "We assume that any system that has Linux epoll() also has Linux > > signalfd()" which was my attempt to explain that there wasn&#

Re: frequency of positive and negative numbers, sizes of numbers and frequency of alteration of polarity

2022-02-17 Thread Thomas Munro
On Fri, Feb 18, 2022 at 9:11 AM Shaozhong SHI wrote: > How to calculate frequency of positive and negative numbers and define and > calculate frequency of alteration of polarity? > > Surely, we can use frequency of alteration of polarity and level of change > (e.g., size of positive and negative

Re: frequency of positive and negative numbers, sizes of numbers and frequency of alteration of polarity

2022-02-17 Thread Thomas Munro
On Fri, Feb 18, 2022 at 10:42 AM Shaozhong SHI wrote: > Given 2 or more such columns, is there any measure that can be calculated to > tell which one alternates more than others? Well, you could report non-flips as NULL and flips as magnitude, and then wrap that query in another query to compute

Re: [GENERAL] Query Using Massive Temp Space

2017-11-21 Thread Thomas Munro
x27;t have a specific suggestion, but it could just be something like 'you threw out or kept more than X% of the tuples'. -- Thomas Munro http://www.enterprisedb.com

Re: ERROR: too many dynamic shared memory segments

2017-11-27 Thread Thomas Munro
lel queries running at the same time. Can you provide reproduction steps? Does your query plan happen to include a Parallel Bitmap Heap Scan? -- Thomas Munro http://www.enterprisedb.com

Re: ERROR: too many dynamic shared memory segments

2017-11-27 Thread Thomas Munro
.4.36 rows=10 width=0) > Index Cond: > ((datasource)::text = 'three'::text) > > > > > In this particular query there were over _100_ partitions connected with the > UNION ALL operator. Ah, so you have many Gather nodes under Append? That's one way to eat arbitrarily many DSM slots. We allow for 64 + 2 * max_backends. Does it help if you increase max_connections? I am concerned about the crash failure mode you mentioned in the first email though: we should always be able to handle that condition gracefully. -- Thomas Munro http://www.enterprisedb.com

Re: ERROR: too many dynamic shared memory segments

2017-11-27 Thread Thomas Munro
, ran out of slots and said so and our error machinery worked as it should), and another crashed with a segfault, because it tried to use a NULL "area" pointer (bad). I think this is a degenerate case where we completely failed to launch parallel query, but we ran the parallel query plan anyway and this code thinks that the DSA is available. Oops. -- Thomas Munro http://www.enterprisedb.com

Re: transaction wrap around

2017-12-05 Thread Thomas Munro
expect to encounter 20TB production databases in the wild that have gone into a wraparound frenzy confounding their owners. -- Thomas Munro http://www.enterprisedb.com

Re: clean out ./data/base/pgsql_tmp

2017-12-06 Thread Thomas Munro
oesn't clear away pgsql_tmp data on the theory that it might be useful for forensics. -- Thomas Munro http://www.enterprisedb.com

Re: ERROR: too many dynamic shared memory segments

2017-12-07 Thread Thomas Munro
discussed, fixed in the next point release, but even with that fix in place you'll still need a high enough max_connections setting to be sure to be able to complete the query without an error. Thanks for the report! -- Thomas Munro http://www.enterprisedb.com

Re: Size of pg_multixact/members increases 11355

2017-12-12 Thread Thomas Munro
members size will be maintained to estimate the peak member space usage. 9.3 is also prone to eating more multixact space than recent major versions in cases involving subtransactions. -- Thomas Munro http://www.enterprisedb.com

Re: transaction wrap around

2017-12-12 Thread Thomas Munro
On Mon, Dec 11, 2017 at 12:07 PM, Jeff Janes wrote: > On Tue, Dec 5, 2017 at 5:50 PM, Thomas Munro > wrote: >> The problem is that our logic (1) focuses on when we should *start* >> freezing, not by when we'd like to be finished, and (2) is defined in >> such a way t

Re: Query error: could not resize shared memory segment

2018-01-02 Thread Thomas Munro
obably a parallel query using parallel bitmap heapscan and seeing the error coming from the change in commit 899bd785c0edf376077d3f5d65c316f92c1b64b5, meaning that it would have risked death by SIGBUS before that commit. What is surprising is that increasing work_mem helped. -- Thomas Munro http://www.enterprisedb.com

Re: Query error: could not resize shared memory segment

2018-01-02 Thread Thomas Munro
ually we can't tell if it's the ftruncate() or posix_fallocate() call that failed, but the latter seems more likely since the former just creates a big hole in the underlying tmpfs file). Can you share the query plan (EXPLAIN SELECT ...)? -- Thomas Munro http://www.enterprisedb.com

Re: Query error: could not resize shared memory segment

2018-01-02 Thread Thomas Munro
ee the failing plan without trying to run it? I'm guessing it's different, because the plan you showed doesn't look like it would want 50MB of DSM. -- Thomas Munro http://www.enterprisedb.com

Re: Query error: could not resize shared memory segment

2018-01-02 Thread Thomas Munro
On Wed, Jan 3, 2018 at 5:39 PM, Tom Lane wrote: > Thomas Munro writes: >> So you have 16GB of RAM and here we're failing to posix_fallocate() >> 50MB (actually we can't tell if it's the ftruncate() or >> posix_fallocate() call that failed, but the latter s

<    1   2   3   >