Re: backend crash on DELETE, reproducible locally

2018-11-07 Thread Ondřej Bouda
Dne 3.11.2018 v 20:38 Tom Lane napsal(a): Yeah, dump/reload would make the problem go away. Same bug. So we dumped and restored all our databases. After that, the crash on DELETE never occurred (before, it was several times a day). However, the crash on UPDATE still occurs on specific rows.

Re: backend crash on DELETE, reproducible locally

2018-11-06 Thread Andres Freund
Hi, On 2018-11-06 17:11:40 -0500, Tom Lane wrote: > Andres Freund writes: > > On 2018-11-06 16:47:20 -0500, Tom Lane wrote: > >> Looks like somebody forgot to list > >> RELOPT_KIND_GIST in RELOPT_KIND_INDEX, but is that the > >> fault of commit c203d6cf8 or was it busted before? > > > Looks new:

Re: backend crash on DELETE, reproducible locally

2018-11-06 Thread Andres Freund
Hi, On 2018-11-06 23:11:29 +0100, Tomas Vondra wrote: > On 11/6/18 10:54 PM, Andres Freund wrote: > > Looks new: > > + RELOPT_KIND_INDEX = > > RELOPT_KIND_BTREE|RELOPT_KIND_HASH|RELOPT_KIND_GIN|RELOPT_KIND_SPGIST, > > > > there aren't any other "for all indexes" type options, so the whole > >

Re: backend crash on DELETE, reproducible locally

2018-11-06 Thread Tom Lane
Andres Freund writes: > On 2018-11-06 16:47:20 -0500, Tom Lane wrote: >> Looks like somebody forgot to list >> RELOPT_KIND_GIST in RELOPT_KIND_INDEX, but is that the >> fault of commit c203d6cf8 or was it busted before? > Looks new: > + RELOPT_KIND_INDEX = > RELOPT_KIND_BTREE|RELOPT_KIND_HASH|

Re: backend crash on DELETE, reproducible locally

2018-11-06 Thread Tomas Vondra
On 11/6/18 10:54 PM, Andres Freund wrote: > On 2018-11-06 16:47:20 -0500, Tom Lane wrote: >> =?UTF-8?Q?Ond=c5=99ej_Bouda?= writes: Ondřej, as a short-term workaround you could prevent the crash by setting that index's recheck_on_update property to false. >> >>> Thanks for the tip. I am u

Re: backend crash on DELETE, reproducible locally

2018-11-06 Thread Andres Freund
On 2018-11-06 16:47:20 -0500, Tom Lane wrote: > =?UTF-8?Q?Ond=c5=99ej_Bouda?= writes: > >> Ondřej, as a short-term workaround you could prevent the crash > >> by setting that index's recheck_on_update property to false. > > > Thanks for the tip. I am unsuccessful using it, though: > > # ALTER IND

Re: backend crash on DELETE, reproducible locally

2018-11-06 Thread Alvaro Herrera
On 2018-Nov-06, Tom Lane wrote: > =?UTF-8?Q?Ond=c5=99ej_Bouda?= writes: > >> Ondřej, as a short-term workaround you could prevent the crash > >> by setting that index's recheck_on_update property to false. > > > Thanks for the tip. I am unsuccessful using it, though: > > # ALTER INDEX public.sch

Re: backend crash on DELETE, reproducible locally

2018-11-06 Thread Tom Lane
=?UTF-8?Q?Ond=c5=99ej_Bouda?= writes: >> Ondřej, as a short-term workaround you could prevent the crash >> by setting that index's recheck_on_update property to false. > Thanks for the tip. I am unsuccessful using it, though: > # ALTER INDEX public.schedulecard_overlap_idx SET (recheck_on_update

Re: backend crash on DELETE, reproducible locally

2018-11-06 Thread Ondřej Bouda
Ondřej, as a short-term workaround you could prevent the crash by setting that index's recheck_on_update property to false. Thanks for the tip. I am unsuccessful using it, though: # ALTER INDEX public.schedulecard_overlap_idx SET (recheck_on_update = FALSE); ERROR: unrecognized parameter "r

Re: backend crash on DELETE, reproducible locally

2018-11-06 Thread Tom Lane
I wrote: > Interestingly, it doesn't crash if I change the index type to btree, > which I was not expecting because the crashing code seems pretty > independent of the index type. Oh ... duh. The problem here is that ProjIndexIsUnchanged thinks that the type of the index column is identical to th

Re: backend crash on DELETE, reproducible locally

2018-11-06 Thread Tom Lane
=?UTF-8?Q?Ond=c5=99ej_Bouda?= writes: >>> Hm, what are the data types of those columns? > scheduletemplate_id bigint NOT NULL, > period_day smallint NOT NULL, > timerange timerange NOT NULL, OK, so here's a minimal reproducer: drop table schedulecard; create table schedulecard ( scheduletemp

Re: backend crash on DELETE, reproducible locally

2018-11-06 Thread Ondřej Bouda
> Hm, what are the data types of those columns? scheduletemplate_id bigint NOT NULL, period_day smallint NOT NULL, timerange timerange NOT NULL, where timerange is defined as follows: CREATE TYPE public.timerange AS RANGE ( SUBTYPE=time, SUBTYPE_OPCLASS = time_ops ); > And I assume you

Re: backend crash on DELETE, reproducible locally

2018-11-06 Thread Ondřej Bouda
> Foreign-key triggers too? There are four trivial foreign keys from public.schedulecard like this: ALTER TABLE public.schedulecard ADD CONSTRAINT fk_schedulecard_schedulecard FOREIGN KEY (schedulecard_id) REFERENCES public.schedulecard (id) MATCH SIMPLE ON UPDATE NO ACTION ON

Re: backend crash on DELETE, reproducible locally

2018-11-06 Thread Tom Lane
=?UTF-8?Q?Ond=c5=99ej_Bouda?= writes: > There are some simple btree indexes due to foreign keys, and this one, > which seems as the cause for the crashes: > CREATE INDEX schedulecard_overlap_idx > ON public.schedulecard USING gist > (scheduletemplate_id, (period_day::integer % 7), time

Re: backend crash on DELETE, reproducible locally

2018-11-06 Thread Tom Lane
Alvaro Herrera writes: > What indexes are there in this table? Indexes on expressions are > particularly suspect. I had not looked at the "projection index" code before, and now that I have, I am desperately unhappy with it. It's seriously buggy, badly underdocumented, unbelievably inefficient,

Re: backend crash on DELETE, reproducible locally

2018-11-06 Thread Tom Lane
=?UTF-8?Q?Ond=c5=99ej_Bouda?= writes: > I thought triggers should be suspected. However, even when all the three > triggers have been disabled (ALTER TABLE DISABLE TRIGGER), the UPDATE > crashed the server. Foreign-key triggers too? > What else could I try? A stack trace would be really helpf

Re: backend crash on DELETE, reproducible locally

2018-11-06 Thread Ondřej Bouda
Dne 6.11.2018 v 20:45 Alvaro Herrera napsal(a): What indexes are there in this table? Indexes on expressions are particularly suspect. There are some simple btree indexes due to foreign keys, and this one, which seems as the cause for the crashes: CREATE INDEX schedulecard_overlap_idx O

Re: backend crash on DELETE, reproducible locally

2018-11-06 Thread Alvaro Herrera
On 2018-Nov-06, Ondřej Bouda wrote: > > Hmm, this one smells like c203d6cf81b4 -- haven't seen any fixes for > > that one. Can you share more details on this? I think the failing > > update is on table with oid=557732818, but I might be wrong. > > That's exactly the table, public.schedulecard.

Re: backend crash on DELETE, reproducible locally

2018-11-06 Thread Ondřej Bouda
> Hmm, this one smells like c203d6cf81b4 -- haven't seen any fixes for > that one. Can you share more details on this? I think the failing > update is on table with oid=557732818, but I might be wrong. That's exactly the table, public.schedulecard. We issue an UPDATE changing some of its column

Re: backend crash on DELETE, reproducible locally

2018-11-06 Thread Alvaro Herrera
On 2018-Nov-06, Ondřej Bouda wrote: > So we dumped and restored all our databases. After that, the crash on DELETE > never occurred (before, it was several times a day). However, the crash on > UPDATE still occurs on specific rows. We are quite certain no ALTER TABLE > statement was executed on th

Re: backend crash on DELETE, reproducible locally

2018-11-03 Thread Ron
On 11/03/2018 02:19 PM, obo...@email.cz wrote: Hello, we reached the exactly same problem after upgrading to PostgreSQL 11 - the server crashed on a DELETE statement with a trigger.We also observed an AFTER DELETE trigger receiving NULL values in OLD. Now I see the problem seems to be solved

Re: backend crash on DELETE, reproducible locally

2018-11-03 Thread Tom Lane
writes: > we reached the exactly same problem after upgrading to PostgreSQL 11 - the > server crashed on a DELETE statement with a trigger. We also observed an > AFTER DELETE trigger receiving NULL values in OLD. Now I see the problem > seems to be solved (theoretically). Unfortunately, we are not

Re: backend crash on DELETE, reproducible locally

2018-11-03 Thread obouda
sten_addr_saved = true         i =         output_config_variable =         __func__ = "PostmasterMain" #18 0x7f6a6b679616 in main (argc=5, argv=0x7f6a6d3af1f0) at /build/ postgresql-11-oDwOcQ/postgresql-11-11.0/build/../src/backend/main/main.c:228 No locals. Best r

Re: backend crash on DELETE, reproducible locally

2018-11-03 Thread Karsten Hilbert
On Sat, Nov 03, 2018 at 11:39:49AM -0400, Tom Lane wrote: > Karsten Hilbert writes: > > On Fri, Nov 02, 2018 at 11:56:58PM -0400, Tom Lane wrote: > >> I was feeling baffled about this, but it suddenly occurs to me that maybe > >> the bug fixed in 040a1df61/372102b81 explains this. > > > So, I gu

Re: backend crash on DELETE, reproducible locally

2018-11-03 Thread Tom Lane
Karsten Hilbert writes: > On Fri, Nov 02, 2018 at 11:56:58PM -0400, Tom Lane wrote: >> I was feeling baffled about this, but it suddenly occurs to me that maybe >> the bug fixed in 040a1df61/372102b81 explains this. > So, I guess I can work around the issue by the above > manoeuvre and report bac

Re: backend crash on DELETE, reproducible locally

2018-11-03 Thread Karsten Hilbert
On Fri, Nov 02, 2018 at 11:56:58PM -0400, Tom Lane wrote: > > On Thu, Nov 01, 2018 at 11:43:56AM -0400, Tom Lane wrote: > >> Yeah, apparently we've passed a null OLD tuple to an RI_FKey_cascade_del > >> trigger, which surely shouldn't happen. It'd be interesting to look at > >> the set of trigger

Re: backend crash on DELETE, reproducible locally

2018-11-02 Thread Tom Lane
Karsten Hilbert writes: > On Thu, Nov 01, 2018 at 11:43:56AM -0400, Tom Lane wrote: >> Yeah, apparently we've passed a null OLD tuple to an RI_FKey_cascade_del >> trigger, which surely shouldn't happen. It'd be interesting to look at >> the set of triggers on this table. I don't entirely trust p

Re: backend crash on DELETE, reproducible locally

2018-11-01 Thread Karsten Hilbert
On Thu, Nov 01, 2018 at 11:43:56AM -0400, Tom Lane wrote: > Alvaro Herrera writes: > > Ah, now this is interesting. Can you please supply the definition of > > the table? I'm wondering if there is a partitioned table with an FK to > > this one. I'm not quite seeing how come 'tup' is NULL there

Re: backend crash on DELETE, reproducible locally

2018-11-01 Thread Karsten Hilbert
On Thu, Nov 01, 2018 at 12:27:23PM -0300, Alvaro Herrera wrote: > > Program received signal SIGSEGV, Segmentation fault. > > heap_attisnull (tup=0x0, attnum=5, tupleDesc=0xb2990ef4) at > > ./build/../src/backend/access/common/heaptuple.c:403 > > 403 ./build/../src/backend/access/common/heaptuple.

Re: backend crash on DELETE, reproducible locally

2018-11-01 Thread Tom Lane
Alvaro Herrera writes: > Ah, now this is interesting. Can you please supply the definition of > the table? I'm wondering if there is a partitioned table with an FK to > this one. I'm not quite seeing how come 'tup' is NULL there. Yeah, apparently we've passed a null OLD tuple to an RI_FKey_cas

Re: backend crash on DELETE, reproducible locally

2018-11-01 Thread Karsten Hilbert
On Thu, Nov 01, 2018 at 12:27:23PM -0300, Alvaro Herrera wrote: > Ah, now this is interesting. Can you please supply the definition of > the table? Attached. > I'm wondering if there is a partitioned table with an FK to > this one. There is. Both ref.auto_hint and clin.suppressed_hint are usin

Re: backend crash on DELETE, reproducible locally

2018-11-01 Thread Alvaro Herrera
On 2018-Nov-01, Karsten Hilbert wrote: > Program received signal SIGSEGV, Segmentation fault. > heap_attisnull (tup=0x0, attnum=5, tupleDesc=0xb2990ef4) at > ./build/../src/backend/access/common/heaptuple.c:403 > 403 ./build/../src/backend/access/common/heaptuple.c: Datei oder > Verzeichnis ni

Re: backend crash on DELETE, reproducible locally

2018-11-01 Thread Karsten Hilbert
On Thu, Nov 01, 2018 at 12:27:23PM -0300, Alvaro Herrera wrote: > In general terms, this bug report would have been more actionable if you > had shown the definition of the tables involved right from the start. Sorry for that, will supply. Karsten -- GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC8

Re: backend crash on DELETE, reproducible locally

2018-11-01 Thread Karsten Hilbert
On Thu, Nov 01, 2018 at 04:11:33PM +0100, Karsten Hilbert wrote: >> Will attempt to get a stack trace ... Eventually, the stack trace (attached). Thanks for insights, Karsten -- GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC80 7D4F C89B Script started on 2018-11-01 16:16:02+01:00 root@hermes:~/tmp

Re: backend crash on DELETE, reproducible locally

2018-11-01 Thread Karsten Hilbert
On Thu, Nov 01, 2018 at 03:42:57PM +0100, Karsten Hilbert wrote: > Will attempt to get a stack trace ... Meanwhile, in case it helps, an strace of the crash. epoll_wait(7, [{EPOLLIN, {u32=47607120, u64=35184419695952}}], 1, -1) = 1 recv(13, "Q\0\0\0`DELETE FROM ref.auto_hint W".

Re: backend crash on DELETE, reproducible locally

2018-11-01 Thread Karsten Hilbert
> I am running this delete statement via psql > > DELETE FROM ref.auto_hint WHERE title = 'Kontraindikation: ACE/Sartan > <-> Schwangerschaft'; > > and it crashes the backend it is running on. For what it is worth I have identified the physical file gnumed_v21=# select pg_relati

Re: backend crash on DELETE, reproducible locally

2018-11-01 Thread Tom Lane
Karsten Hilbert writes: > I am running this delete statement via psql > DELETE FROM ref.auto_hint WHERE title = 'Kontraindikation: ACE/Sartan > <-> Schwangerschaft'; Doesn't look particularly special ... > What else can I provide/test/look into ? Stack trace would be helpful, self-contai