Re: seq vs index scan in join query

2017-11-29 Thread Andres Freund
age pointing to 100 tuples of the same value - that'd result in at least a 100 buffer accesses, but it'd be highly likely that they'll be in cache. Greetings, Andres Freund

Re: [GENERAL] postmaster deadlock while logging after syslogger exited

2017-12-04 Thread Andres Freund
; and the workload that generates the very large query strings only runs > occasionally. Those strings are also not logged unless something's gone > wrong. FWIW, I'd like to see a report of this around the time the issue occurred before doing anything further here. Greetings, Andres Freund

Re: [GENERAL] postmaster deadlock while logging after syslogger exited

2017-12-04 Thread Andres Freund
On 2017-12-04 13:57:52 -0800, David Pacheco wrote: > On Mon, Dec 4, 2017 at 12:23 PM, Andres Freund wrote: > > FWIW, I'd like to see a report of this around the time the issue > > occurred before doing anything further here. > > > > > This failure begins when

Re: Does Postgresql 10 query partitions in parallel?

2017-12-06 Thread Andres Freund
ll use > multiple cores.  you have to explicity invoke it.   "you have to explicitly invoke it" - huh? > As this is a first > implementation, its fairly limited as to the sorts of queries that can be > parallized, but this will be enhanced in future versions. It's already been expanded a lot since 9.6. Greetings, Andres Freund

Re: SearchSysCache() tutorial?

2017-12-11 Thread Andres Freund
On 2017-12-11 14:42:33 -0500, Melvin Davidson wrote: > There is no guarantee that information will be in syscache at any point in > time. It will, however, always be in the postgreSQL catalogs. That is the > whole point > of having them, and the SQL language. This doesn't make much sense. Paul's w

Re: SearchSysCache() tutorial?

2017-12-11 Thread Andres Freund
re on a single attribute > of type geometry". I don't think there's a way to do this with a single syscache, there won't be an index than can cover all of these. I'd suggest using RelationGetIndexList(), and then filtering for gist and attribute type on the returned value. Greetings, Andres Freund

Re: SearchSysCache() tutorial?

2017-12-11 Thread Andres Freund
On 2017-12-11 12:48:27 -0800, Paul Ramsey wrote: > > > On Dec 11, 2017, at 12:40 PM, Andres Freund wrote: > > > > On 2017-12-11 11:26:51 -0800, Paul Ramsey wrote: > >> On Mon, Dec 11, 2017 at 11:25 AM, Paul Ramsey > >> wrote: > >>> Is there a

Re: AWS Aurora and PG 10

2017-12-19 Thread Andres Freund
Hi, On 2017-12-19 17:15:54 +0100, Rakesh Kumar wrote: > Has AWS Aurora started supporting PG 10. Their website still talks about 9.6. This is a question for Amazon, not for the postgres community. Greetings, Andres Freund

Re: data-checksums

2018-01-09 Thread Andres Freund
On 2018-01-09 18:58:41 +0100, Rakesh Kumar wrote: > > > That said, imv anyway, the performance hit is small and having checksums > > is well worth it. > > I also would like to believe that the hit is small, but when PG > official document writes "noticeable performance penalty", it becomes > diff

Re: data-checksums

2018-01-09 Thread Andres Freund
l :-) > > > > Why believe, when you can measure? > > yup doing that. But I still feel that PG documentation should stay > away from such scare mongering. Or did the lawyers write that :) So we should rather lie about it having a potential for performance impact? Who'd be helped by that? Greetings, Andres Freund

Re: Sv: Re: data-checksums

2018-01-09 Thread Andres Freund
ome impact, but if there's bigger impact it's much more likely to be related to the fact that some hint bit writes to a page now needs to be WAL logged. Andres Freund

Re: Sv: Re: Sv: Re: Sv: Re: data-checksums

2018-01-09 Thread Andres Freund
much more likely to be related to the fact that some hint bit > writes to a page now needs to be WAL logged. which isn't mitigated by SIMD / hardware CRC / whatnot. Greetings, Andres Freund

Re: Sv: Re: Sv: Re: Sv: Re: Sv: Re: data-checksums

2018-01-09 Thread Andres Freund
On 2018-01-10 01:31:58 +0100, Andreas Joseph Krogh wrote: > På onsdag 10. januar 2018 kl. 01:01:26, skrev Andres Freund > <mailto:and...@anarazel.de>>: > On 2018-01-10 00:25:08 +0100, Andreas Joseph Krogh wrote: > > But SIMD-instructions are also HW-accellerated by mode

Re: Sv: Re: Sv: Re: Sv: Re: Sv: Re: data-checksums

2018-01-10 Thread Andres Freund
quot; with the above, as it should be clear that doing > more work implies an impact on performance and that avoids the whole > question of trying to characterize in a general way something that can't > be generalized (as it's workload dependent). -1. I think this is underplaying the cost. Greetings, Andres Freund

Re: New Copy Formats - avro/orc/parquet

2018-02-11 Thread Andres Freund
27;s going to be way way much more than 1%. It's trivial to construct cases where input parsing / output formatting takes the majority of the time. And a lot of that you're going to be able to avoid with binary formats. Greetings, Andres Freund

Re: New Copy Formats - avro/orc/parquet

2018-02-11 Thread Andres Freund
On February 11, 2018 12:00:12 PM PST, Nicolas Paris wrote: >> > That is true, but the question is how significant the overhead is. >If >> > it's 50% then reducing it would make perfect sense. If it's 1% then >no >> > one if going to be bothered by it. >> >> I think it's pretty clear that it's g

Re: New Copy Formats - avro/orc/parquet

2018-02-11 Thread Andres Freund
o 30% larger than the text one. On the > contrary, an ORC file can be up to 10 times smaller than a text base > format. That seems largely irrelevant when arguing about using PROGRAM though, right? Greetings, Andres Freund

Re: New Copy Formats - avro/orc/parquet

2018-02-11 Thread Andres Freund
s / nulls arrays - finish copy in / out we'd also need to expose a few more utility functions from copy.c externally. I think this'd require a good bit of cleanup in copy.c... Greetings, Andres Freund

Re: New Copy Formats - avro/orc/parquet

2018-02-11 Thread Andres Freund
On February 11, 2018 2:48:13 PM PST, Tom Lane wrote: > (Any such patch should manage >to turn COPY-CSV into an extension, at least so far as copy.c is >concerned, even if we don't package it as one.) Yea, I was thinking we should move all three (default, csv, binary) supported formats to using

Re: PQConsumeinput stuck on recv

2018-02-23 Thread Andres Freund
e the same. > I wonder if anyone has any tip on what to look for next... Any chance you're occasionally forking and then interacting with the connection in the forked process? Greetings, Andres Freund

Re: PQConsumeinput stuck on recv

2018-02-23 Thread Andres Freund
the backtrace again, after installing debugging symbols? It'd certainly be helpful to see the exact path to the blocking syscall. Greetings, Andres Freund

<    1   2   3