Re: Aggregate versions of hashing functions (md5, sha1, etc...)

2025-07-11 Thread Merlin Moncure
On Fri, Jul 11, 2025 at 3:01 PM Adrian Klaver wrote: > On 7/11/25 13:30, Merlin Moncure wrote: > > On Fri, Jul 11, 2025 at 10:17 AM Dominique Devienne > > Aside: This may be an unpopular position, but I think the postgres > > extension system is useless for 3rd party contributions until there is

Re: Aggregate versions of hashing functions (md5, sha1, etc...)

2025-07-11 Thread Adrian Klaver
On 7/11/25 13:30, Merlin Moncure wrote: On Fri, Jul 11, 2025 at 10:17 AM Dominique Devienne Aside: This may be an unpopular position, but I think the postgres extension system is useless for 3rd party contributions until there is some way to introduce items in the vein of npm, pip, etc. Ha

Re: Aggregate versions of hashing functions (md5, sha1, etc...)

2025-07-11 Thread Merlin Moncure
On Fri, Jul 11, 2025 at 10:17 AM Dominique Devienne wrote: > On Fri, Jul 11, 2025 at 6:05 PM Florents Tselai > wrote: > > On Fri, Jul 11, 2025, 18:27 Adrian Klaver > wrote: > >> [...] create an extension that incorporates the code. > > > > That's an ideal use case for an extension indeed . > >

Re: Aggregate versions of hashing functions (md5, sha1, etc...)

2025-07-11 Thread Dominique Devienne
On Fri, Jul 11, 2025 at 6:05 PM Florents Tselai wrote: > On Fri, Jul 11, 2025, 18:27 Adrian Klaver wrote: >> [...] create an extension that incorporates the code. > > That's an ideal use case for an extension indeed . Extensions are of no use to me, unfortunately, unless built-in and official. S

Re: Aggregate versions of hashing functions (md5, sha1, etc...)

2025-07-11 Thread Adrian Klaver
On 7/11/25 09:03, Florents Tselai wrote: That's an ideal use case for an extension indeed . @Dominique I'd suggest searching for these function on GitHub, just in case there are already implementations out there. I've seen a lot of weird aggregates out there for niche cases. If not, it migh

Re: Aggregate versions of hashing functions (md5, sha1, etc...)

2025-07-11 Thread Florents Tselai
On Fri, Jul 11, 2025, 18:27 Adrian Klaver wrote: > On 7/11/25 01:49, Dominique Devienne wrote: > > On Thu, Jul 10, 2025 at 6:25 PM Adrian Klaver > wrote: > >> On 7/10/25 04:48, Dominique Devienne wrote: > >>> Seems so logical to me, that these hashing functions were available > >>> are aggregate

Re: Aggregate versions of hashing functions (md5, sha1, etc...)

2025-07-11 Thread Adrian Klaver
On 7/11/25 01:49, Dominique Devienne wrote: On Thu, Jul 10, 2025 at 6:25 PM Adrian Klaver wrote: On 7/10/25 04:48, Dominique Devienne wrote: Seems so logical to me, that these hashing functions were available are aggregates, I can't be the first one to think of that, can it? I've been on thi

Re: Aggregate versions of hashing functions (md5, sha1, etc...)

2025-07-11 Thread Dominique Devienne
On Fri, Jul 11, 2025 at 11:00 AM Dominique Devienne wrote: > The current md5() and pgcrypto.digest() functions roll the x1 > init, xN process, and x1 finish into a single call, processing a > single bytea (or perhaps more intelligently for TOAST'ed values, the > 2K "rows" of those in streaming-fas

Re: Aggregate versions of hashing functions (md5, sha1, etc...)

2025-07-11 Thread Ron Johnson
On Fri, Jul 11, 2025 at 5:01 AM Dominique Devienne wrote: > On Thu, Jul 10, 2025 at 7:11 PM Ron Johnson > wrote: > > On Thu, Jul 10, 2025 at 12:26 PM Adrian Klaver < > adrian.kla...@aklaver.com> wrote: > >> On 7/10/25 04:48, Dominique Devienne wrote: > >> > Seems so logical to me, that these has

Re: Aggregate versions of hashing functions (md5, sha1, etc...)

2025-07-11 Thread Dominique Devienne
On Thu, Jul 10, 2025 at 9:03 PM Merlin Moncure wrote: > On Thu, Jul 10, 2025 at 5:48 AM Dominique Devienne > wrote: >> We store scientific information in PostgreSQL, and some of that is >> bytea and large, thus we must "chunk it" both for performance, and not >> be limited to 1GB (we do exceed t

Re: Aggregate versions of hashing functions (md5, sha1, etc...)

2025-07-11 Thread Dominique Devienne
On Thu, Jul 10, 2025 at 7:11 PM Ron Johnson wrote: > On Thu, Jul 10, 2025 at 12:26 PM Adrian Klaver > wrote: >> On 7/10/25 04:48, Dominique Devienne wrote: >> > Seems so logical to me, that these hashing functions were available >> > are aggregates, I can't be the first one to think of that, can

Re: Aggregate versions of hashing functions (md5, sha1, etc...)

2025-07-11 Thread Dominique Devienne
On Thu, Jul 10, 2025 at 6:25 PM Adrian Klaver wrote: > On 7/10/25 04:48, Dominique Devienne wrote: > > Seems so logical to me, that these hashing functions were available > > are aggregates, I can't be the first one to think of that, can it? > > I've been on this list since late 2002 and I don't r

Re: Aggregate versions of hashing functions (md5, sha1, etc...)

2025-07-10 Thread Merlin Moncure
On Thu, Jul 10, 2025 at 5:48 AM Dominique Devienne wrote: > We store scientific information in PostgreSQL, and some of that is > bytea and large, thus we must "chunk it" both for performance, and not > be limited to 1GB (we do exceed that, in rare occasions). > > Recently I added md5/sha1 hashi

Re: Aggregate versions of hashing functions (md5, sha1, etc...)

2025-07-10 Thread Ron Johnson
On Thu, Jul 10, 2025 at 12:26 PM Adrian Klaver wrote: > On 7/10/25 04:48, Dominique Devienne wrote: > > > Seems so logical to me, that these hashing functions were available > > are aggregates, I can't be the first one to think of that, can it? > > > > I've been on this list since late 2002 and I

Re: Aggregate versions of hashing functions (md5, sha1, etc...)

2025-07-10 Thread Adrian Klaver
On 7/10/25 04:48, Dominique Devienne wrote: Seems so logical to me, that these hashing functions were available are aggregates, I can't be the first one to think of that, can it? I've been on this list since late 2002 and I don't recall this ever being brought up. Now it is entirely possible

Aggregate versions of hashing functions (md5, sha1, etc...)

2025-07-10 Thread Dominique Devienne
We store scientific information in PostgreSQL, and some of that is bytea and large, thus we must "chunk it" both for performance, and not be limited to 1GB (we do exceed that, in rare occasions). Recently I added md5/sha1 hashing support for such values (for various reasons, to track corruptions i