BackupRepository changes

[Bruno Roustant]

I noticed two points that I propose to change in some BackupRepository
implementations and in BackupRepositoryFactory:

1- Two implementations (S3BackupRepository, GCSBackupRepository) compute
and verify the checksum of index files being copied. Other implementations
do not.
=> I propose to remove this checksum verification, to simply copy the bytes.
First, because it seems to me it is useless to verify the checksum before
backup (checking it after restoring seems more useful).
Second, because for the encryption module in dev in solr-sandbox, I need to
ensure the bytes copied stay encrypted, and the checksum is not verifiable
in this case.

2- There is a special logic in BackupRepositoryFactory when creating the
TrackingBackupRepository. TrackingBackupRepository delegates to another
BackupDirectory, and it is only used for tests.
=> I propose to introduce FilterBackupRepository to delegate to a
BackupDirectory in a more generic way. TrackingBackupRepository would
become a FilterBackupRepository.
Again, I would use an EncryptionBackupRepository in solr-sandbox to
delegate to any BackupRepository, while ensuring copied bytes stay
encrypted.

I would appreciate any feedback :)

Re: New branch and feature freeze for Solr 9.5.0

[Jason Gerlowski]

I'm working on the Lucene upgrade now.  That will still need some time in
review (and maybe a day to let tests bake?), so I'm fine if you backport
SOLR-17068 in that timeframe.  But let's draw the line once the Lucene
9.9.2 upgrade is in, unless something else urgent comes up.  I'll target
Wednesday or Thursday for the Solr RC1.

Jason

On Mon, Jan 29, 2024 at 4:15 PM Eric Pugh 
wrote:

> Jason, I tackled SOLR-17068 (the one you reminded me of) and I’d love to
> get it into 9.5 since right now we have a terrible mish mash of bin/solr
> post and bin/post in the ref guide and docs.
>
> Could someone review https://github.com/apache/solr/pull/2227 and if it
> looks good could we sneak it into 9.5?
>
> Eric
>
>
> > On Jan 26, 2024, at 6:29 PM, Eric Pugh 
> wrote:
> >
> > Backport to branch_9_5 is done.
> >
> >
> >> On Jan 26, 2024, at 1:11 PM, Jason Gerlowski 
> wrote:
> >>
> >> Go ahead and backport on your own!  I'm still waiting on Lucene 9.9.2,
> so
> >> there shouldn't be any branch-contention on my end.
> >>
> >> Relatedly, Lucene has their RC1 out there and things look good a day or
> two
> >> into their VOTE, so with any luck we'll be able to get a Solr 9.5 RC
> >> together early next week!
> >>
> >> Best,
> >>
> >> Jason
> >>
> >> On Fri, Jan 26, 2024 at 8:49 AM Eric Pugh  wrote:
> >>
> >>> I am about to merge SOLR-17112, and will backport it to branch_9x.
> Jason,
> >>> do you backport it over to the branch_9_5 or do I?
> >>>
> >>> ERic
> >>>
> >>>
> >>> On 2024/01/23 19:08:10 Jason Gerlowski wrote:
> > It was hoped SOLR-17112 would make 9.4.1 but it didn't as no PR was
>  proposed.
> 
> > SOLR-17120 [is] nominated for inclusion in the 9.5.0 release
> 
>  Those both sound quick and reasonable; they've got a +1 from me to go
> >>> into
>  9.5 (assuming the contributor decides to continue with SOLR-17112).
> 
> > Considering Lucene 9.9.2 is being planned, I think it would be better
> >>> to
> > upgrade Solr to the to-be-released version so users have to deal with
> > fewer upgrade cycles.
> 
>  Yeah, that might be best; I hadn't realized we weren't already on the
>  latest Lucene 9.x.  I've created SOLR-17128 to track our Lucene
> upgrade
>  once 9.9.2 is available.
> 
>  Obviously this is a longer delay than some of the tickets above, and
> will
>  mean we won't be cutting a Solr RC this week.  We can pick a new date
> for
>  the initial Solr 9.5 RC once Lucene 9.9.2 is available.
> 
>  Best,
> 
>  Jason
> 
>  On Tue, Jan 23, 2024 at 1:37 PM Anshum Gupta 
> >>> wrote:
> 
> > Considering Lucene 9.9.2 is being planned, I think it would be better
> >>> to
> > upgrade Solr to the to-be-released version so users have to deal with
> >>> fewer
> > upgrade cycles.
> >
> > To highlight, there are about 90 odd changes in the Lucene 9.9.x
> line.
> >
> > -Anshum
> >
> > On Tue, Jan 23, 2024 at 8:47 AM David Smiley 
> >>> wrote:
> >
> >> FYI It was hoped SOLR-17112
> >> https://issues.apache.org/jira/browse/SOLR-17112 "bin/solr script
> >> doesn't do ps properly on some systems" would make 9.4.1 but it
> >>> didn't
> >> as no PR was proposed.  There still isn't one but a contributor is
> >> thinking about it.
> >>
> >> On Tue, Jan 23, 2024 at 11:30 AM Christine Poerschke (BLOOMBERG/
> >> LONDON)  wrote:
> >>>
> >>> Just to cross-reference things further (Jason is already aware) --
> >> https://issues.apache.org/jira/browse/SOLR-17120 and
> >> https://github.com/apache/solr/pull/2214 are nominated for
> >>> inclusion in
> >> the 9.5 release, and as always additional reviews and inputs are
> >>> welcome.
> >>>
> >>> Regards,
> >>> Christine
> >>>
> >>> From: dev@solr.apache.org At: 01/22/24 17:30:35 UTCTo:
> >> dev@solr.apache.org
> >>> Subject: New branch and feature freeze for Solr 9.5.0
> >>>
> >>> NOTICE:
> >>>
> >>> Branch branch_9_5 has been cut and versions updated to 9.6 on the
> > stable
> >>> branch.
> >>>
> >>> Please observe the normal rules:
> >>>
> >>> * No new features may be committed to the branch.
> >>> * Documentation patches, build patches and serious bug fixes may be
> >>>  committed to the branch. However, you should submit all patches
> >>> you
> >>>  want to commit to Jira first to give others the chance to review
> >>>  and possibly vote against the patch. Keep in mind that it is our
> >>>  main intention to keep the branch as stable as possible.
> >>> * All patches that are intended for the branch should first be
> > committed
> >>>  to the unstable branch, merged into the stable branch, and then
> >>> into
> >>>  the current release branch.
> >>> * Normal unstable and stable branch development may continue as
> >>> usual.
> >>>  However, if you plan to commit a big change to the unstabl

Re: New branch and feature freeze for Solr 9.5.0

[Nazerke S]

Hi Jason,

fyi, we have an open PR for  the
Lucene upgrade (v9.9.2) waiting for review.

On Tue, Jan 30, 2024 at 6:21 PM Jason Gerlowski 
wrote:

> I'm working on the Lucene upgrade now.  That will still need some time in
> review (and maybe a day to let tests bake?), so I'm fine if you backport
> SOLR-17068 in that timeframe.  But let's draw the line once the Lucene
> 9.9.2 upgrade is in, unless something else urgent comes up.  I'll target
> Wednesday or Thursday for the Solr RC1.
>
> Jason
>
> On Mon, Jan 29, 2024 at 4:15 PM Eric Pugh  >
> wrote:
>
> > Jason, I tackled SOLR-17068 (the one you reminded me of) and I’d love to
> > get it into 9.5 since right now we have a terrible mish mash of bin/solr
> > post and bin/post in the ref guide and docs.
> >
> > Could someone review https://github.com/apache/solr/pull/2227 and if it
> > looks good could we sneak it into 9.5?
> >
> > Eric
> >
> >
> > > On Jan 26, 2024, at 6:29 PM, Eric Pugh <
> ep...@opensourceconnections.com>
> > wrote:
> > >
> > > Backport to branch_9_5 is done.
> > >
> > >
> > >> On Jan 26, 2024, at 1:11 PM, Jason Gerlowski 
> > wrote:
> > >>
> > >> Go ahead and backport on your own!  I'm still waiting on Lucene 9.9.2,
> > so
> > >> there shouldn't be any branch-contention on my end.
> > >>
> > >> Relatedly, Lucene has their RC1 out there and things look good a day
> or
> > two
> > >> into their VOTE, so with any luck we'll be able to get a Solr 9.5 RC
> > >> together early next week!
> > >>
> > >> Best,
> > >>
> > >> Jason
> > >>
> > >> On Fri, Jan 26, 2024 at 8:49 AM Eric Pugh  wrote:
> > >>
> > >>> I am about to merge SOLR-17112, and will backport it to branch_9x.
> > Jason,
> > >>> do you backport it over to the branch_9_5 or do I?
> > >>>
> > >>> ERic
> > >>>
> > >>>
> > >>> On 2024/01/23 19:08:10 Jason Gerlowski wrote:
> > > It was hoped SOLR-17112 would make 9.4.1 but it didn't as no PR was
> >  proposed.
> > 
> > > SOLR-17120 [is] nominated for inclusion in the 9.5.0 release
> > 
> >  Those both sound quick and reasonable; they've got a +1 from me to
> go
> > >>> into
> >  9.5 (assuming the contributor decides to continue with SOLR-17112).
> > 
> > > Considering Lucene 9.9.2 is being planned, I think it would be
> better
> > >>> to
> > > upgrade Solr to the to-be-released version so users have to deal
> with
> > > fewer upgrade cycles.
> > 
> >  Yeah, that might be best; I hadn't realized we weren't already on
> the
> >  latest Lucene 9.x.  I've created SOLR-17128 to track our Lucene
> > upgrade
> >  once 9.9.2 is available.
> > 
> >  Obviously this is a longer delay than some of the tickets above, and
> > will
> >  mean we won't be cutting a Solr RC this week.  We can pick a new
> date
> > for
> >  the initial Solr 9.5 RC once Lucene 9.9.2 is available.
> > 
> >  Best,
> > 
> >  Jason
> > 
> >  On Tue, Jan 23, 2024 at 1:37 PM Anshum Gupta <
> ans...@anshumgupta.net>
> > >>> wrote:
> > 
> > > Considering Lucene 9.9.2 is being planned, I think it would be
> better
> > >>> to
> > > upgrade Solr to the to-be-released version so users have to deal
> with
> > >>> fewer
> > > upgrade cycles.
> > >
> > > To highlight, there are about 90 odd changes in the Lucene 9.9.x
> > line.
> > >
> > > -Anshum
> > >
> > > On Tue, Jan 23, 2024 at 8:47 AM David Smiley 
> > >>> wrote:
> > >
> > >> FYI It was hoped SOLR-17112
> > >> https://issues.apache.org/jira/browse/SOLR-17112 "bin/solr script
> > >> doesn't do ps properly on some systems" would make 9.4.1 but it
> > >>> didn't
> > >> as no PR was proposed.  There still isn't one but a contributor is
> > >> thinking about it.
> > >>
> > >> On Tue, Jan 23, 2024 at 11:30 AM Christine Poerschke (BLOOMBERG/
> > >> LONDON)  wrote:
> > >>>
> > >>> Just to cross-reference things further (Jason is already aware)
> --
> > >> https://issues.apache.org/jira/browse/SOLR-17120 and
> > >> https://github.com/apache/solr/pull/2214 are nominated for
> > >>> inclusion in
> > >> the 9.5 release, and as always additional reviews and inputs are
> > >>> welcome.
> > >>>
> > >>> Regards,
> > >>> Christine
> > >>>
> > >>> From: dev@solr.apache.org At: 01/22/24 17:30:35 UTCTo:
> > >> dev@solr.apache.org
> > >>> Subject: New branch and feature freeze for Solr 9.5.0
> > >>>
> > >>> NOTICE:
> > >>>
> > >>> Branch branch_9_5 has been cut and versions updated to 9.6 on the
> > > stable
> > >>> branch.
> > >>>
> > >>> Please observe the normal rules:
> > >>>
> > >>> * No new features may be committed to the branch.
> > >>> * Documentation patches, build patches and serious bug fixes may
> be
> > >>>  committed to the branch. However, you should submit all patches
> > >>> you
> > >>>  want to commit to Jira first to give ot

Re: BackupRepository changes

[David Smiley]

On Tue, Jan 30, 2024 at 4:02 AM Bruno Roustant  wrote:
>
> I noticed two points that I propose to change in some BackupRepository
> implementations and in BackupRepositoryFactory:
>
> 1- Two implementations (S3BackupRepository, GCSBackupRepository) compute
> and verify the checksum of index files being copied. Other implementations
> do not.
> => I propose to remove this checksum verification, to simply copy the bytes.
> First, because it seems to me it is useless to verify the checksum before
> backup (checking it after restoring seems more useful).

Isn't the intent to ensure we don't waste time/space creating a
useless backup of something that is, I suppose, already corrupted?

> Second, because for the encryption module in dev in solr-sandbox, I need to
> ensure the bytes copied stay encrypted, and the checksum is not verifiable
> in this case.
>
> 2- There is a special logic in BackupRepositoryFactory when creating the
> TrackingBackupRepository. TrackingBackupRepository delegates to another
> BackupDirectory, and it is only used for tests.
> => I propose to introduce FilterBackupRepository to delegate to a
> BackupDirectory in a more generic way. TrackingBackupRepository would
> become a FilterBackupRepository.
> Again, I would use an EncryptionBackupRepository in solr-sandbox to
> delegate to any BackupRepository, while ensuring copied bytes stay
> encrypted.
>
> I would appreciate any feedback :)

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: New branch and feature freeze for Solr 9.5.0

[Jason Gerlowski]

Yes - I saw that right after my last email.  Thanks for all your work on
that Nazerke!  I've merged and backported that PR; we're now on Lucene
9.9.2!

As soon as Eric's P #2227 is merged and backported I'll start on our RC!

Best,

Jason

On Tue, Jan 30, 2024 at 7:33 AM Nazerke S  wrote:

> Hi Jason,
>
> fyi, we have an open PR for  the
> Lucene upgrade (v9.9.2) waiting for review.
>
> On Tue, Jan 30, 2024 at 6:21 PM Jason Gerlowski 
> wrote:
>
> > I'm working on the Lucene upgrade now.  That will still need some time in
> > review (and maybe a day to let tests bake?), so I'm fine if you backport
> > SOLR-17068 in that timeframe.  But let's draw the line once the Lucene
> > 9.9.2 upgrade is in, unless something else urgent comes up.  I'll target
> > Wednesday or Thursday for the Solr RC1.
> >
> > Jason
> >
> > On Mon, Jan 29, 2024 at 4:15 PM Eric Pugh <
> ep...@opensourceconnections.com
> > >
> > wrote:
> >
> > > Jason, I tackled SOLR-17068 (the one you reminded me of) and I’d love
> to
> > > get it into 9.5 since right now we have a terrible mish mash of
> bin/solr
> > > post and bin/post in the ref guide and docs.
> > >
> > > Could someone review https://github.com/apache/solr/pull/2227 and if
> it
> > > looks good could we sneak it into 9.5?
> > >
> > > Eric
> > >
> > >
> > > > On Jan 26, 2024, at 6:29 PM, Eric Pugh <
> > ep...@opensourceconnections.com>
> > > wrote:
> > > >
> > > > Backport to branch_9_5 is done.
> > > >
> > > >
> > > >> On Jan 26, 2024, at 1:11 PM, Jason Gerlowski  >
> > > wrote:
> > > >>
> > > >> Go ahead and backport on your own!  I'm still waiting on Lucene
> 9.9.2,
> > > so
> > > >> there shouldn't be any branch-contention on my end.
> > > >>
> > > >> Relatedly, Lucene has their RC1 out there and things look good a day
> > or
> > > two
> > > >> into their VOTE, so with any luck we'll be able to get a Solr 9.5 RC
> > > >> together early next week!
> > > >>
> > > >> Best,
> > > >>
> > > >> Jason
> > > >>
> > > >> On Fri, Jan 26, 2024 at 8:49 AM Eric Pugh  wrote:
> > > >>
> > > >>> I am about to merge SOLR-17112, and will backport it to branch_9x.
> > > Jason,
> > > >>> do you backport it over to the branch_9_5 or do I?
> > > >>>
> > > >>> ERic
> > > >>>
> > > >>>
> > > >>> On 2024/01/23 19:08:10 Jason Gerlowski wrote:
> > > > It was hoped SOLR-17112 would make 9.4.1 but it didn't as no PR
> was
> > >  proposed.
> > > 
> > > > SOLR-17120 [is] nominated for inclusion in the 9.5.0 release
> > > 
> > >  Those both sound quick and reasonable; they've got a +1 from me to
> > go
> > > >>> into
> > >  9.5 (assuming the contributor decides to continue with
> SOLR-17112).
> > > 
> > > > Considering Lucene 9.9.2 is being planned, I think it would be
> > better
> > > >>> to
> > > > upgrade Solr to the to-be-released version so users have to deal
> > with
> > > > fewer upgrade cycles.
> > > 
> > >  Yeah, that might be best; I hadn't realized we weren't already on
> > the
> > >  latest Lucene 9.x.  I've created SOLR-17128 to track our Lucene
> > > upgrade
> > >  once 9.9.2 is available.
> > > 
> > >  Obviously this is a longer delay than some of the tickets above,
> and
> > > will
> > >  mean we won't be cutting a Solr RC this week.  We can pick a new
> > date
> > > for
> > >  the initial Solr 9.5 RC once Lucene 9.9.2 is available.
> > > 
> > >  Best,
> > > 
> > >  Jason
> > > 
> > >  On Tue, Jan 23, 2024 at 1:37 PM Anshum Gupta <
> > ans...@anshumgupta.net>
> > > >>> wrote:
> > > 
> > > > Considering Lucene 9.9.2 is being planned, I think it would be
> > better
> > > >>> to
> > > > upgrade Solr to the to-be-released version so users have to deal
> > with
> > > >>> fewer
> > > > upgrade cycles.
> > > >
> > > > To highlight, there are about 90 odd changes in the Lucene 9.9.x
> > > line.
> > > >
> > > > -Anshum
> > > >
> > > > On Tue, Jan 23, 2024 at 8:47 AM David Smiley  >
> > > >>> wrote:
> > > >
> > > >> FYI It was hoped SOLR-17112
> > > >> https://issues.apache.org/jira/browse/SOLR-17112 "bin/solr
> script
> > > >> doesn't do ps properly on some systems" would make 9.4.1 but it
> > > >>> didn't
> > > >> as no PR was proposed.  There still isn't one but a contributor
> is
> > > >> thinking about it.
> > > >>
> > > >> On Tue, Jan 23, 2024 at 11:30 AM Christine Poerschke (BLOOMBERG/
> > > >> LONDON)  wrote:
> > > >>>
> > > >>> Just to cross-reference things further (Jason is already aware)
> > --
> > > >> https://issues.apache.org/jira/browse/SOLR-17120 and
> > > >> https://github.com/apache/solr/pull/2214 are nominated for
> > > >>> inclusion in
> > > >> the 9.5 release, and as always additional reviews and inputs are
> > > >>> welcome.
> > > >>>
> > > >>> Regards,
> > > >>> Christine
> > > >>>
> > > >>> From: dev@solr.apache.org At: 01/22/24

Re: New branch and feature freeze for Solr 9.5.0

[Ishan Chattopadhyaya]

Should we wait at least 2-3 days for the Lucene 9.9.2 upgrade to bake
before we cut a release?

On Tue, 30 Jan, 2024, 7:50 pm Jason Gerlowski, 
wrote:

> Yes - I saw that right after my last email.  Thanks for all your work on
> that Nazerke!  I've merged and backported that PR; we're now on Lucene
> 9.9.2!
>
> As soon as Eric's P #2227 is merged and backported I'll start on our RC!
>
> Best,
>
> Jason
>
> On Tue, Jan 30, 2024 at 7:33 AM Nazerke S  wrote:
>
> > Hi Jason,
> >
> > fyi, we have an open PR for
> the
> > Lucene upgrade (v9.9.2) waiting for review.
> >
> > On Tue, Jan 30, 2024 at 6:21 PM Jason Gerlowski 
> > wrote:
> >
> > > I'm working on the Lucene upgrade now.  That will still need some time
> in
> > > review (and maybe a day to let tests bake?), so I'm fine if you
> backport
> > > SOLR-17068 in that timeframe.  But let's draw the line once the Lucene
> > > 9.9.2 upgrade is in, unless something else urgent comes up.  I'll
> target
> > > Wednesday or Thursday for the Solr RC1.
> > >
> > > Jason
> > >
> > > On Mon, Jan 29, 2024 at 4:15 PM Eric Pugh <
> > ep...@opensourceconnections.com
> > > >
> > > wrote:
> > >
> > > > Jason, I tackled SOLR-17068 (the one you reminded me of) and I’d love
> > to
> > > > get it into 9.5 since right now we have a terrible mish mash of
> > bin/solr
> > > > post and bin/post in the ref guide and docs.
> > > >
> > > > Could someone review https://github.com/apache/solr/pull/2227 and if
> > it
> > > > looks good could we sneak it into 9.5?
> > > >
> > > > Eric
> > > >
> > > >
> > > > > On Jan 26, 2024, at 6:29 PM, Eric Pugh <
> > > ep...@opensourceconnections.com>
> > > > wrote:
> > > > >
> > > > > Backport to branch_9_5 is done.
> > > > >
> > > > >
> > > > >> On Jan 26, 2024, at 1:11 PM, Jason Gerlowski <
> gerlowsk...@gmail.com
> > >
> > > > wrote:
> > > > >>
> > > > >> Go ahead and backport on your own!  I'm still waiting on Lucene
> > 9.9.2,
> > > > so
> > > > >> there shouldn't be any branch-contention on my end.
> > > > >>
> > > > >> Relatedly, Lucene has their RC1 out there and things look good a
> day
> > > or
> > > > two
> > > > >> into their VOTE, so with any luck we'll be able to get a Solr 9.5
> RC
> > > > >> together early next week!
> > > > >>
> > > > >> Best,
> > > > >>
> > > > >> Jason
> > > > >>
> > > > >> On Fri, Jan 26, 2024 at 8:49 AM Eric Pugh 
> wrote:
> > > > >>
> > > > >>> I am about to merge SOLR-17112, and will backport it to
> branch_9x.
> > > > Jason,
> > > > >>> do you backport it over to the branch_9_5 or do I?
> > > > >>>
> > > > >>> ERic
> > > > >>>
> > > > >>>
> > > > >>> On 2024/01/23 19:08:10 Jason Gerlowski wrote:
> > > > > It was hoped SOLR-17112 would make 9.4.1 but it didn't as no PR
> > was
> > > >  proposed.
> > > > 
> > > > > SOLR-17120 [is] nominated for inclusion in the 9.5.0 release
> > > > 
> > > >  Those both sound quick and reasonable; they've got a +1 from me
> to
> > > go
> > > > >>> into
> > > >  9.5 (assuming the contributor decides to continue with
> > SOLR-17112).
> > > > 
> > > > > Considering Lucene 9.9.2 is being planned, I think it would be
> > > better
> > > > >>> to
> > > > > upgrade Solr to the to-be-released version so users have to
> deal
> > > with
> > > > > fewer upgrade cycles.
> > > > 
> > > >  Yeah, that might be best; I hadn't realized we weren't already
> on
> > > the
> > > >  latest Lucene 9.x.  I've created SOLR-17128 to track our Lucene
> > > > upgrade
> > > >  once 9.9.2 is available.
> > > > 
> > > >  Obviously this is a longer delay than some of the tickets above,
> > and
> > > > will
> > > >  mean we won't be cutting a Solr RC this week.  We can pick a new
> > > date
> > > > for
> > > >  the initial Solr 9.5 RC once Lucene 9.9.2 is available.
> > > > 
> > > >  Best,
> > > > 
> > > >  Jason
> > > > 
> > > >  On Tue, Jan 23, 2024 at 1:37 PM Anshum Gupta <
> > > ans...@anshumgupta.net>
> > > > >>> wrote:
> > > > 
> > > > > Considering Lucene 9.9.2 is being planned, I think it would be
> > > better
> > > > >>> to
> > > > > upgrade Solr to the to-be-released version so users have to
> deal
> > > with
> > > > >>> fewer
> > > > > upgrade cycles.
> > > > >
> > > > > To highlight, there are about 90 odd changes in the Lucene
> 9.9.x
> > > > line.
> > > > >
> > > > > -Anshum
> > > > >
> > > > > On Tue, Jan 23, 2024 at 8:47 AM David Smiley <
> dsmi...@apache.org
> > >
> > > > >>> wrote:
> > > > >
> > > > >> FYI It was hoped SOLR-17112
> > > > >> https://issues.apache.org/jira/browse/SOLR-17112 "bin/solr
> > script
> > > > >> doesn't do ps properly on some systems" would make 9.4.1 but
> it
> > > > >>> didn't
> > > > >> as no PR was proposed.  There still isn't one but a
> contributor
> > is
> > > > >> thinking about it.
> > > > >>
> > > > >> On Tue, Jan 23, 2024 at 11:30 AM Christine Poerschk

Re: New branch and feature freeze for Solr 9.5.0

[Jason Gerlowski]

Maybe?  Obviously that was my initial plan as I mentioned above.  But I was
pleasantly surprised to see the number of folks that chimed in with their
own test results and +1's on Nazerke's Lucene-upgrade PR.  It seemed
comparable to the data points we'd get out of a multi-day "bake" in
Jenkins.  So I figured we might expedite the baking, or at least let it
"bake" in parallel while I work on the RC.

But I can also wait an additional few days if folks think it's
necessary/valuable?

(Also worth mentioning that this is still theoretical at this point - we're
still waiting on a fix for SOLR-17068 unless Eric changes his mind about
getting it in 9.5.)

On Tue, Jan 30, 2024 at 9:32 AM Ishan Chattopadhyaya <
ichattopadhy...@gmail.com> wrote:

> Should we wait at least 2-3 days for the Lucene 9.9.2 upgrade to bake
> before we cut a release?
>
> On Tue, 30 Jan, 2024, 7:50 pm Jason Gerlowski, 
> wrote:
>
> > Yes - I saw that right after my last email.  Thanks for all your work on
> > that Nazerke!  I've merged and backported that PR; we're now on Lucene
> > 9.9.2!
> >
> > As soon as Eric's P #2227 is merged and backported I'll start on our RC!
> >
> > Best,
> >
> > Jason
> >
> > On Tue, Jan 30, 2024 at 7:33 AM Nazerke S  wrote:
> >
> > > Hi Jason,
> > >
> > > fyi, we have an open PR for
> > the
> > > Lucene upgrade (v9.9.2) waiting for review.
> > >
> > > On Tue, Jan 30, 2024 at 6:21 PM Jason Gerlowski  >
> > > wrote:
> > >
> > > > I'm working on the Lucene upgrade now.  That will still need some
> time
> > in
> > > > review (and maybe a day to let tests bake?), so I'm fine if you
> > backport
> > > > SOLR-17068 in that timeframe.  But let's draw the line once the
> Lucene
> > > > 9.9.2 upgrade is in, unless something else urgent comes up.  I'll
> > target
> > > > Wednesday or Thursday for the Solr RC1.
> > > >
> > > > Jason
> > > >
> > > > On Mon, Jan 29, 2024 at 4:15 PM Eric Pugh <
> > > ep...@opensourceconnections.com
> > > > >
> > > > wrote:
> > > >
> > > > > Jason, I tackled SOLR-17068 (the one you reminded me of) and I’d
> love
> > > to
> > > > > get it into 9.5 since right now we have a terrible mish mash of
> > > bin/solr
> > > > > post and bin/post in the ref guide and docs.
> > > > >
> > > > > Could someone review https://github.com/apache/solr/pull/2227 and
> if
> > > it
> > > > > looks good could we sneak it into 9.5?
> > > > >
> > > > > Eric
> > > > >
> > > > >
> > > > > > On Jan 26, 2024, at 6:29 PM, Eric Pugh <
> > > > ep...@opensourceconnections.com>
> > > > > wrote:
> > > > > >
> > > > > > Backport to branch_9_5 is done.
> > > > > >
> > > > > >
> > > > > >> On Jan 26, 2024, at 1:11 PM, Jason Gerlowski <
> > gerlowsk...@gmail.com
> > > >
> > > > > wrote:
> > > > > >>
> > > > > >> Go ahead and backport on your own!  I'm still waiting on Lucene
> > > 9.9.2,
> > > > > so
> > > > > >> there shouldn't be any branch-contention on my end.
> > > > > >>
> > > > > >> Relatedly, Lucene has their RC1 out there and things look good a
> > day
> > > > or
> > > > > two
> > > > > >> into their VOTE, so with any luck we'll be able to get a Solr
> 9.5
> > RC
> > > > > >> together early next week!
> > > > > >>
> > > > > >> Best,
> > > > > >>
> > > > > >> Jason
> > > > > >>
> > > > > >> On Fri, Jan 26, 2024 at 8:49 AM Eric Pugh 
> > wrote:
> > > > > >>
> > > > > >>> I am about to merge SOLR-17112, and will backport it to
> > branch_9x.
> > > > > Jason,
> > > > > >>> do you backport it over to the branch_9_5 or do I?
> > > > > >>>
> > > > > >>> ERic
> > > > > >>>
> > > > > >>>
> > > > > >>> On 2024/01/23 19:08:10 Jason Gerlowski wrote:
> > > > > > It was hoped SOLR-17112 would make 9.4.1 but it didn't as no
> PR
> > > was
> > > > >  proposed.
> > > > > 
> > > > > > SOLR-17120 [is] nominated for inclusion in the 9.5.0 release
> > > > > 
> > > > >  Those both sound quick and reasonable; they've got a +1 from
> me
> > to
> > > > go
> > > > > >>> into
> > > > >  9.5 (assuming the contributor decides to continue with
> > > SOLR-17112).
> > > > > 
> > > > > > Considering Lucene 9.9.2 is being planned, I think it would
> be
> > > > better
> > > > > >>> to
> > > > > > upgrade Solr to the to-be-released version so users have to
> > deal
> > > > with
> > > > > > fewer upgrade cycles.
> > > > > 
> > > > >  Yeah, that might be best; I hadn't realized we weren't already
> > on
> > > > the
> > > > >  latest Lucene 9.x.  I've created SOLR-17128 to track our
> Lucene
> > > > > upgrade
> > > > >  once 9.9.2 is available.
> > > > > 
> > > > >  Obviously this is a longer delay than some of the tickets
> above,
> > > and
> > > > > will
> > > > >  mean we won't be cutting a Solr RC this week.  We can pick a
> new
> > > > date
> > > > > for
> > > > >  the initial Solr 9.5 RC once Lucene 9.9.2 is available.
> > > > > 
> > > > >  Best,
> > > > > 
> > > > >  Jason
> > > > > 
> > > > >  On Tue, Jan 23, 2024

Re: BackupRepository changes

[Bruno Roustant]

> Isn't the intent to ensure we don't waste time/space creating a
useless backup of something that is, I suppose, already corrupted?

That's right. And I didn't read the code enough; a clear effort has been
put here since the last time I read the code, to make all implementations
consistent to verify the checksum.

Hum, this is annoying for directory-based encryption. The only way becomes
to have an encryption extension for each and all different implementations.
Less clean than a FilterBackupRepository.

Re: BackupRepository changes

[Jason Gerlowski]

> Isn't the intent to ensure we don't waste time/space creating a
> useless backup of something that is, I suppose, already corrupted?

Space is one benefit, yep.

The other reason is to avoid giving users a false sense of security.  A
user would be very frustrated to find out at restore-time that the failsafe
backup they've been relying on is useless.  Better to surface that
information at backup time when the source collection is healthy and the
backup can be retried, etc.


On Tue, Jan 30, 2024 at 10:18 AM Bruno Roustant 
wrote:

> > Isn't the intent to ensure we don't waste time/space creating a
> useless backup of something that is, I suppose, already corrupted?
>
> That's right. And I didn't read the code enough; a clear effort has been
> put here since the last time I read the code, to make all implementations
> consistent to verify the checksum.
>
> Hum, this is annoying for directory-based encryption. The only way becomes
> to have an encryption extension for each and all different implementations.
> Less clean than a FilterBackupRepository.
>

Re: [dev help wanted] /admin/segments handler: expose the term count

[Rahul Goswami]

Submitted PR: https://github.com/apache/solr/pull/2233

Open to further discussion on the JIRA/PR if more fields need to be added
to the handler output.


On Mon, Jan 29, 2024 at 8:03 AM Christine Poerschke (BLOOMBERG/ LONDON) <
cpoersc...@bloomberg.net> wrote:

> Wonderful. Please feel free to directly open a pull request or draft pull
> request, or if you prefer first leave an "i'm working on this" style
> comment and/or ask any questions on the JIRA issue. Thanks!
>
> From: us...@solr.apache.org At: 01/26/24 19:34:36 UTCTo:
> us...@solr.apache.org
> Cc:  dev@solr.apache.org
> Subject: Re: [dev help wanted] /admin/segments handler: expose the term
> count
>
> I would love to take this up.
>
> On Fri, Jan 26, 2024 at 6:46 AM Christine Poerschke (BLOOMBERG/ LONDON) <
> cpoersc...@bloomberg.net> wrote:
>
> > Hi Everyone,
> >
> > Have you used or are you curious about the segments info handler and/or
> > screen?
> >
>
> https://solr.apache.org/guide/solr/latest/configuration-guide/index-segments-mer
> ging.html#segments-info-screen
> 
> >
> > If so then would you be interested in contributing to the
> > https://issues.apache.org/jira/browse/SOLR-17038 issue?
> >
> > Thanks,
> > Christine
> >
> >
>
>
>

Re: New branch and feature freeze for Solr 9.5.0

[Christine Poerschke (BLOOMBERG/ LONDON)]

Unrelated to baking times, I stumbled across a "unreferenced files under 
license folder" mystery: 
https://github.com/apache/solr/pull/2178#issuecomment-1917513343

Could someone try to reproduce the "running clean once and then precommit 
twice" sequence?

Thanks,
Christine

From: dev@solr.apache.org At: 01/30/24 15:15:30 UTCTo:  dev@solr.apache.org
Subject: Re: New branch and feature freeze for Solr 9.5.0

Maybe?  Obviously that was my initial plan as I mentioned above.  But I was
pleasantly surprised to see the number of folks that chimed in with their
own test results and +1's on Nazerke's Lucene-upgrade PR.  It seemed
comparable to the data points we'd get out of a multi-day "bake" in
Jenkins.  So I figured we might expedite the baking, or at least let it
"bake" in parallel while I work on the RC.

But I can also wait an additional few days if folks think it's
necessary/valuable?

(Also worth mentioning that this is still theoretical at this point - we're
still waiting on a fix for SOLR-17068 unless Eric changes his mind about
getting it in 9.5.)

On Tue, Jan 30, 2024 at 9:32 AM Ishan Chattopadhyaya <
ichattopadhy...@gmail.com> wrote:

> Should we wait at least 2-3 days for the Lucene 9.9.2 upgrade to bake
> before we cut a release?
>
> On Tue, 30 Jan, 2024, 7:50 pm Jason Gerlowski, 
> wrote:
>
> > Yes - I saw that right after my last email.  Thanks for all your work on
> > that Nazerke!  I've merged and backported that PR; we're now on Lucene
> > 9.9.2!
> >
> > As soon as Eric's P #2227 is merged and backported I'll start on our RC!
> >
> > Best,
> >
> > Jason
> >
> > On Tue, Jan 30, 2024 at 7:33 AM Nazerke S  wrote:
> >
> > > Hi Jason,
> > >
> > > fyi, we have an open PR for
> > the
> > > Lucene upgrade (v9.9.2) waiting for review.
> > >
> > > On Tue, Jan 30, 2024 at 6:21 PM Jason Gerlowski  >
> > > wrote:
> > >
> > > > I'm working on the Lucene upgrade now.  That will still need some
> time
> > in
> > > > review (and maybe a day to let tests bake?), so I'm fine if you
> > backport
> > > > SOLR-17068 in that timeframe.  But let's draw the line once the
> Lucene
> > > > 9.9.2 upgrade is in, unless something else urgent comes up.  I'll
> > target
> > > > Wednesday or Thursday for the Solr RC1.
> > > >
> > > > Jason
> > > >
> > > > On Mon, Jan 29, 2024 at 4:15 PM Eric Pugh <
> > > ep...@opensourceconnections.com
> > > > >
> > > > wrote:
> > > >
> > > > > Jason, I tackled SOLR-17068 (the one you reminded me of) and I’d
> love
> > > to
> > > > > get it into 9.5 since right now we have a terrible mish mash of
> > > bin/solr
> > > > > post and bin/post in the ref guide and docs.
> > > > >
> > > > > Could someone review https://github.com/apache/solr/pull/2227 and
> if
> > > it
> > > > > looks good could we sneak it into 9.5?
> > > > >
> > > > > Eric
> > > > >
> > > > >
> > > > > > On Jan 26, 2024, at 6:29 PM, Eric Pugh <
> > > > ep...@opensourceconnections.com>
> > > > > wrote:
> > > > > >
> > > > > > Backport to branch_9_5 is done.
> > > > > >
> > > > > >
> > > > > >> On Jan 26, 2024, at 1:11 PM, Jason Gerlowski <
> > gerlowsk...@gmail.com
> > > >
> > > > > wrote:
> > > > > >>
> > > > > >> Go ahead and backport on your own!  I'm still waiting on Lucene
> > > 9.9.2,
> > > > > so
> > > > > >> there shouldn't be any branch-contention on my end.
> > > > > >>
> > > > > >> Relatedly, Lucene has their RC1 out there and things look good a
> > day
> > > > or
> > > > > two
> > > > > >> into their VOTE, so with any luck we'll be able to get a Solr
> 9.5
> > RC
> > > > > >> together early next week!
> > > > > >>
> > > > > >> Best,
> > > > > >>
> > > > > >> Jason
> > > > > >>
> > > > > >> On Fri, Jan 26, 2024 at 8:49 AM Eric Pugh 
> > wrote:
> > > > > >>
> > > > > >>> I am about to merge SOLR-17112, and will backport it to
> > branch_9x.
> > > > > Jason,
> > > > > >>> do you backport it over to the branch_9_5 or do I?
> > > > > >>>
> > > > > >>> ERic
> > > > > >>>
> > > > > >>>
> > > > > >>> On 2024/01/23 19:08:10 Jason Gerlowski wrote:
> > > > > > It was hoped SOLR-17112 would make 9.4.1 but it didn't as no
> PR
> > > was
> > > > >  proposed.
> > > > > 
> > > > > > SOLR-17120 [is] nominated for inclusion in the 9.5.0 release
> > > > > 
> > > > >  Those both sound quick and reasonable; they've got a +1 from
> me
> > to
> > > > go
> > > > > >>> into
> > > > >  9.5 (assuming the contributor decides to continue with
> > > SOLR-17112).
> > > > > 
> > > > > > Considering Lucene 9.9.2 is being planned, I think it would
> be
> > > > better
> > > > > >>> to
> > > > > > upgrade Solr to the to-be-released version so users have to
> > deal
> > > > with
> > > > > > fewer upgrade cycles.
> > > > > 
> > > > >  Yeah, that might be best; I hadn't realized we weren't already
> > on
> > > > the
> > > > >  latest Lucene 9.x.  I've created SOLR-17128 to track our
> Lucene
> > > > > upgrade
> > > > >  once 9.9.2 is available.
> > > > > >

Re: New branch and feature freeze for Solr 9.5.0

[Kevin Risden]

Christine - David brought this up a few months ago -
https://lists.apache.org/thread/hlh1bmtgnmp55q8knhjtltf8t57pbl5q

Kevin Risden


On Tue, Jan 30, 2024 at 1:07 PM Christine Poerschke (BLOOMBERG/ LONDON) <
cpoersc...@bloomberg.net> wrote:

> Unrelated to baking times, I stumbled across a "unreferenced files under
> license folder" mystery:
> https://github.com/apache/solr/pull/2178#issuecomment-1917513343
>
> Could someone try to reproduce the "running clean once and then precommit
> twice" sequence?
>
> Thanks,
> Christine
>
> From: dev@solr.apache.org At: 01/30/24 15:15:30 UTCTo:
> dev@solr.apache.org
> Subject: Re: New branch and feature freeze for Solr 9.5.0
>
> Maybe?  Obviously that was my initial plan as I mentioned above.  But I was
> pleasantly surprised to see the number of folks that chimed in with their
> own test results and +1's on Nazerke's Lucene-upgrade PR.  It seemed
> comparable to the data points we'd get out of a multi-day "bake" in
> Jenkins.  So I figured we might expedite the baking, or at least let it
> "bake" in parallel while I work on the RC.
>
> But I can also wait an additional few days if folks think it's
> necessary/valuable?
>
> (Also worth mentioning that this is still theoretical at this point - we're
> still waiting on a fix for SOLR-17068 unless Eric changes his mind about
> getting it in 9.5.)
>
> On Tue, Jan 30, 2024 at 9:32 AM Ishan Chattopadhyaya <
> ichattopadhy...@gmail.com> wrote:
>
> > Should we wait at least 2-3 days for the Lucene 9.9.2 upgrade to bake
> > before we cut a release?
> >
> > On Tue, 30 Jan, 2024, 7:50 pm Jason Gerlowski, 
> > wrote:
> >
> > > Yes - I saw that right after my last email.  Thanks for all your work
> on
> > > that Nazerke!  I've merged and backported that PR; we're now on Lucene
> > > 9.9.2!
> > >
> > > As soon as Eric's P #2227 is merged and backported I'll start on our
> RC!
> > >
> > > Best,
> > >
> > > Jason
> > >
> > > On Tue, Jan 30, 2024 at 7:33 AM Nazerke S 
> wrote:
> > >
> > > > Hi Jason,
> > > >
> > > > fyi, we have an open PR  >for
> > > the
> > > > Lucene upgrade (v9.9.2) waiting for review.
> > > >
> > > > On Tue, Jan 30, 2024 at 6:21 PM Jason Gerlowski <
> gerlowsk...@gmail.com
> > >
> > > > wrote:
> > > >
> > > > > I'm working on the Lucene upgrade now.  That will still need some
> > time
> > > in
> > > > > review (and maybe a day to let tests bake?), so I'm fine if you
> > > backport
> > > > > SOLR-17068 in that timeframe.  But let's draw the line once the
> > Lucene
> > > > > 9.9.2 upgrade is in, unless something else urgent comes up.  I'll
> > > target
> > > > > Wednesday or Thursday for the Solr RC1.
> > > > >
> > > > > Jason
> > > > >
> > > > > On Mon, Jan 29, 2024 at 4:15 PM Eric Pugh <
> > > > ep...@opensourceconnections.com
> > > > > >
> > > > > wrote:
> > > > >
> > > > > > Jason, I tackled SOLR-17068 (the one you reminded me of) and I’d
> > love
> > > > to
> > > > > > get it into 9.5 since right now we have a terrible mish mash of
> > > > bin/solr
> > > > > > post and bin/post in the ref guide and docs.
> > > > > >
> > > > > > Could someone review https://github.com/apache/solr/pull/2227
> and
> > if
> > > > it
> > > > > > looks good could we sneak it into 9.5?
> > > > > >
> > > > > > Eric
> > > > > >
> > > > > >
> > > > > > > On Jan 26, 2024, at 6:29 PM, Eric Pugh <
> > > > > ep...@opensourceconnections.com>
> > > > > > wrote:
> > > > > > >
> > > > > > > Backport to branch_9_5 is done.
> > > > > > >
> > > > > > >
> > > > > > >> On Jan 26, 2024, at 1:11 PM, Jason Gerlowski <
> > > gerlowsk...@gmail.com
> > > > >
> > > > > > wrote:
> > > > > > >>
> > > > > > >> Go ahead and backport on your own!  I'm still waiting on
> Lucene
> > > > 9.9.2,
> > > > > > so
> > > > > > >> there shouldn't be any branch-contention on my end.
> > > > > > >>
> > > > > > >> Relatedly, Lucene has their RC1 out there and things look
> good a
> > > day
> > > > > or
> > > > > > two
> > > > > > >> into their VOTE, so with any luck we'll be able to get a Solr
> > 9.5
> > > RC
> > > > > > >> together early next week!
> > > > > > >>
> > > > > > >> Best,
> > > > > > >>
> > > > > > >> Jason
> > > > > > >>
> > > > > > >> On Fri, Jan 26, 2024 at 8:49 AM Eric Pugh 
> > > wrote:
> > > > > > >>
> > > > > > >>> I am about to merge SOLR-17112, and will backport it to
> > > branch_9x.
> > > > > > Jason,
> > > > > > >>> do you backport it over to the branch_9_5 or do I?
> > > > > > >>>
> > > > > > >>> ERic
> > > > > > >>>
> > > > > > >>>
> > > > > > >>> On 2024/01/23 19:08:10 Jason Gerlowski wrote:
> > > > > > > It was hoped SOLR-17112 would make 9.4.1 but it didn't as
> no
> > PR
> > > > was
> > > > > >  proposed.
> > > > > > 
> > > > > > > SOLR-17120 [is] nominated for inclusion in the 9.5.0
> release
> > > > > > 
> > > > > >  Those both sound quick and reasonable; they've got a +1 from
> > me
> > > to
> > > > > go
> > > > > > >>> into
> > > > > >  9.5 (assuming the contributor 

Re: New branch and feature freeze for Solr 9.5.0

[Christine Poerschke (BLOOMBERG/ LONDON)]

Thanks! https://issues.apache.org/jira/browse/SOLR-17142 created for it, not a 
blocker for 9.5 I think.

From: dev@solr.apache.org At: 01/30/24 18:28:23 UTCTo:  dev@solr.apache.org
Subject: Re: New branch and feature freeze for Solr 9.5.0

Christine - David brought this up a few months ago -
https://lists.apache.org/thread/hlh1bmtgnmp55q8knhjtltf8t57pbl5q

Kevin Risden


On Tue, Jan 30, 2024 at 1:07 PM Christine Poerschke (BLOOMBERG/ LONDON) <
cpoersc...@bloomberg.net> wrote:

> Unrelated to baking times, I stumbled across a "unreferenced files under
> license folder" mystery:
> https://github.com/apache/solr/pull/2178#issuecomment-1917513343
>
> Could someone try to reproduce the "running clean once and then precommit
> twice" sequence?
>
> Thanks,
> Christine
>
> From: dev@solr.apache.org At: 01/30/24 15:15:30 UTCTo:
> dev@solr.apache.org
> Subject: Re: New branch and feature freeze for Solr 9.5.0
>
> Maybe?  Obviously that was my initial plan as I mentioned above.  But I was
> pleasantly surprised to see the number of folks that chimed in with their
> own test results and +1's on Nazerke's Lucene-upgrade PR.  It seemed
> comparable to the data points we'd get out of a multi-day "bake" in
> Jenkins.  So I figured we might expedite the baking, or at least let it
> "bake" in parallel while I work on the RC.
>
> But I can also wait an additional few days if folks think it's
> necessary/valuable?
>
> (Also worth mentioning that this is still theoretical at this point - we're
> still waiting on a fix for SOLR-17068 unless Eric changes his mind about
> getting it in 9.5.)
>
> On Tue, Jan 30, 2024 at 9:32 AM Ishan Chattopadhyaya <
> ichattopadhy...@gmail.com> wrote:
>
> > Should we wait at least 2-3 days for the Lucene 9.9.2 upgrade to bake
> > before we cut a release?
> >
> > On Tue, 30 Jan, 2024, 7:50 pm Jason Gerlowski, 
> > wrote:
> >
> > > Yes - I saw that right after my last email.  Thanks for all your work
> on
> > > that Nazerke!  I've merged and backported that PR; we're now on Lucene
> > > 9.9.2!
> > >
> > > As soon as Eric's P #2227 is merged and backported I'll start on our
> RC!
> > >
> > > Best,
> > >
> > > Jason
> > >
> > > On Tue, Jan 30, 2024 at 7:33 AM Nazerke S 
> wrote:
> > >
> > > > Hi Jason,
> > > >
> > > > fyi, we have an open PR  >for
> > > the
> > > > Lucene upgrade (v9.9.2) waiting for review.
> > > >
> > > > On Tue, Jan 30, 2024 at 6:21 PM Jason Gerlowski <
> gerlowsk...@gmail.com
> > >
> > > > wrote:
> > > >
> > > > > I'm working on the Lucene upgrade now.  That will still need some
> > time
> > > in
> > > > > review (and maybe a day to let tests bake?), so I'm fine if you
> > > backport
> > > > > SOLR-17068 in that timeframe.  But let's draw the line once the
> > Lucene
> > > > > 9.9.2 upgrade is in, unless something else urgent comes up.  I'll
> > > target
> > > > > Wednesday or Thursday for the Solr RC1.
> > > > >
> > > > > Jason
> > > > >
> > > > > On Mon, Jan 29, 2024 at 4:15 PM Eric Pugh <
> > > > ep...@opensourceconnections.com
> > > > > >
> > > > > wrote:
> > > > >
> > > > > > Jason, I tackled SOLR-17068 (the one you reminded me of) and I’d
> > love
> > > > to
> > > > > > get it into 9.5 since right now we have a terrible mish mash of
> > > > bin/solr
> > > > > > post and bin/post in the ref guide and docs.
> > > > > >
> > > > > > Could someone review https://github.com/apache/solr/pull/2227
> and
> > if
> > > > it
> > > > > > looks good could we sneak it into 9.5?
> > > > > >
> > > > > > Eric
> > > > > >
> > > > > >
> > > > > > > On Jan 26, 2024, at 6:29 PM, Eric Pugh <
> > > > > ep...@opensourceconnections.com>
> > > > > > wrote:
> > > > > > >
> > > > > > > Backport to branch_9_5 is done.
> > > > > > >
> > > > > > >
> > > > > > >> On Jan 26, 2024, at 1:11 PM, Jason Gerlowski <
> > > gerlowsk...@gmail.com
> > > > >
> > > > > > wrote:
> > > > > > >>
> > > > > > >> Go ahead and backport on your own!  I'm still waiting on
> Lucene
> > > > 9.9.2,
> > > > > > so
> > > > > > >> there shouldn't be any branch-contention on my end.
> > > > > > >>
> > > > > > >> Relatedly, Lucene has their RC1 out there and things look
> good a
> > > day
> > > > > or
> > > > > > two
> > > > > > >> into their VOTE, so with any luck we'll be able to get a Solr
> > 9.5
> > > RC
> > > > > > >> together early next week!
> > > > > > >>
> > > > > > >> Best,
> > > > > > >>
> > > > > > >> Jason
> > > > > > >>
> > > > > > >> On Fri, Jan 26, 2024 at 8:49 AM Eric Pugh 
> > > wrote:
> > > > > > >>
> > > > > > >>> I am about to merge SOLR-17112, and will backport it to
> > > branch_9x.
> > > > > > Jason,
> > > > > > >>> do you backport it over to the branch_9_5 or do I?
> > > > > > >>>
> > > > > > >>> ERic
> > > > > > >>>
> > > > > > >>>
> > > > > > >>> On 2024/01/23 19:08:10 Jason Gerlowski wrote:
> > > > > > > It was hoped SOLR-17112 would make 9.4.1 but it didn't as
> no
> > PR
> > > > was
> > > > > >  proposed.
> > > > > > 
> > > > > > > SOLR-17120 [is

Re: Unreferenced license file detection

[Christine Poerschke (BLOOMBERG/ LONDON)]

Stumbled across the same issue and Kevin pointed me to this thread, thanks!

https://issues.apache.org/jira/browse/SOLR-17142 created for further looking 
into.

From: dev@solr.apache.org At: 12/08/23 04:11:15 UTCTo:  dev@solr.apache.org
Subject: Re: Unreferenced license file detection

Thanks Kevin.  Maybe one of us will look into this further someday.

~ David


On Wed, Dec 6, 2023 at 11:00 AM Kevin Risden  wrote:

> I think it came in as part of https://github.com/apache/solr/pull/1319
>
> It has been this way for a while if you don't do a clean/full build.
>
> At first I thought we could remove the licenses, but they are still needed
> just because they aren't referenced with whatever steps were run.
>
> Kevin Risden
>
>
> On Tue, Dec 5, 2023 at 6:54 PM David Smiley  wrote:
>
> > I often find that after running "gradle check", I get the following
> warning
> > at the very end, right after the slow tests are printed:
> >
> > WARNING: there were unreferenced files under license folder:
> >   -
> > /Users/dsmiley/SFDCDev/solr_9x/solr/licenses/HdrHistogram-LICENSE-PD.txt
> >   - /Users/dsmiley/SFDCDev/solr_9x/solr/licenses/HdrHistogram-NOTICE.txt
> >   -
> > /Users/dsmiley/SFDCDev/solr_9x/solr/licenses/LatencyUtils-LICENSE-PD.txt
> >   - /Users/dsmiley/SFDCDev/solr_9x/solr/licenses/LatencyUtils-NOTICE.txt
> >   -
> > /Users/dsmiley/SFDCDev/solr_9x/solr/licenses/SparseBitSet-LICENSE-ASL.txt
> >   - /Users/dsmiley/SFDCDev/solr_9x/solr/licenses/SparseBitSet-NOTICE.txt
> > etc.
> >
> > Basically every *.txt file that exists in this folder, which is ~567 of
> > them.  Each is dumped to the console.  Is this happening for others?  Has
> > anyone dug into why this is occurring?
> >
> > ~ David Smiley
> > Apache Lucene/Solr Search Developer
> > http://www.linkedin.com/in/davidwsmiley
> >
>

CLI: bin/solr bin/post bin/postlogs -- what's happening

[David Smiley]

I'm writing this to get input on where we're going in the CLI domain
with respect to organizational choices of our commands.  Looking on
9x, I see bin/solr, bin/post, bin/postlogs scripts.  Recently, most
internal command plumbing has been centralized under bin/solr and thus
you can do "bin/solr post" or "bin/solr postlogs" instead of
"bin/post" and "bin/postlogs" respectively.  Disclaimer:  So I hear; I
haven't tried them.

At this point, I think we have a choice:
(A) Remove bin/post and bin/postlogs in 10.  There's no question the
code duplication should be eliminated but the question is really about
the DX (developer user experience).  Do we want one shell command,
"bin/solr" to be all things Solr, not just Solr itself (starting
Solr), but posting data to Solr (basically a Curl alternative)?  It's
already a kitchen sink of some miscellaneous things, granted.  This
annoys my organizational sensibilities.
(B) Keep the scripts, but implement them as one-liners calling into
"bin/solr post" or similar, and possibly not document on the bin/solr
side that these commands are there as it's just for implementation
convenience.  After all, this was the actual motivation of the changes
that have happened lately -- it's improving code maintenance/support.
Good stuff; but do we need to change the DX on users too?  Is this
better for them?

Separately, since there are so many commands in bin/solr that don't
relate to starting Solr, maybe bin/solr-start should exist?  (again,
could be a one-liner call into one CLI backend for our convenience).

This decision is a matter of taste; it's not really technical.

~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: BackupRepository changes

[David Smiley]

I could imagine making it toggle-able if it serves the index
encryption feature.  Dunno if that's easy, hard, or impossible.

On Tue, Jan 30, 2024 at 10:31 AM Jason Gerlowski  wrote:
>
> > Isn't the intent to ensure we don't waste time/space creating a
> > useless backup of something that is, I suppose, already corrupted?
>
> Space is one benefit, yep.
>
> The other reason is to avoid giving users a false sense of security.  A
> user would be very frustrated to find out at restore-time that the failsafe
> backup they've been relying on is useless.  Better to surface that
> information at backup time when the source collection is healthy and the
> backup can be retried, etc.
>
>
> On Tue, Jan 30, 2024 at 10:18 AM Bruno Roustant 
> wrote:
>
> > > Isn't the intent to ensure we don't waste time/space creating a
> > useless backup of something that is, I suppose, already corrupted?
> >
> > That's right. And I didn't read the code enough; a clear effort has been
> > put here since the last time I read the code, to make all implementations
> > consistent to verify the checksum.
> >
> > Hum, this is annoying for directory-based encryption. The only way becomes
> > to have an encryption extension for each and all different implementations.
> > Less clean than a FilterBackupRepository.
> >

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: CLI: bin/solr bin/post bin/postlogs -- what's happening

[Eric Pugh]

Thanks David for weighing in.   A big part of pushing the bin/post and 
bin/postlogs into bin/solr as subcommands is that then the sub command lives in 
Java land, and we just “magically” picked up Windows support.   

It feels like right now we are in the worst of all worlds..  We have a mix of 
bin/{somescript} that is each implemented slightly different, and then a lot of 
nested commands under bin/solr, arguably some which should be their own 
commands?   Do we then move to bin/create, bin/delete, bin/zk, bin/export, 
bin/api ?  

I’m also thinking of a future where we have more CLI support, for example a CLI 
for running a streaming expression.   Or a CLI for all the various commands 
that the V2 API’s are exposing, and that would be nice to have be scripted via 
a CLI.   So bin/split-shard?   

It may be that we have a pattern in the future like “bin/server” and 
“bin/client” or maybe you have “bin/solr api split-shard” and “bin/solr server 
start”, and then commands get grouped like that?

I sometimes dream of having a separate “Solr-cli” sub project that has CLI that 
uses tooling like https://oclif.io/ to build something really amazing ;-).   

I know someone is going to say “can’t we just use Curl”, and that may work for 
some folks, but doing a lot of interactions with Solr require multiple steps, 
and that’s where the CLI shines.   Add in supporting Basic auth and someday 
oAuth, and Curl starts to break down.


> On Jan 30, 2024, at 3:10 PM, David Smiley  wrote:
> 
> I'm writing this to get input on where we're going in the CLI domain
> with respect to organizational choices of our commands.  Looking on
> 9x, I see bin/solr, bin/post, bin/postlogs scripts.  Recently, most
> internal command plumbing has been centralized under bin/solr and thus
> you can do "bin/solr post" or "bin/solr postlogs" instead of
> "bin/post" and "bin/postlogs" respectively.  Disclaimer:  So I hear; I
> haven't tried them.
> 
> At this point, I think we have a choice:
> (A) Remove bin/post and bin/postlogs in 10.  There's no question the
> code duplication should be eliminated but the question is really about
> the DX (developer user experience).  Do we want one shell command,
> "bin/solr" to be all things Solr, not just Solr itself (starting
> Solr), but posting data to Solr (basically a Curl alternative)?  It's
> already a kitchen sink of some miscellaneous things, granted.  This
> annoys my organizational sensibilities.
> (B) Keep the scripts, but implement them as one-liners calling into
> "bin/solr post" or similar, and possibly not document on the bin/solr
> side that these commands are there as it's just for implementation
> convenience.  After all, this was the actual motivation of the changes
> that have happened lately -- it's improving code maintenance/support.
> Good stuff; but do we need to change the DX on users too?  Is this
> better for them?
> 
> Separately, since there are so many commands in bin/solr that don't
> relate to starting Solr, maybe bin/solr-start should exist?  (again,
> could be a one-liner call into one CLI backend for our convenience).
> 
> This decision is a matter of taste; it's not really technical.
> 
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> http://www.linkedin.com/in/davidwsmiley
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> For additional commands, e-mail: dev-h...@solr.apache.org
> 

___
Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | 
http://www.opensourceconnections.com  | 
My Free/Busy   
Co-Author: Apache Solr Enterprise Search Server, 3rd Ed 


This e-mail and all contents, including attachments, is considered to be 
Company Confidential unless explicitly stated otherwise, regardless of whether 
attachments are marked as such.

Re: CLI: bin/solr bin/post bin/postlogs -- what's happening

[Jason Gerlowski]

For "post" specifically, I think option (A) is better from both a developer
and a user-interface perspective.

On the development side, "bin/solr" integration gets us a lot: it's easy to
hook new tools/subcommands into bin/solr, the logic can live in Java,
support for various features (e.g. basic-auth) comes "for free".

On the user-interface side: "bin/solr" is the only option that's available
to Windows users.  That's a huge benefit.  Even if we organized the code as
described in option (B) above, I think we'd cause a lot of confusion
requiring Linux and Windows users to execute different scripts (i.e.
"bin/post" on Linux/Mac and "bin/solr post" on Windows).

Also from a user-interface perspective - I personally don't mind having
many sub-commands accessible through one main entrypoint.  It seems
relatively common and well received nowadays - take "kubectl" and
"aws-shell", by way of example.

Best,

Jason

On Tue, Jan 30, 2024 at 5:40 PM Eric Pugh 
wrote:

> Thanks David for weighing in.   A big part of pushing the bin/post and
> bin/postlogs into bin/solr as subcommands is that then the sub command
> lives in Java land, and we just “magically” picked up Windows support.
>
> It feels like right now we are in the worst of all worlds..  We have a mix
> of bin/{somescript} that is each implemented slightly different, and then a
> lot of nested commands under bin/solr, arguably some which should be their
> own commands?   Do we then move to bin/create, bin/delete, bin/zk,
> bin/export, bin/api ?
>
> I’m also thinking of a future where we have more CLI support, for example
> a CLI for running a streaming expression.   Or a CLI for all the various
> commands that the V2 API’s are exposing, and that would be nice to have be
> scripted via a CLI.   So bin/split-shard?
>
> It may be that we have a pattern in the future like “bin/server” and
> “bin/client” or maybe you have “bin/solr api split-shard” and “bin/solr
> server start”, and then commands get grouped like that?
>
> I sometimes dream of having a separate “Solr-cli” sub project that has CLI
> that uses tooling like https://oclif.io/ to build something really
> amazing ;-).
>
> I know someone is going to say “can’t we just use Curl”, and that may work
> for some folks, but doing a lot of interactions with Solr require multiple
> steps, and that’s where the CLI shines.   Add in supporting Basic auth and
> someday oAuth, and Curl starts to break down.
>
>
> > On Jan 30, 2024, at 3:10 PM, David Smiley  wrote:
> >
> > I'm writing this to get input on where we're going in the CLI domain
> > with respect to organizational choices of our commands.  Looking on
> > 9x, I see bin/solr, bin/post, bin/postlogs scripts.  Recently, most
> > internal command plumbing has been centralized under bin/solr and thus
> > you can do "bin/solr post" or "bin/solr postlogs" instead of
> > "bin/post" and "bin/postlogs" respectively.  Disclaimer:  So I hear; I
> > haven't tried them.
> >
> > At this point, I think we have a choice:
> > (A) Remove bin/post and bin/postlogs in 10.  There's no question the
> > code duplication should be eliminated but the question is really about
> > the DX (developer user experience).  Do we want one shell command,
> > "bin/solr" to be all things Solr, not just Solr itself (starting
> > Solr), but posting data to Solr (basically a Curl alternative)?  It's
> > already a kitchen sink of some miscellaneous things, granted.  This
> > annoys my organizational sensibilities.
> > (B) Keep the scripts, but implement them as one-liners calling into
> > "bin/solr post" or similar, and possibly not document on the bin/solr
> > side that these commands are there as it's just for implementation
> > convenience.  After all, this was the actual motivation of the changes
> > that have happened lately -- it's improving code maintenance/support.
> > Good stuff; but do we need to change the DX on users too?  Is this
> > better for them?
> >
> > Separately, since there are so many commands in bin/solr that don't
> > relate to starting Solr, maybe bin/solr-start should exist?  (again,
> > could be a one-liner call into one CLI backend for our convenience).
> >
> > This decision is a matter of taste; it's not really technical.
> >
> > ~ David Smiley
> > Apache Lucene/Solr Search Developer
> > http://www.linkedin.com/in/davidwsmiley
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> > For additional commands, e-mail: dev-h...@solr.apache.org
> >
>
> ___
> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 |
> http://www.opensourceconnections.com <
> http://www.opensourceconnections.com/> | My Free/Busy <
> http://tinyurl.com/eric-cal>
> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <
> https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw>
>
> This e-mail and all contents, incl

Re: CLI: bin/solr bin/post bin/postlogs -- what's happening

[David Smiley]

If "B" were to be done, there would similarly need to be a Windows
redirect script (a one or two line thing) following suit with the Unix
side.

I'm not disagreeing with having the internal plumbing redirection that
has already occurred -- would not change in "B".

Any way -- fine.  Not a big deal.

On Tue, Jan 30, 2024 at 10:12 PM Jason Gerlowski  wrote:
>
> For "post" specifically, I think option (A) is better from both a developer
> and a user-interface perspective.
>
> On the development side, "bin/solr" integration gets us a lot: it's easy to
> hook new tools/subcommands into bin/solr, the logic can live in Java,
> support for various features (e.g. basic-auth) comes "for free".
>
> On the user-interface side: "bin/solr" is the only option that's available
> to Windows users.  That's a huge benefit.  Even if we organized the code as
> described in option (B) above, I think we'd cause a lot of confusion
> requiring Linux and Windows users to execute different scripts (i.e.
> "bin/post" on Linux/Mac and "bin/solr post" on Windows).
>
> Also from a user-interface perspective - I personally don't mind having
> many sub-commands accessible through one main entrypoint.  It seems
> relatively common and well received nowadays - take "kubectl" and
> "aws-shell", by way of example.
>
> Best,
>
> Jason
>
> On Tue, Jan 30, 2024 at 5:40 PM Eric Pugh 
> wrote:
>
> > Thanks David for weighing in.   A big part of pushing the bin/post and
> > bin/postlogs into bin/solr as subcommands is that then the sub command
> > lives in Java land, and we just “magically” picked up Windows support.
> >
> > It feels like right now we are in the worst of all worlds..  We have a mix
> > of bin/{somescript} that is each implemented slightly different, and then a
> > lot of nested commands under bin/solr, arguably some which should be their
> > own commands?   Do we then move to bin/create, bin/delete, bin/zk,
> > bin/export, bin/api ?
> >
> > I’m also thinking of a future where we have more CLI support, for example
> > a CLI for running a streaming expression.   Or a CLI for all the various
> > commands that the V2 API’s are exposing, and that would be nice to have be
> > scripted via a CLI.   So bin/split-shard?
> >
> > It may be that we have a pattern in the future like “bin/server” and
> > “bin/client” or maybe you have “bin/solr api split-shard” and “bin/solr
> > server start”, and then commands get grouped like that?
> >
> > I sometimes dream of having a separate “Solr-cli” sub project that has CLI
> > that uses tooling like https://oclif.io/ to build something really
> > amazing ;-).
> >
> > I know someone is going to say “can’t we just use Curl”, and that may work
> > for some folks, but doing a lot of interactions with Solr require multiple
> > steps, and that’s where the CLI shines.   Add in supporting Basic auth and
> > someday oAuth, and Curl starts to break down.
> >
> >
> > > On Jan 30, 2024, at 3:10 PM, David Smiley  wrote:
> > >
> > > I'm writing this to get input on where we're going in the CLI domain
> > > with respect to organizational choices of our commands.  Looking on
> > > 9x, I see bin/solr, bin/post, bin/postlogs scripts.  Recently, most
> > > internal command plumbing has been centralized under bin/solr and thus
> > > you can do "bin/solr post" or "bin/solr postlogs" instead of
> > > "bin/post" and "bin/postlogs" respectively.  Disclaimer:  So I hear; I
> > > haven't tried them.
> > >
> > > At this point, I think we have a choice:
> > > (A) Remove bin/post and bin/postlogs in 10.  There's no question the
> > > code duplication should be eliminated but the question is really about
> > > the DX (developer user experience).  Do we want one shell command,
> > > "bin/solr" to be all things Solr, not just Solr itself (starting
> > > Solr), but posting data to Solr (basically a Curl alternative)?  It's
> > > already a kitchen sink of some miscellaneous things, granted.  This
> > > annoys my organizational sensibilities.
> > > (B) Keep the scripts, but implement them as one-liners calling into
> > > "bin/solr post" or similar, and possibly not document on the bin/solr
> > > side that these commands are there as it's just for implementation
> > > convenience.  After all, this was the actual motivation of the changes
> > > that have happened lately -- it's improving code maintenance/support.
> > > Good stuff; but do we need to change the DX on users too?  Is this
> > > better for them?
> > >
> > > Separately, since there are so many commands in bin/solr that don't
> > > relate to starting Solr, maybe bin/solr-start should exist?  (again,
> > > could be a one-liner call into one CLI backend for our convenience).
> > >
> > > This decision is a matter of taste; it's not really technical.
> > >
> > > ~ David Smiley
> > > Apache Lucene/Solr Search Developer
> > > http://www.linkedin.com/in/davidwsmiley
> > >
> > > -
> > > To unsubscribe, e-mail: dev-unsubs