Re: BackupRepository changes

[Ilan Ginzburg]

Having used the BackupRepository for other things than backups in the ZERO
branch, I suspect there might be two separate concerns that are mixed into
the same class/abstraction.
One is verifying the data to be persisted is not corrupted (this heavily
depends on the type of data being persisted obviously), and the other one
is doing the actual persisting work.
It makes sense to verify the checksum before taking a backup, but IMO this
doesn't belong in an implementation dealing with S3 or GCS.

Would separating the Backup part of the BackupRepository abstraction from
the Repository part solve the problems you're facing Bruno?

Ilan

On Tue, Jan 30, 2024 at 9:12 PM David Smiley  wrote:

> I could imagine making it toggle-able if it serves the index
> encryption feature.  Dunno if that's easy, hard, or impossible.
>
> On Tue, Jan 30, 2024 at 10:31 AM Jason Gerlowski 
> wrote:
> >
> > > Isn't the intent to ensure we don't waste time/space creating a
> > > useless backup of something that is, I suppose, already corrupted?
> >
> > Space is one benefit, yep.
> >
> > The other reason is to avoid giving users a false sense of security.  A
> > user would be very frustrated to find out at restore-time that the
> failsafe
> > backup they've been relying on is useless.  Better to surface that
> > information at backup time when the source collection is healthy and the
> > backup can be retried, etc.
> >
> >
> > On Tue, Jan 30, 2024 at 10:18 AM Bruno Roustant <
> bruno.roust...@gmail.com>
> > wrote:
> >
> > > > Isn't the intent to ensure we don't waste time/space creating a
> > > useless backup of something that is, I suppose, already corrupted?
> > >
> > > That's right. And I didn't read the code enough; a clear effort has
> been
> > > put here since the last time I read the code, to make all
> implementations
> > > consistent to verify the checksum.
> > >
> > > Hum, this is annoying for directory-based encryption. The only way
> becomes
> > > to have an encryption extension for each and all different
> implementations.
> > > Less clean than a FilterBackupRepository.
> > >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> For additional commands, e-mail: dev-h...@solr.apache.org
>
>

Regarding Solr issue(HTTP 502; : Bad Gateway Curl error)

[Lokulwar, Amit Kamalakar (external - Project)]

Hello Team,

I am working on one of big project. There I am using Apache Solr with Drupal 
CMS. So far that project we used Solr Indexer and Solr Attachment and Solr 
Search as well.
Below are thing that we used in our Project:
Solr: 7.7.3
JDK: 17
Drupal: 7
PHP: 7.4.30

So When we upgrade JDK to 17 then we are getting "HTTP 502; : Bad Gateway Curl 
error". So We tried to resolve this issue done below changes and increase 
timeout.
1.Increase the PHP execution time, when we see in the networking tab it is 
taking a long time to receive the response.
/ariba/community/main_jdk17/config/php.ini.cfg
2.apache config added below lines:
Timeout 2400  ProxyTimeout 2400 ProxyBadHeader Igonore
//ariba/community/main_jdk17/config/httpd.conf.cfg


3. I had changed below file to increase Max Indexing time.
  $max_indexing_time = variable_get('ariba_search_index_max_time_seconds', 
12600);
File path: 
/drupal/sites/all/modules/ariba/ariba_search/drush/ariba_search.drush.inc

But Still above issue is present. So please let us know what we need to do for 
resolve this issue.

I am waiting for your reply. Let me know if you need any more details.

Regards,
Amit Lokulwar
(+91-9028291825)

Overseer, expiring queued messages

[David Smiley]

I have a proposal and am curious what folks think.  When the Overseer
dequeues an admin command message to process, imagine it being
enhanced to examine the "ctime" (creation time) of the ZK message node
to determine how long it has been enqueued, and thus roughly how long
the client has been waiting.  If it's greater than a configured
threshold (1 minute?), respond with an error of a timeout nature.
"Sorry, the Overseer is so backed up that we fear you have given up;
please try again".  This would not apply to an "async" style
submission.

Motivation:  Due to miscellaneous reasons at scale that are very user
/ situation dependent, the Overseer can get seriously backed up.  The
client, making a typical synchronous call to, say, create a
collection, may reach its timeout (say a minute) and has given up.
Today, SolrCloud doesn't know this; it goes on its merry way and
creates a collection anyway.  Depending on how Solr is used, this can
be an orphaned collection that the client doesn't want anymore.  That
is to say, the client wants a collection but it wanted it at the time
it asked for it with the name it asked for at that time.  If it fails,
it will come back later and propose a new name.  This doesn't have to
be collection creation specific; I'm thinking that in principle it
doesn't really matter what the command is.  If Solr takes too long for
the Overseer to receive the message; just timeout, basically.

Thoughts?

This wouldn't be a concern for the distributed mode of collection
processing as there is no queue bottleneck; the receiving node
processes the request immediately.

~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: Regarding Solr issue(HTTP 502; : Bad Gateway Curl error)

[Jan Høydahl]

Hi,

This sounds like a question for the Drupal community. I see nothing we can help 
with here.

Good luck with debugging.

PS: If you in the future have direct questions aboout Apache Solr, such as if 
you identify error messages in Solr's server log, please use the 
us...@solr.apache.org  mailing list instead of 
this developers list.


Jan Høydahl

> 31. jan. 2024 kl. 11:24 skrev Lokulwar, Amit Kamalakar (external - Project) 
> :
> 
> Hello Team,
> 
> I am working on one of big project. There I am using Apache Solr with Drupal 
> CMS. So far that project we used Solr Indexer and Solr Attachment and Solr 
> Search as well.
> Below are thing that we used in our Project:
> Solr: 7.7.3
> JDK: 17
> Drupal: 7
> PHP: 7.4.30
> 
> So When we upgrade JDK to 17 then we are getting "HTTP 502; : Bad Gateway 
> Curl error". So We tried to resolve this issue done below changes and 
> increase timeout.
> 1.Increase the PHP execution time, when we see in the networking tab it is 
> taking a long time to receive the response.
> /ariba/community/main_jdk17/config/php.ini.cfg
> 2.apache config added below lines:
> Timeout 2400  ProxyTimeout 2400 ProxyBadHeader Igonore
> //ariba/community/main_jdk17/config/httpd.conf.cfg
> 
> 
> 3. I had changed below file to increase Max Indexing time.
>  $max_indexing_time = variable_get('ariba_search_index_max_time_seconds', 
> 12600);
> File path: 
> /drupal/sites/all/modules/ariba/ariba_search/drush/ariba_search.drush.inc
> 
> But Still above issue is present. So please let us know what we need to do 
> for resolve this issue.
> 
> I am waiting for your reply. Let me know if you need any more details.
> 
> Regards,
> Amit Lokulwar
> (+91-9028291825)
> 

RE: Regarding Solr issue(HTTP 502; : Bad Gateway Curl error)

[Venu Kumar Merugu]

Thanks @Jan Høydahl for the response.

Hi @Lokulwar, Amit Kamalakar (external - 
Project)
Can you please check with Drupal Community in separate email.

Thanks
Venu
From: Jan Høydahl 
Sent: Wednesday, January 31, 2024 11:40 PM
To: Lokulwar, Amit Kamalakar (external - Project) 

Cc: HANMANT KADAM, KUNAL (external - Project) ; 
Botla, Sowjanya (external - Project) ; Venu Kumar 
Merugu ; dev@solr.apache.org
Subject: Re: Regarding Solr issue(HTTP 502; : Bad Gateway Curl error)

Hi,

This sounds like a question for the Drupal community. I see nothing we can help 
with here.

Good luck with debugging.

PS: If you in the future have direct questions aboout Apache Solr, such as if 
you identify error messages in Solr's server log, please use the 
us...@solr.apache.org mailing list instead of 
this developers list.


Jan Høydahl


31. jan. 2024 kl. 11:24 skrev Lokulwar, Amit Kamalakar (external - Project) 
mailto:amit.kamalakar.lokul...@sap.com.INVALID>>:

Hello Team,

I am working on one of big project. There I am using Apache Solr with Drupal 
CMS. So far that project we used Solr Indexer and Solr Attachment and Solr 
Search as well.
Below are thing that we used in our Project:
Solr: 7.7.3
JDK: 17
Drupal: 7
PHP: 7.4.30

So When we upgrade JDK to 17 then we are getting "HTTP 502; : Bad Gateway Curl 
error". So We tried to resolve this issue done below changes and increase 
timeout.
1.Increase the PHP execution time, when we see in the networking tab it is 
taking a long time to receive the response.
/ariba/community/main_jdk17/config/php.ini.cfg
2.apache config added below lines:
Timeout 2400  ProxyTimeout 2400 ProxyBadHeader Igonore
//ariba/community/main_jdk17/config/httpd.conf.cfg


3. I had changed below file to increase Max Indexing time.
 $max_indexing_time = variable_get('ariba_search_index_max_time_seconds', 
12600);
File path: 
/drupal/sites/all/modules/ariba/ariba_search/drush/ariba_search.drush.inc

But Still above issue is present. So please let us know what we need to do for 
resolve this issue.

I am waiting for your reply. Let me know if you need any more details.

Regards,
Amit Lokulwar
(+91-9028291825)

Caution: Please do not click links or open attachments unless you recognize the 
sender and know the content is safe.


 Disclaimer: This message and the information contained herein is proprietary 
and confidential and subject to the Tech Mahindra policy statement, you may 
review the policy at http://www.techmahindra.com/Disclaimer.html externally 
http://tim.techmahindra.com/tim/disclaimer.html internally within TechMahindra. 

Re: Overseer, expiring queued messages

[6harat]

Thanks David for starting this thread. We have also seen this behavior from
overseer resulting in "orphan collections" or "more than 1 replica created"
due to timeouts especially when our cluster is scaled up during peak
traffic days.
While I am still at a nascent stage of my understanding of solr internals,
I wanted to highlight the below points: (pardon me if these doesn't make
much sense),

1. There may be situations where we want solr to still honor the late
message and hence the functionality needs to be configurable and not a
default. For instance, during decommissioning of boxes (when we are scaling
down to our normal cluster size from peak), we send delete replica commands
for 20+ boxes in a short time frame. Majority of these API hits inevitably
times out, however we rely upon the behaviour that the cluster after X mins
is able to reach to the desired state.

2. How do we intend to communicate the timeout based rejection of overseer
message to the end-user

3. In case of fail-over scenario where the overseer leader node goes down
and is re-elected, the election may have some overhead which may inevitably
result in many of the piled up messages being rejected due to time
constraints. Do we intend to pause the clock ticks during this phase or the
guidance should be to set timeout higher than sum of such possible overheads


On Wed, Jan 31, 2024 at 11:18 PM David Smiley  wrote:

> I have a proposal and am curious what folks think.  When the Overseer
> dequeues an admin command message to process, imagine it being
> enhanced to examine the "ctime" (creation time) of the ZK message node
> to determine how long it has been enqueued, and thus roughly how long
> the client has been waiting.  If it's greater than a configured
> threshold (1 minute?), respond with an error of a timeout nature.
> "Sorry, the Overseer is so backed up that we fear you have given up;
> please try again".  This would not apply to an "async" style
> submission.
>
> Motivation:  Due to miscellaneous reasons at scale that are very user
> / situation dependent, the Overseer can get seriously backed up.  The
> client, making a typical synchronous call to, say, create a
> collection, may reach its timeout (say a minute) and has given up.
> Today, SolrCloud doesn't know this; it goes on its merry way and
> creates a collection anyway.  Depending on how Solr is used, this can
> be an orphaned collection that the client doesn't want anymore.  That
> is to say, the client wants a collection but it wanted it at the time
> it asked for it with the name it asked for at that time.  If it fails,
> it will come back later and propose a new name.  This doesn't have to
> be collection creation specific; I'm thinking that in principle it
> doesn't really matter what the command is.  If Solr takes too long for
> the Overseer to receive the message; just timeout, basically.
>
> Thoughts?
>
> This wouldn't be a concern for the distributed mode of collection
> processing as there is no queue bottleneck; the receiving node
> processes the request immediately.
>
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> http://www.linkedin.com/in/davidwsmiley
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> For additional commands, e-mail: dev-h...@solr.apache.org
>
>

-- 
Regards
6harat