Re: SolrCloud keeps crashing

Oops, I should have referenced this document rather: https://www.tenable.com/cve/CVE-2019-17558 On 2/3/21 2:42 PM, TK Solr wrote: Victor & Satish, Is your Solr accessible from the Internet by anyone? If so, your site is being attacked by a bot us

Re: SolrCloud keeps crashing

Victor & Satish, Is your Solr accessible from the Internet by anyone? If so, your site is being attacked by a bot using this security hole: https://www.tenable.com/blog/cve-2019-17558-apache-solr-vulnerable-to-remote-code-execution-zero-day-vulnerability If that is the case, try blocking the

Re: SolrCloud keeps crashing

I am facing the same issue. Did u find any solution for this? -- Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Solrcloud - Reads on specific nodes

On 1/17/2021 11:12 PM, Doss wrote: Thanks Michael Gibney , Shawn Heisey for pointing in the right direction. 1. Will there be any performance degrade if we use shards.preference? 2. How about leader election if we decided to use NRT + PULL ? TLOG has the advantage of participating in leader elec

Re: Solrcloud - Reads on specific nodes

Thanks Michael Gibney , Shawn Heisey for pointing in the right direction. 1. Will there be any performance degrade if we use shards.preference? 2. How about leader election if we decided to use NRT + PULL ? TLOG has the advantage of participating in leader election correct? 3. NRT + TLOG is there

Re: Solrcloud - Reads on specific nodes

On 1/15/2021 7:56 AM, Doss wrote: 1. Suppose we have 10 node SOLR Cloud setup, is it possible to dedicate 4 nodes for writes and 6 nodes for selects? 2. We have a SOLR cloud setup for our customer facing applications, and we would like to have two more SOLR nodes for some backend jobs. Is it goo

Re: Solrcloud - Reads on specific nodes

I know you're asking about nodes, not replicas; but depending on what you're trying to achieve you might be as well off routing requests based on replica. Have you considered the various options available via the `shards.preference` param [1]? For instance, you could set up your "write" replicas as

Re: solrcloud with EKS kubernetes

Hi Jonathan, it was really helpful. Some of the metrics were crossing threshold like network bandwidth etc. Regards, Abhishek On Sat, Dec 26, 2020 at 7:54 PM Jonathan Tan wrote: > Hi Abhishek, > > Merry Christmas to you too! > I think it's really a question regarding your indexing speed NFRs. >

Re: Solrcloud load balancing / failover

Hi, Thank you for your response. Dominique Le mar. 15 déc. 2020 à 08:06, Shalin Shekhar Mangar a écrit : > No, the load balancing is based on random selection of replicas and > CPU is not consulted. There are limited ways to influence the replica > selection, see > https://lucene.apache.org/solr

Re: solrcloud with EKS kubernetes

Hi Abhishek, Merry Christmas to you too! I think it's really a question regarding your indexing speed NFRs. Have you had a chance to take a look at your IOPS & write bytes/second graphs for that host & PVC? I'd suggest that's the first thing to go look at, so that you can find out whether you're

Re: solrcloud with EKS kubernetes

Hi Jonathan, Merry Christmas. Thanks for the suggestion. To manage IOPS can we do something on rate-limiting behalf? Regards, Abhishek On Thu, Dec 17, 2020 at 5:07 AM Jonathan Tan wrote: > Hi Abhishek, > > We're running Solr Cloud 8.6 on GKE. > 3 node cluster, running 4 cpus (configured) and 8

Re: solrCloud client socketTimeout initiates retries

Hi erick, Thanks. Yes we will be upgrading soon to 8.8 till we upgrade we are increasing socket timeout and it helps for time being to some extent. regards, kshitij On Fri, Dec 18, 2020 at 7:48 PM Erick Erickson wrote: > Right, there are several alternatives. Try going here: > http://jirasearc

Re: solrCloud client socketTimeout initiates retries

Right, there are several alternatives. Try going here: http://jirasearch.mikemccandless.com/search.py?index=jira and search for “circuit breaker” and you’ll find a bunch of JIRAs. Unfortunately, some are in 8.8.. That said, some of the circuit breakers are in much earlier releases. Would it suffi

Re: solrCloud client socketTimeout initiates retries

Hi Erick, I agree but in a huge cluster the retries keeps on happening, cant we have this feature implemented in client. i was referring to this jira https://issues.apache.org/jira/browse/SOLR-10479 We have seen that some malicious queries come to system which takes significant time and these que

Re: solrCloud client socketTimeout initiates retries

Why do you want to do this? This sounds like an XY problem, you think you’re going to solve some problem X by doing Y. Y in this case is setting the numServersToTry, but you haven’t explained what X, the problem you’re trying to solve is. Offhand, this seems like a terrible idea. If you’re request

Re: solrcloud with EKS kubernetes

Hi Abhishek, We're running Solr Cloud 8.6 on GKE. 3 node cluster, running 4 cpus (configured) and 8gb of min & max JVM configured, all with anti-affinity so they never exist on the same node. It's got 2 collections of ~13documents each, 6 shards, 3 replicas each, disk usage on each node is ~54gb (

Re: Solrcloud load balancing / failover

No, the load balancing is based on random selection of replicas and CPU is not consulted. There are limited ways to influence the replica selection, see https://lucene.apache.org/solr/guide/8_4/distributed-requests.html#shards-preference-parameter If a replica fails then the query fails and an er

Re: solrcloud with EKS kubernetes

FWIW, I have seen Solr exhaust the IOPS burst quota on AWS causing slow replication and high latency for search and indexing operations. You may want to dig into cloud watch metrics and see if you are running into a similar issue. The default IOPS quota on gp2 is very low (100?). Another thing to

Re: solrcloud with EKS kubernetes

Hi Houston, Sorry for the late reply. Each shard has a 9GB size around. Yeah, we are providing enough resources to pods. We are currently using c5.4xlarge. XMS and XMX is 16GB. The machine is having 32 GB and 16 core. No, I haven't run it outside Kubernetes. But I do have colleagues who did the sam

Re: SolrCloud crashing due to memory error - 'Cannot allocate memory' (errno=12)

How much RAM do you have on those machines? That message says you ran out. 32 GB is a HUGE heap. Unless you have a specific need for that, run with a 8 GB heap and see how that works. wunder Walter Underwood wun...@wunderwood.org http://observer.wunderwood.org/ (my blog) > On Dec 10, 2020, at

Re: solrcloud with EKS kubernetes

Hello Abhishek, It's really hard to provide any advice without knowing any information about your setup/usage. Are you giving your Solr pods enough resources on EKS? Have you run Solr in the same configuration outside of kubernetes in the past without timeouts? What type of storage volumes are yo

Re: SolrCloud shows cluster still healthy even the node data directory is deleted

Hi community, I create a Solr Jira to track this issue. https://issues.apache.org/jira/browse/SOLR-15028 Regards, Amy From: Radar Lei Sent: Friday, November 20, 2020 5:13 PM To: solr-user@lucene.apache.org Subject: Re: SolrCloud shows cluster still healthy

Re: SolrCloud shows cluster still healthy even the node data directory is deleted

Date: Wednesday, November 11, 2020 at 8:19 PM To: solr-user@lucene.apache.org Subject: Re: SolrCloud shows cluster still healthy even the node data directory is deleted Hi Erick, Thanks for your kindly reply. There are two things that confuse me: 1. index/search queries keep failing because one o

Re: SolrCloud shows cluster still healthy even the node data directory is deleted

x27;t failover to other healthy replicas. Regards, Amy From: Erick Erickson Sent: Monday, November 9, 2020 8:43 PM To: solr-user@lucene.apache.org Subject: Re: SolrCloud shows cluster still healthy even the node data directory is deleted Depends. *nix systems have dele

Re: SolrCloud shows cluster still healthy even the node data directory is deleted

Depends. *nix systems have delete-on-close semantics, that is as long as there’s a single file handle open, the file will be still be available to the process using it. Only when the last file handle is closed will the file actually be deleted. Solr (Lucene actually) has file handle open to every

Re: Solrcloud create collection ignores createNodeSet parameter

You’re confusing replicas and shards a bit. Solr tries its best to put multiple replicas _of the same shard_ on different nodes. You have two shards though with _one_ replica. Thi is a bit of a nit, but important to keep in mind when your replicatinFactor increases. So from an HA perspective, th

Re: SolrCloud 6.6.2 suddenly crash due to slow queries and Log4j issue

Shawn, According to the log4j description ( https://bz.apache.org/bugzilla/show_bug.cgi?id=57714), the issue is related to lock during appenders collection process. In addition to CONSOLE and file appenders in the default log4j.properties, my customer added 2 extra FileAppender dedicated to all r

Re: SolrCloud 6.6.2 suddenly crash due to slow queries and Log4j issue

Hi Shawn, Thank you for your response. You are confirming my diagnosis. This is in fact a 8 nodes cluster with one single collection with 4 shards and 1 replica (8 cores). 4 Gb heap and 90 Gb Ram When no issue occurs nearly 50% of the heap is used. Num Docs in collection : 10.000.000 Num Do

Re: SolrCloud 6.6.2 suddenly crash due to slow queries and Log4j issue

On 10/18/2020 3:22 AM, Dominique Bejean wrote: A few months ago, I reported an issue with Solr nodes crashing due to the old generation heap growing suddenly and generating OOM. This problem occurred again this week. I have threads dumps for each minute during the 3 minutes the problem occured. I

RE: SolrCloud (6.6.6) SSL Setup - Unable to create collection

I solved my problem by using just the certificate from my first node and copying that to the second node. I'm not sure whether all three are necessary, but I copied: * solr-ssl.keystore.jks * solr-ssl-keystore.p12 * solr-ssl.pem. If you originally made separate certificates for

Re: Solrcloud tlog are not deleted

Hello, Thanks for your reply. Yes, the CDCR buffer is disable when we check it. We finally found that the increase of tlog files was due to the version of Zookeeper used. We re-installed Zookeeper in the same version as the one embedded by Solr, and this fixed the problem of non-deleted tlogs.

Re: Solrcloud tlog are not deleted

Hi, Did you disable CDCR buffer ? solr//cdcr?action=DISABLEBUFFER You can check with "cdcr?action=STATUS" Regards Dominique Le mar. 11 août 2020 à 10:57, Michel Bamouni a écrit : > Hello, > > > We had setup a synchronization between our solr instances on 2 datacenters > by using the CDCR.

Re: SolrCloud on PublicCloud

On 8/3/2020 12:04 PM, Mathew Mathew wrote: Have been looking for architectural guidance on correctly configuring SolrCloud on Public Cloud (eg Azure/AWS) In particular the zookeeper based autoscaling seems to overlap with the auto scaling capabilities of cloud platforms. I have the following q

Re: Solrcloud 6.6 becomes nuts

Hi, We had the problem again a few days ago. I have noticed that each time the problem occurs the old generation of the heap suddenly grows. Its size is generally between 0,5 et 1,5Gb on 3Gg limit. In 4 minutes the old generation grows to 3Gb and never goes down as consecutive GC reclaims 0 bytes

Re: SolrCloud upgrade concern

Thanks for all this information. It clears lot of confusion surrounding CDCR feature. Although, I should say that if CDCR functionality is so fragile in SolrCloud and not worth pursuing much, does it make sense to add some warning about its possible shortcomings in the documentation? On Thu, May 2

Re: SolrCloud upgrade concern

I had a client who asked a lot about CDCR a few years ago, but I kept recommending aginst it and recommended them to go for Ericks’s alternative (2), since they anyway needed to replicate their Oracle DBs in each DC as well. Much cleaner design to let each cluster have a local datasource and alw

Re: SolrCloud upgrade concern

The biggest issue with CDCR is it’s rather fragile and requires monitoring, it’s not a “fire and forget” type of functionality. For instance, the use of the tlogs as a queueing mechanism means that if, for any reason, the communications between DCs is broken, the tlogs will grow forever until the

Re: SolrCloud upgrade concern

Thanks, Jason. This is very helpful. I should clarify though that I am not using CDCR currently with my existing master-slave architecture. What I meant to say earlier was that we will be relying heavily on the CDCR feature if we migrate from solr master-slave architecture to solrcloud architectur

Re: SolrCloud upgrade concern

Hi Arnold, >From what I saw in the community, CDCR saw an initial burst of development around when it was contributed, but hasn't seen much attention or improvement since. So while it's been around for a few years, I'm not sure it's improved much in terms of stability or compatibility with other

Re: SolrCloud upgrade concern

Thanks for this reply, Jason. I am mostly worried about CDCR feature. I am relying heavily on it. Although, I am planning to use Solr 8.3. It has been long time since CDCR was first introduced. I wonder what is the state of CDCR is 8.3. Is it stable now? On Wed, Jan 22, 2020, 8:01 AM Jason Gerlow

Re: Solrcloud 6.6 becomes nuts

Hi Shawn, In fact, I was using logs from a core at WARN log level so with only slow queries (>500ms). I just checked in a core at INFO log level with all queries (we set the log level top INFO for one core after the previous crash) and there is no more queries with these two facets when the probl

Re: Solrcloud 6.6 becomes nuts

On 5/17/2020 4:18 PM, Dominique Bejean wrote: I was not thinking that queries using facet with fields with high number of unique value but with low hits count can be the origin of this problem. Performance for most things does not depend on numFound (hit count) or the rows parameter. The numb

Re: Solrcloud 6.6 becomes nuts

Mickhail, Thank you for your response. --- For the logs On not leader replica, there are no error in log, only WARN due to slow queries. On leader replica, there are these errors: * Twice per minute during all the day before the problem starts and also after the problem start RequestHandlerB

Re: Solrcloud 6.6 becomes nuts

Hi Shawn, There is no OOM error in logs. I gave more details in response to Mickhail. The problem starts with full GC near 15h20 but Young GC changed a little starting 15h10. Here are the heap usage before and after during this period. https://www.eolya.fr/solr_issue_heap_before_after.png There

Re: Solrcloud 6.6 becomes nuts

On 5/17/2020 2:05 AM, Dominique Bejean wrote: One or two hours before the nodes stop with OOM, we see this scenario on all six nodes during the same five minutes time frame : * a little bit more young gc : from one each second (duration<0.05secs) to one each two or three seconds (duration <0.15 s

Re: Solrcloud 6.6 becomes nuts

Hello, Dominique. What did it log? Which exception? Do you have a chance to review heap dump? What did consume whole heap? On Sun, May 17, 2020 at 11:05 AM Dominique Bejean wrote: > Hi, > > I have a six node Solrcoud that suddenly has its six nodes failed with OOM > at the same time. > This can

Re: Solrcloud Garbage Collection Suspension linked across nodes?

Thanks Furkan! > 13. mai 2020 kl. 19:20 skrev Furkan KAMACI : > > Hi John, > > I've denied and dropped him from mail list. > > Kind Regards, > Furkan KAMACI > > On Wed, May 13, 2020 at 8:06 PM John Blythe wrote: > >> can we get this person blocked? >> -- >> John Blythe >> >> >> On Wed, May

Re: Solrcloud Garbage Collection Suspension linked across nodes?

Hi John, I've denied and dropped him from mail list. Kind Regards, Furkan KAMACI On Wed, May 13, 2020 at 8:06 PM John Blythe wrote: > can we get this person blocked? > -- > John Blythe > > > On Wed, May 13, 2020 at 1:05 PM ART GALLERY wrote: > > > check out the videos on this website TROO.TUB

Re: Solrcloud Garbage Collection Suspension linked across nodes?

can we get this person blocked? -- John Blythe On Wed, May 13, 2020 at 1:05 PM ART GALLERY wrote: > check out the videos on this website TROO.TUBE don't be such a > sheep/zombie/loser/NPC. Much love! > https://troo.tube/videos/watch/aaa64864-52ee-4201-922f-41300032f219 > > On Mon, May 4, 2020 a

Re: Solrcloud Garbage Collection Suspension linked across nodes?

check out the videos on this website TROO.TUBE don't be such a sheep/zombie/loser/NPC. Much love! https://troo.tube/videos/watch/aaa64864-52ee-4201-922f-41300032f219 On Mon, May 4, 2020 at 5:43 PM Webster Homer wrote: > > My company has several Solrcloud environments. In our most active cloud we

Re: SolrCloud degraded during backup and batch CSV update

If the errors happen with garbage collection then potentially, yes. You should never pause longer than your zk timeout (both sides). On Thu, Apr 30, 2020 at 11:03 PM Ganesh Sethuraman wrote: > > Any other JVM settings change possible? > > On Tue, Apr 28, 2020, 10:15 PM Sethuraman, Ganesh > wrot

Re: SolrCloud degraded during backup and batch CSV update

Any other JVM settings change possible? On Tue, Apr 28, 2020, 10:15 PM Sethuraman, Ganesh wrote: > Hi > > We are using SolrCloud 7.2.1 with 3 node Zookeeper ensemble. We have 92 > collection each on avg. having 8 shards and 2 replica with 2 EC2 nodes, > with JVM size of 18GB (G1 GC). We need you

Re: SolrCloud degraded during backup and batch CSV update

meters it will certainly help > > Regards > Ganesh > > -Original Message- > From: matthew sporleder > Sent: Wednesday, April 29, 2020 11:47 AM > To: solr-user@lucene.apache.org > Subject: Re: SolrCloud degraded during backup and batch CSV update > > CAUTION:

RE: SolrCloud degraded during backup and batch CSV update

increase the timeout setting in Solr or Zookeeper, if you can share parameters it will certainly help Regards Ganesh -Original Message- From: matthew sporleder Sent: Wednesday, April 29, 2020 11:47 AM To: solr-user@lucene.apache.org Subject: Re: SolrCloud degraded during backup and batch CSV

Re: SolrCloud degraded during backup and batch CSV update

FWIW I've had some luck with strategy 3 (increase zk timeout) when you overwhelm the connection to zk or the disk on zk. Is zk on the same boxes as solr? On Tue, Apr 28, 2020 at 10:15 PM Sethuraman, Ganesh wrote: > > Hi > > We are using SolrCloud 7.2.1 with 3 node Zookeeper ensemble. We have 92

Re: SolrCloud 8.2.0 - adding a field

Nevermind - I see that I need to specify an existing collection not a schema.  There is no collection called UNCLASS - only a schema. -Joe On 4/1/2020 4:52 PM, Joe Obernberger wrote: Hi All - I'm trying this: curl -X POST -H 'Content-type:application/json' --data-binary '{"add-field":{"name"

Re: Solrcloud 7.6 OOM due to unable to create native threads

We have defined a “search feed” as a file of JSONL objects, one per line. The feed files can be stored in S3, reloaded, sent to two clusters, etc. Each destination can keep its own log of failures and retries. We’ve been doing this for full batch feeds and incrementals for a few years. We’ve been

Re: Solrcloud 7.6 OOM due to unable to create native threads

One approach could be to buffer the messages in Kafka before pushing to Solr. And then use "Kafka mirror" to replicate the messages to the other DC. Now both DCs' Kafka pipelines are in sync by the mirror and you can run storm/spark/flink etc jobs to consume local Kafka and publish to local Solr cl

Re: Solrcloud 7.6 OOM due to unable to create native threads

Hi Eric, What are you recommendations for SolrCloud DR strategy. Thanks, Raji On Sun, Mar 29, 2020 at 6:25 PM Erick Erickson wrote: > I don’t recommend CDCR at this point, I think there better approaches. > > The root problem is that CDCR uses tlog files as a queueing mechanism. > If the conne

Re: Solrcloud 7.6 OOM due to unable to create native threads

Thanks Eric. I don't seeing anywhere that CDCR is not recommended for production use. Took the thread dump. Seeing about 140 CDCR threads cdcr-replicator-219-thread-8" #787 prio=5 os_prio=0 tid=0x7f7c34009000 nid=0x50a waiting on condition [0x7f7ec871b000] java.lang.Thread.State: WAIT

Re: Solrcloud 7.6 OOM due to unable to create native threads

I don’t recommend CDCR at this point, I think there better approaches. The root problem is that CDCR uses tlog files as a queueing mechanism. If the connection between the DCs is broken for any reason, the tlogs grow without limit. This could probably be fixed, but a better alternative is to use s

Re: Solrcloud 7.6 OOM due to unable to create native threads

Is CDCR even recommended to be used in production? Or it was abandoned before it could become production ready ? Thanks SG On Sun, Mar 29, 2020 at 5:18 AM Erick Erickson wrote: > What that error usually means is that there are a zillion threads running. > > Try taking a thread dump. It’s _prob

Re: Solrcloud 7.6 OOM due to unable to create native threads

What that error usually means is that there are a zillion threads running. Try taking a thread dump. It’s _probable_ that it’s CDCR, but take a look at the thread dump to see if you have lots of threads that are running. Any by “lots” here, I mean 100s of threads that reference the same component,

Re: SolrCloud - Underlying core creation failed while creating collection with new configset

Hi Erick, Did see any extra error in solr logs. Its the same error I mentioned earlier. I'm using SolrCloud by the way. On Wed, Mar 4, 2020 at 8:06 PM Erick Erickson wrote: > You need to look at the solr logs on the machine where the attempt was > made to create the replica... > > Best, > Erick

Re: SolrCloud - Underlying core creation failed while creating collection with new configset

You need to look at the solr logs on the machine where the attempt was made to create the replica... Best, Erick > On Mar 4, 2020, at 03:24, Vignan Malyala wrote: > > Hi > I created a new config set as mentioned in Solr Cloud documentation using > upload zip. > I get this error when I try to

Re: SolrCloud location for solr.xml

0 7:31 PM To: Jan Høydahl ; solr-user@lucene.apache.org Subject: Re: SolrCloud location for solr.xml Hey Jan, Thanks for the info re swap there’s some interesting observations you’ve mentioned below particularly the container swap by default. There was this note on the Docker forum describin

Re: SolrCloud location for solr.xml

, Dwane PS: I’m assuming you're testing Solr 8.4.1 on Linux hosts? From: Jan Høydahl Sent: Monday, 2 March 2020 12:01 AM To: solr-user@lucene.apache.org Subject: Re: SolrCloud location for solr.xml As long as solr.xml is a mix of setting that need to be

Re: SolrCloud location for solr.xml

As long as solr.xml is a mix of setting that need to be separate per node and cluster wide settings, it makes no sense to enforce it in zk. Perhaps we instead should stop requiring solr.xml and allow nodes to start without it. Solr can then use a hard coded version as fallback. Most users just

Re: SolrCloud location for solr.xml

Actually, I do this all the time. However, it’s because I’m always blowing everything away and installing a different version of Solr or some such, mostly laziness. We should move away from allowing solr.xml to be in SOLR_HOME when running in cloud mode IMO, but that’ll need to be done in phase

Re: SolrCloud upgrade concern

Hi Arnold, The stability and complexity issues Mark highlighted in his post aren't just imagined - there are real, sometimes serious, bugs in SolrCloud features. But at the same time there are many many stable deployments out there where SolrCloud is a real success story for users. Small example

Re: SolrCloud upgrade concern

ha, im on that thread, didnt know they got stored on a site, thats good to know! -i stand by what i said in there. so i have nothing more to add On Thu, Jan 16, 2020 at 3:29 PM Arnold Bronley wrote: > Hi, > > I am trying to upgrade my system from Solr master-slave architecture to > SolrCloud a

Re: Solrcloud export all results sorted by score

Hi Chris, Good info, thank you for that! > What's your UI & middle layer like for this application and > eventual "download" ? I'm working in a team on the back-end side of things, where we providing a REST API that can be used by clients, which include our UI, which is a React JS based app with

Re: Solrcloud export all results sorted by score

: We show a table of search results ordered by score (relevancy) that was : obtained from sending a query to the standard /select handler. We're : working in the life-sciences domain and it is common for our result sets to : contain many millions of results (unfortunately). After users browse the

Re: Solrcloud export all results sorted by score

Hi Walter, Thank you also for your reply. Good to know of your experience. Roughly how many documents were you fetching? Unfortunately, it's possible that some of our users could attempt to "download" many records, meaning we'd need to make a request to Solr where rows >= 150M. A key challenge for

Re: Solrcloud export all results sorted by score

Maybe you can sort later using Spark or similar. For that you don’t need a full blown cluster - it runs also on localhost. > Am 03.10.2019 um 09:49 schrieb Edward Turner : > > Hi Erick, > > Many thanks for your detailed reply. It's really good information for us to > know, and although not exa

Re: Solrcloud export all results sorted by score

Hi Erick, Many thanks for your detailed reply. It's really good information for us to know, and although not exactly what we wanted to hear (that /export wasn't designed to handle ranking), it's much better for us to definitively know one way or the other -- and this allows us to move forward. We'

Re: Solrcloud export all results sorted by score

I had to do this recently on a Solr Cloud cluster. I wanted to export all the IDs, but they weren’t stored as docvalues. The fastest approach was to fetch all the IDs in one request. First, I make a request for zero rows to get the numFound. Then I fetch numFound+1000 (in case docs were added w

Re: Solrcloud export all results sorted by score

First, thanks for taking the time to ask a question with enough supporting details that I can hope to be able to answer in one exchange ;). It’s a pleasure to see. Second, NP with asking on Stack Overflow, they have some excellent answers there. But you’re right, this list gets more Solr-center

RE: SolrCloud recommended I/O RAID level

We'll take all these into consideration in order to size and simulate both indexing and search flow in load. Thanks again, Adi -Original Message- From: Erick Erickson Sent: Sunday, August 4, 2019 6:51 PM To: solr-user@lucene.apache.org Subject: Re: SolrCloud recommended I/O RAID level

Re: SolrCloud recommended I/O RAID level

er to have 400 shards with > replication factor 2 > or 200 shards with replication factor 4 ? What utilizes better the CPUs - > shards/or their replicas ? > > Thanks, > Adi > > -Original Message- > From: Erick Erickson > Sent: Thursday, August 1, 2019 6:48 PM &g

Re: SolrCloud recommended I/O RAID level

On 8/4/2019 8:53 AM, Kaminski, Adi wrote: Erick - thanks a lot for answering and sharing the below article, it's very helpful ! I have another follow-up question - assuming we have 400 vCPUs across our SolrCloud cluster nodes, will it be better to have 400 shards with replication factor 2 or

RE: SolrCloud recommended I/O RAID level

What utilizes better the CPUs - shards/or their replicas ? Thanks, Adi -Original Message- From: Erick Erickson Sent: Thursday, August 1, 2019 6:48 PM To: solr-user@lucene.apache.org Subject: Re: SolrCloud recommended I/O RAID level Yes, I’m exactly talking about the idea of someone pu

Re: SolrCloud recommended I/O RAID level

queries and lower QPS ? > > > > Thanks, > > Adi > > > > -----Original Message- > From: Erick Erickson > Sent: Thursday, August 1, 2019 2:03 PM > To: solr-user@lucene.apache.org > Subject: Re: SolrCloud recommended I/O RAID level > > >

RE: SolrCloud recommended I/O RAID level

k per total amout of vCPUs, and actually cause delays in queries and lower QPS ? Thanks, Adi -Original Message- From: Erick Erickson Sent: Thursday, August 1, 2019 2:03 PM To: solr-user@lucene.apache.org Subject: Re: SolrCloud recommended I/O RAID level “why would I need a replic

Re: SolrCloud recommended I/O RAID level

n ? Won't it be double > protection layer, while the IO layer mirroring of RAID10 actually brings the > value, and no need to copy anything > when we have IO failures ? > > Thanks, > Adi > > -Original Message- > From: Shawn Heisey > Sent: Tuesday

RE: SolrCloud recommended I/O RAID level

PM To: solr-user@lucene.apache.org Subject: Re: SolrCloud recommended I/O RAID level On 7/30/2019 12:12 PM, Kaminski, Adi wrote: > Indeed RAID10 with both mirroring and striping should satisfy the > need, but per some benchmarks in the network there is still an impact > on write performance

Re: SolrCloud recommended I/O RAID level

On 7/30/2019 12:12 PM, Kaminski, Adi wrote: Indeed RAID10 with both mirroring and striping should satisfy the need, but per some benchmarks in the network there is still an impact on write performance on it compared to RAID0 which is considered as much better (attaching a table that summarizes

Re: SolrCloud recommended I/O RAID level

Hi Furkan, Thanks for your response ! Indeed RAID10 with both mirroring and striping should satisfy the need, but per some benchmarks in the network there is still an impact on write performance on it compared to RAID0 which is considered as much better (attaching a table that summarizes differ

Re: SolrCloud recommended I/O RAID level

Hi Adi, RAID10 is good for satisfying both indexing and query, striping across mirror sets. However, you lose half of your raw disk space, just like with RAID1. Here is a mail thread of mine which discusses RAID levels for Solr specific: https://lists.apache.org/thread.html/462d7467b2f2d064223eb4

RE: solrcloud backup null pointer exeption

can you share: solr version? zookeeper ensemble type number of shards in the collection? distribution of shard replicas in the SolrCloud? from there the most obvious question is whether the stack trace is from the shard leader for the collection or the localhost; if they are not the same? There

Re: solrcloud backup null pointer exeption

Follow the instructions here: http://lucene.apache.org/solr/community.html#mailing-lists-irc . You must use the _exact_ same e-mail as you used to subscribe. If the initial try doesn't work and following the suggestions at the "problems" link doesn't work for you, let us know. But note you need

Re: solrcloud backup null pointer exeption

unsubscribe On Thu, Jul 25, 2019 at 5:03 PM rffleaie wrote: > I have a solrcloud cluster installed on k8s. > I have created a nfs PVC that is mounted under /backup of every pod of the > solr cluster. > > When I start the backup with > > > http://127.0.0.1:8983/solr/admin/collections?action=BACKU

Re: SolrCloud indexing triggers merges and timeouts

Upon further investigation on this issue, I see the below log lines during the indexing process: 2019-06-06 22:24:56.203 INFO (qtp1169794610-5652) [c:UM_IndexServer_MailArchiv_Spelle_66AC8340-4734-438A-9D1D-A84B659B1623 s:shard22 r:core_node87 x:UM_IndexServer_MailArchiv_Spelle_66AC8340-4734-438A

Re: SolrCloud indexing triggers merges and timeouts

Shawn,Erick, Thank you for the explanation. The merge scheduler params make sense now. Thanks, Rahul On Wed, Jul 3, 2019 at 11:30 AM Erick Erickson wrote: > Two more tidbits to add to Shawn’s explanation: > > There are heuristics built in to ConcurrentMergeScheduler. > From the Javadocs: > * If

Re: SolrCloud indexing triggers merges and timeouts

Two more tidbits to add to Shawn’s explanation: There are heuristics built in to ConcurrentMergeScheduler. From the Javadocs: * If it's an SSD, * {@code maxThreadCount} is set to {@code max(1, min(4, cpuCoreCount/2))}, * otherwise 1. Note that detection only currently works on * Linux; other p

Re: SolrCloud indexing triggers merges and timeouts

On 7/2/2019 10:53 PM, Rahul Goswami wrote: Hi Shawn, Thank you for the detailed suggestions. Although, I would like to understand the maxMergeCount and maxThreadCount params better. The documentation mention

Re: SolrCloud indexing triggers merges and timeouts

Hi Shawn, Thank you for the detailed suggestions. Although, I would like to understand the maxMergeCount and maxThreadCount params better. The documentation mentions that maxMergeCount : The maximum number of

Re: SolrCloud: Configured socket timeouts not reflecting

Hi Gus, Have created a pull request for JIRA 12550 and updated the affected Solr version (7.2.1) in the comments. The provided fix is on branch_7_2. I haven't tried reproducing the issue on the latest version, but see that the code for this part i

Re: SolrCloud: Configured socket timeouts not reflecting

Hi Gus, Thanks for the response and referencing the umbrella JIRA for these kind of issues. I see that it won't solve the problem since the builder object which is used to instantiate a ConcurrentUpdateSolrClient itself doesn't contain the timeout values. I did create a local solr-core binary to tr

Re: SolrCloud: Configured socket timeouts not reflecting

Hi Rahul, Did you try the patch int that issue? Also food for thought: https://issues.apache.org/jira/browse/SOLR-13457 -Gus On Tue, Jun 18, 2019 at 5:52 PM Rahul Goswami wrote: > Hello, > > I was looking into the code to try to get to the root of this issue. Looks > like this is an issue afte

  1   2   3   4   5   6   7   8   9   10   >