Re: Solr 7.6.0 High CPU Usage Deadlock (Generating 70% of futex system call)

I set up Solr Cloud 7.7.2 and follow the tutorial https://lucene.apache.org/solr/guide/8_2/getting-started-with-solrcloud.html#getting-started-with-solrcloud I still see exactly the same behavior, even with 0 records and out of the box. Why would it generates 40.000 Futex syscall in 30 seconds eve

Re: Solr 7.6.0 High CPU Usage Deadlock (Generating 70% of futex system call)

Hello Shalin, It's not an OS bug, Linux samba 4.15.0-55-generic #60-Ubuntu SMP Tue Jul 2 18:22:20 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux I'm sending the full thread dump log: 2019-11-01 11:05:13 Full thread dump Java HotSpot(TM) 64-Bit Server VM (25.231-b11 mixed mode): "Attach Listener" #259 d

Re: Solr 7.6.0 High CPU Usage Deadlock (Generating 70% of futex system call)

Perhaps this bug in the kernel? https://groups.google.com/forum/#!topic/mechanical-sympathy/QbmpZxp6C64 Can you check if you are running an affected kernel? On Fri, Nov 1, 2019 at 7:09 AM Daniel de Oliveira Mantovani < daniel.oliveira.mantov...@gmail.com> wrote: > I have a Solr 7.6.0 Cloud with

Solr 7.6.0 High CPU Usage Deadlock (Generating 70% of futex system call)

I have a Solr 7.6.0 Cloud with 4 nodes and while indexing Solr hangs. $ java -version java version "1.8.0_231" Java(TM) SE Runtime Environment (build 1.8.0_231-b11) Java HotSpot(TM) 64-Bit Server VM (build 25.231-b11, mixed mode $ sudo strace -f -c -p 5972 % time seconds usecs/call calls

RE: [EXTERNAL] Re: High cpu usage when adding documents to v7.7 solr cloud

ERNAL] Re: High cpu usage when adding documents to v7.7 solr cloud Hi Peter, This bug was introduced in Solr 7.7.0. It is related to Java 8. And it was fixed in Solr 7.7.2. Here are the ways to deal with it: 1. Upgrade to Solr 7.7.2 2. Patch your Solr 7.7 3. Use Java 9+ You can read more on this

Re: High cpu usage when adding documents to v7.7 solr cloud

, Oleksandr On Tue, Oct 15, 2019 at 8:31 PM Peter Lancaster < peter.lancas...@findmypast.com> wrote: > We have a solr cloud on v7.7.0 and we observe very high cpu usage when > we're indexing new documents. > > The solr cloud in question has 50 shards and 2 replicas of eac

High cpu usage when adding documents to v7.7 solr cloud

We have a solr cloud on v7.7.0 and we observe very high cpu usage when we're indexing new documents. The solr cloud in question has 50 shards and 2 replicas of each and we're using NRT. Obviously indexing takes some resources but we see pretty much 100% cpu usage when we're ind

Re: very high query time on solr due to high CPU usage

On 5/25/2019 6:43 AM, Saurabh Sharma wrote: Hi, Link to image https://ibb.co/8g6gXwr That screenshot is not sorted the way that was mentioned on the wiki page - by the RES memory column. I do see several other Java processes besides Solr. There might be other high-memory use processes that

Re: very high query time on solr due to high CPU usage

Hi, Link to image https://ibb.co/8g6gXwr Thanks & Regards On Sat 25 May, 2019, 6:09 PM Shawn Heisey, wrote: > On 5/25/2019 5:11 AM, Saurabh Sharma wrote: > > I again faced the issue and restarting the leader worked for me this > time. > > Please find attached the top command for further insigh

Re: very high query time on solr due to high CPU usage

On 5/25/2019 5:11 AM, Saurabh Sharma wrote: I again faced the issue and restarting the leader worked for me this time. Please find attached the top command for further insights. First java process in screenshot is solr. Can it be a possibility that there are some issue with this particular node

Re: very high query time on solr due to high CPU usage

Hi Shawn, I again faced the issue and restarting the leader worked for me this time. Please find attached the top command for further insights. First java process in screenshot is solr. Can it be a possibility that there are some issue with this particular node? Looking forward to hearing from

Re: very high query time on solr due to high CPU usage

On 5/11/2019 12:49 PM, Saurabh Sharma wrote: Full collection is present on all 3 nodes.I have checked max docs on every node and they were around 1.5 million on each node with 0.9 Millions active records. *How much disk space do all the indexes take?* -> index size is around 2GB/per node. *What

Re: very high query time on solr due to high CPU usage

sage keeps on increasing for around 30 mins and even restart > don't > > help. RAM usage remains constant for this duration.Do solr start behaving > > like this under high traffic situations ? what can be the possible > reasons > > of very high CPU usage in real time sol

Re: very high query time on solr due to high CPU usage

around 30 mins and even restart don't help. RAM usage remains constant for this duration.Do solr start behaving like this under high traffic situations ? what can be the possible reasons of very high CPU usage in real time solr. If you get the maxDoc number from every core (index) in that

very high query time on solr due to high CPU usage

x27;t help. RAM usage remains constant for this duration.Do solr start behaving like this under high traffic situations ? what can be the possible reasons of very high CPU usage in real time solr. I am using solr 7.3.1 and java 8 . Thanks Saurabh

Re: High CPU usage with Solr 7.7.0

ist. > I just ran across the same high CPU usage issue. I believe it’’s caused by > this commit which was introduced in Solr 7.7.0 > https://github.com/apache/lucene-solr/commit/eb652b84edf441d8369f5188cdd5e3ae2b151434#diff-e54b251d166135a1afb7938cfe152bb5 > There is a bug in JDK versi

Re: High CPU usage with Solr 7.7.0

I forward this message. Thanks Adam. Hi, Apologies, I can’t figure out how to reply to the Solr mailing list. I just ran across the same high CPU usage issue. I believe it’’s caused by this commit which was introduced in Solr 7.7.0 https://github.com/apache/lucene-solr/commit

Antwort: Re: Re: High CPU usage with Solr 7.7.0

$ReferenceHandler.run​(Reference.java:153) Von:"Tomás Fernández Löbbe" An: solr-user@lucene.apache.org, Datum: 27.02.2019 19:34 Betreff: Re: Re: High CPU usage with Solr 7.7.0 Maybe a thread dump would be useful if you still have some instance running on 7.7 On Wed, Feb 27, 2019

Re: Re: High CPU usage with Solr 7.7.0

solr-user@lucene.apache.org, "Lukas Weiss" > , > Datum: 27.02.2019 15:59 > Betreff:Re: High CPU usage with Solr 7.7.0 > > > > Just to add to this. We upgraded to 7.7.0 and saw very large CPU usage > on multi core boxes - sustained in the 1200% range. We

Antwort: Re: High CPU usage with Solr 7.7.0

I can confirm this. Downgrading to 7.6.0 solved the issue. Thanks for the hint. Von:"Joe Obernberger" An: solr-user@lucene.apache.org, "Lukas Weiss" , Datum: 27.02.2019 15:59 Betreff: Re: High CPU usage with Solr 7.7.0 Just to add to this. We upgra

Re: High CPU usage with Solr 7.7.0

Just to add to this.  We upgraded to 7.7.0 and saw very large CPU usage on multi core boxes - sustained in the 1200% range.  We then switched to 7.6.0 (no other configuration changes) and the problem went away. We have a 40 node cluster and all 40 nodes had high CPU usage with 3 indexes

High CPU usage with Solr 7.7.0

Hello, we recently updated our Solr server from 6.6.5 to 7.7.0. Since then, we have problems with the server's CPU usage. We have two Solr cores configured, but even if we clear all indexes and do not start the index process, we see 100 CPU usage for both cores. Here's what our top says: root@

Re: High CPU Usage in export handler

Joel: I did a little work with SOLR-9296 to try to reduce the number of objects created, which would relieve GC pressure both at creation and collection time. I didn't measure CPU utilization before/after, but I did see up to a 11% increase in throughput. It wouldn't hurt my feelings at all to ha

Re: High CPU Usage in export handler

Thanks Joel. 2016-11-08 11:43 GMT-08:00 Joel Bernstein : > It sounds like your scenario, is around 25 queries per second, each pulling > entire results. This would be enough to drive up CPU usage as you have more > concurrent requests then CPU's. Since there isn't much IO blocking > happening, in

Re: High CPU Usage in export handler

It sounds like your scenario, is around 25 queries per second, each pulling entire results. This would be enough to drive up CPU usage as you have more concurrent requests then CPU's. Since there isn't much IO blocking happening, in the scenario you describe, I would expect some pretty busy CPU's.

Re: High CPU Usage in export handler

Hello: Any follow up? 2016-11-03 11:18 GMT-07:00 Ray Niu : > the soft commit is 15 seconds and hard commit is 10 minutes. > > 2016-11-03 11:11 GMT-07:00 Erick Erickson : > >> Followup question: You say you're indexing 100 docs/second. How often >> are you _committing_? Either >> soft commit >

Re: High CPU Usage in export handler

Followup question: You say you're indexing 100 docs/second. How often are you _committing_? Either soft commit or hardcommit with openSearcher=true ? Best, Erick On Thu, Nov 3, 2016 at 11:00 AM, Ray Niu wrote: > Thanks Joel > here is the information you requested. > Are you doing heavy writes

Re: High CPU Usage in export handler

the soft commit is 15 seconds and hard commit is 10 minutes. 2016-11-03 11:11 GMT-07:00 Erick Erickson : > Followup question: You say you're indexing 100 docs/second. How often > are you _committing_? Either > soft commit > or > hardcommit with openSearcher=true > > ? > > Best, > Erick > > On Th

Re: High CPU Usage in export handler

Thanks Joel here is the information you requested. Are you doing heavy writes at the time? we are doing write very frequently, but not very heavy, we will update about 100 solr document per second. How many concurrent reads are are happening? the concurrent reads are about 1000-2000 per minute per

Re: High CPU Usage in export handler

Are you doing heavy writes at the time? How many concurrent reads are are happening? What version of Solr are you using? What is the field definition for the double, is it docValues? Joel Bernstein http://joelsolr.blogspot.com/ On Thu, Nov 3, 2016 at 12:56 AM, Ray Niu wrote: > Hello: >

High CPU Usage in export handler

Hello: We are using export handler in Solr Cloud to get some data, we only request for one field, which type is tdouble, it works well at the beginning, but recently we saw high CPU issue in all the solr cloud nodes, we took some thread dump and found following information: java.lang.Thread.

Re: High CPU usage after import

happen in all the files. Maybe they're broken. > > > Thanks for participating. Unfortunately it's not that. Set stored = "false" > for > all fields, nothing has changed.Re: High CPU usage after import > > Are you storing the data? That is, the raw binary of t

Re: High CPU usage after import

This does not happen in all the files. Maybe they're broken. Thanks for participating. Unfortunately it's not that. Set stored = "false" for all fields, nothing has changed.Re: High CPU usage after import Are you storing the data? That is, the raw binary of the MP3? B/c when

Re: High CPU usage after import

Thanks for participating. Unfortunately it's not that. Set stored = "false" for all fields, nothing has changed.Re: High CPU usage after import Are you storing the data? That is, the raw binary of the MP3? B/c when stored="true", Solr will try to compress the data, perh

Re: High CPU usage after import

Александр Вандышев wrote: > I realized what the problem was. One of the Solr threads freezes when > importing > MP3 files. When there are many such files Solr loads all processors. Is > there a > way to free thread? > > Re: High CPU usage after import That could mean that t

Re: High CPU usage after import

I realized what the problem was. One of the Solr threads freezes when importing MP3 files. When there are many such files Solr loads all processors. Is there a way to free thread? Re: High CPU usage after import That could mean that the code is hung somehow. Or, maybe Solr is just working on the

Re: High CPU usage after import

CPU usage does settle down. -- Jack Krupansky -Original Message- From: Александр Вандышев Sent: Thursday, April 3, 2014 3:24 AM To: Solr User Subject: High CPU usage after import Thanks for the answer. I meant that the CPU does not free after the end of import.Tomtcat or Solr continue

High CPU usage after import

PDF files. -- Jack Krupansky -Original Message- From: Александр Вандышев Sent: Tuesday, April 1, 2014 9:28 AM To: Solr User Subject: High CPU usage after import I use a update/extract handler for indexing a large number of files. If during indexing a CPU loads was not maximum at the end of

Re: High CPU usage after import

Some document types can consume significant CPU resources, such as large PDF files. -- Jack Krupansky -Original Message- From: Александр Вандышев Sent: Tuesday, April 1, 2014 9:28 AM To: Solr User Subject: High CPU usage after import I use a update/extract handler for indexing a

High CPU usage after import

I use a update/extract handler for indexing a large number of files. If during indexing a CPU loads was not maximum at the end of import loading decreases. If CPU loading was max then loading remain high. Who can help me?

Re: solr-4.0 and high cpu usage [SOLVED]

fault operator > is used, which defaults to OR. That made return much more results for each > query I was running, increasing the response time and the CPU usage. > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/solr-4-0-and-high-cpu-usage-tp399318

Re: solr-4.0 and high cpu usage

t QTime. Best Erick On Thu, Jul 5, 2012 at 7:18 AM, Anatoli Matuskova wrote: > Hello, > I'm testing solr-4.0-alpha compared to 1.4. My index is optimized to one > segment. I've seen a decrease in memory usage but a very high increase in > CPU. This high cpu usage ends up giv

Re: solr-4.0 and high cpu usage [SOLVED]

. -- View this message in context: http://lucene.472066.n3.nabble.com/solr-4-0-and-high-cpu-usage-tp3993187p3993275.html Sent from the Solr - User mailing list archive at Nabble.com.

solr-4.0 and high cpu usage

Hello, I'm testing solr-4.0-alpha compared to 1.4. My index is optimized to one segment. I've seen a decrease in memory usage but a very high increase in CPU. This high cpu usage ends up giving me slower response times with 4.0 than 1.4 The server I'm using: Linux version 2.6.30-2-a

Re: high cpu usage

011 4:31:53 AM > Subject: high cpu usage > > Hello, > We have been running read only solr instances for a few months now, > yesterday >i have noticed an high cpu usage coming from the JVM, it simply use 100% of >the >CPU for no reason. > Nothing was changed, we are usi

Re: high cpu usage

ances for a few months now, > yesterday i have noticed an high cpu usage coming from the JVM, it simply > use 100% of the CPU for no reason. > Nothing was changed, we are using Jetty as a Servlet container for solr. > Where can i start looking what cause it? it has been using 100% CPU

high cpu usage

Hello, We have been running read only solr instances for a few months now, yesterday i have noticed an high cpu usage coming from the JVM, it simply use 100% of the CPU for no reason. Nothing was changed, we are using Jetty as a Servlet container for solr. Where can i start looking what cause

Re: high CPU usage and SelectCannelConnector threads used a lot

Hi John, sounds like this bug in NIO: http://jira.codehaus.org/browse/JETTY-937 http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6403933 I think recent versions of jetty work around this bug, or maybe try the non-NIO socket connector Kent On Tue, Dec 7, 2010 at 9:10 AM, John Russell wrote:

high CPU usage and SelectCannelConnector threads used a lot

Hi, I'm using solr and have been load testing it for around 4 days. We use the solrj client to communicate with a separate jetty based solr process on the same box. After a few days solr's CPU% is now consistently at or above 100% (multiple processors available) and the application using it is mo

Re: Master/Slave High CPU Usage

You're welcome. On Tue, Nov 23, 2010 at 3:55 PM, Ofer Fort wrote: > ok, we ran some tests and doing the commit for the "slave" as a post commit > event of the "master" reloaded the index and allowed us to achieve a master > slave configuration, without replication > This is useful only if your ma

Re: Master/Slave High CPU Usage

ok, we ran some tests and doing the commit for the "slave" as a post commit event of the "master" reloaded the index and allowed us to achieve a master slave configuration, without replication This is useful only if your master and slave are on the same machine, and it helps reducing the resources

Re: Master/Slave High CPU Usage

do i really need a commit? or can i use the *readercycle*script? since i don't need to comit anything, just reopen the reader. thanks On Sun, Nov 21, 2010 at 12:17 PM, Lance Norskog wrote: > Yes, the Solr commit operations always reloads the index

Re: Master/Slave High CPU Usage

ok, i'll try that and update the group thanks On Sun, Nov 21, 2010 at 12:17 PM, Lance Norskog wrote: > Yes, the Solr commit operations always reloads the index. And it > always throws away the Solr caches: queryresult, document, filter > query. > > If you do this, please post your results. > > O

Re: Master/Slave High CPU Usage

Yes, the Solr commit operations always reloads the index. And it always throws away the Solr caches: queryresult, document, filter query. If you do this, please post your results. On Sat, Nov 20, 2010 at 11:16 PM, Ofer Fort wrote: > OK, > so to make sure i understand, even though the "slave" doe

Re: Master/Slave High CPU Usage

OK, so to make sure i understand, even though the "slave" doesn't do any indexing, i will call commit and it will do nothing to the index itself, but will reload it? thanks On Sun, Nov 21, 2010 at 8:26 AM, Lance Norskog wrote: > Ah! If the program doing the indexing has manual commits, the progr

Re: Master/Slave High CPU Usage

Ah! If the program doing the indexing has manual commits, the program could send a commit to the slave. If the indexer uses automatic commits, there is a trick: you can add a program as a postCommit event in solrconfig.xml. This can just be a shell script or a curl command that sends a commit to th

Re: Master/Slave High CPU Usage

thanks Erick, but my question was regard the configuration Lance suggested, a configuration where i have two servers, set set logical master and slave, not as a true replication. Since both are running on the same machine, just have one only doing updates, and the other only queries, but both are u

Re: Master/Slave High CPU Usage

The slave polls. See: http://wiki.apache.org/solr/SolrReplication Best Erick On Sat, Nov 20, 2010 at 1:13 PM, Ofer Fort wrote: > Another question on that configuration, when the "master" commits, how does > the "slave" knows that the index has changed? Does it check the index and > finds out th

Re: Master/Slave High CPU Usage

Another question on that configuration, when the "master" commits, how does the "slave" knows that the index has changed? Does it check the index and finds out that it has a newer version? Thanks again for the help, Ofer ב-19 בנוב 2010, בשעה 05:30, Lance Norskog כתב/ה: If they are on the same

Re: Master/Slave High CPU Usage

That sounds like a great option, and it will also free some storage space on that sever (now each index is about 130GB). Other than the lock policy (we use single), any other things to look out to? Thanks ב-19 בנוב 2010, בשעה 05:30, Lance Norskog כתב/ה: If they are on the same server, you do no

Re: Master/Slave High CPU Usage

If they are on the same server, you do not need to replicate. If you only do queries, the query server can use the same index directory as the master. Works quite well. Both have to have the same LockPolicy in solrconfig.xml. For security reasons, I would run the query server as a different user w

Re: Master/Slave High CPU Usage

anybody? On Wed, Nov 17, 2010 at 12:09 PM, Ofer Fort wrote: > > Hi, I'm working with Erez, > we experienced this again, and this time the slave index folder didn't > contain the index.XXX folder, only one index folder. > if we shutdown the slave, the CPU on the master was normal, as soon as we

Re: Master/Slave High CPU Usage

Hi, I'm working with Erez, we experienced this again, and this time the slave index folder didn't contain the index.XXX folder, only one index folder. if we shutdown the slave, the CPU on the master was normal, as soon as we started the slave again, the CPU went up to 100% again. thanks for any hel

Master/Slave High CPU Usage

Hi all, We've been seeing this for the second time already. I have a solr (1.4.1) master and a slave. both are located on the same machine (16GB RAM, 4GB allocated to the slave and 3GB to the master) All our updates are going towards the master, and all the queries are towards the slave. Once in