On 9/22/2016 3:27 PM, vsolakhian wrote:
> This is not the cause of the problem though. The disk cache is
> important for queries and overall performance during optimization, but
> once it is done, everything should go back to "normal" (whatever that
> normal is). In our case it is the SOFT COMMIT (
message in context:
http://lucene.472066.n3.nabble.com/Very-Slow-Commits-After-Solr-Index-Optimization-tp4297022p4297588.html
Sent from the Solr - User mailing list archive at Nabble.com.
On 9/22/2016 1:01 PM, vsolakhian wrote:
> Our index is in HDFS, but we did not change any configuration after we
> deleted 35% of records and optimized.
>
> The relatively slow commit (soft commit and warming up took 1.5 minutes) is
> OK for our use case (adding hundreds of thousands and even milli
.n3.nabble.com/Very-Slow-Commits-After-Solr-Index-Optimization-tp4297022p4297548.html
Sent from the Solr - User mailing list archive at Nabble.com.
n the
remote filesystem device/server might need more memory and/or
configuration adjustments. The speed of the network might be a factor
with remote filesystems.
Side note: A commit that takes 1.5 minutes is ALREADY very slow.
Commits should normally take seconds. Well-tuned N
segment, then how is it
possible to split this segment into smaller ones (without sharding)?
Thanks,
Victor
--
View this message in context:
http://lucene.472066.n3.nabble.com/Very-Slow-Commits-After-Solr-Index-Optimization-tp4297022.html
Sent from the Solr - User mailing list archive at Nabble.com.
include
that.
From: Yonik Seeley [ysee...@gmail.com]
Sent: 22 February 2016 15:43
To: solr-user@lucene.apache.org
Subject: Re: Slow commits
On Mon, Feb 22, 2016 at 10:22 AM, Adam Neal [Extranet] wrote:
> Highest count is fairly equal between string
a JIRA issue for this?
-Yonik
>
> From: Yonik Seeley [ysee...@gmail.com]
> Sent: 22 February 2016 14:40
> To: solr-user@lucene.apache.org
> Subject: Re: Slow commits
>
>
> What are the types of the fields with the highest count? I assume
Highest count is fairly equal between string and text. They are not indexed but
stored and no docvalues used
From: Yonik Seeley [ysee...@gmail.com]
Sent: 22 February 2016 14:40
To: solr-user@lucene.apache.org
Subject: Re: Slow commits
What are the types
around 1 second.
>
> From: Adam Neal [Extranet] [an...@mass.co.uk]
> Sent: 19 February 2016 17:43
> To: solr-user@lucene.apache.org
> Subject: RE: Slow commits
>
> I'm out of the office now so I don't have the numbers to hand but from memory
> I thin
e.org
Subject: Re: Slow commits
Sorry, I see now you mentioned 56K docs which is pretty small.
On Mon, Feb 22, 2016 at 8:30 AM, Susheel Kumar
wrote:
> Adam - how many documents you have in your index?
>
> Thanks,
> Susheel
>
> On Mon, Feb 22, 2016 at 4:37 AM, Adam Neal [Extranet]
&
slower than the
>> 4.10.2 commit on the original 66000 fields which was around 1 second.
>>
>> From: Adam Neal [Extranet] [an...@mass.co.uk]
>> Sent: 19 February 2016 17:43
>> To: solr-user@lucene.apache.org
>> Subject: RE:
ass.co.uk]
> Sent: 19 February 2016 17:43
> To: solr-user@lucene.apache.org
> Subject: RE: Slow commits
>
> I'm out of the office now so I don't have the numbers to hand but from
> memory I think there are probably around 800-1000 fields or so. I will
> confirm on Mo
commit on the
original 66000 fields which was around 1 second.
From: Adam Neal [Extranet] [an...@mass.co.uk]
Sent: 19 February 2016 17:43
To: solr-user@lucene.apache.org
Subject: RE: Slow commits
I'm out of the office now so I don't have the numbe
p a sample.
From: Yonik Seeley [ysee...@gmail.com]
Sent: 19 February 2016 16:25
To: solr-user@lucene.apache.org
Subject: Re: Slow commits
On Fri, Feb 19, 2016 at 8:51 AM, Adam Neal [Extranet] wrote:
> I've recently upgraded from 4.10.2 to 5.3.1 and I've hit an issue
On Fri, Feb 19, 2016 at 8:51 AM, Adam Neal [Extranet] wrote:
> I've recently upgraded from 4.10.2 to 5.3.1 and I've hit an issue with slow
> commits on one of my cores. The core in question is relatively small (56k
> docs) and the issue only shows when commiting after
@lucene.apache.org
Subject: Re: Slow commits
On 2/19/2016 6:51 AM, Adam Neal [Extranet] wrote:
> I've recently upgraded from 4.10.2 to 5.3.1 and I've hit an issue with slow
> commits on one of my cores. The core in question is relatively small (56k
> docs) and the issue only shows when commi
On 2/19/2016 6:51 AM, Adam Neal [Extranet] wrote:
> I've recently upgraded from 4.10.2 to 5.3.1 and I've hit an issue with slow
> commits on one of my cores. The core in question is relatively small (56k
> docs) and the issue only shows when commiting after a number of de
February 2016 13:51
To: solr-user@lucene.apache.org
Subject: Slow commits
I've recently upgraded from 4.10.2 to 5.3.1 and I've hit an issue with slow
commits on one of my cores. The core in question is relatively small (56k docs)
and the issue only shows when commiting after a number
I've recently upgraded from 4.10.2 to 5.3.1 and I've hit an issue with slow
commits on one of my cores. The core in question is relatively small (56k docs)
and the issue only shows when commiting after a number of deletes, commiting
after additions is fine. As an example commi
slow commits and overlapping commits
I managed to get a thread dump during a slow commit:
"resin-tcp-connection-*:5062-129" Id=12721 in RUNNABLE total cpu
time=391530.ms user time=390620.ms at java.lang.String.intern(Native
Method) at
org.apache.lucene.util.SimpleStringInter
I managed to get a thread dump during a slow commit:
"resin-tcp-connection-*:5062-129" Id=12721 in RUNNABLE total cpu
time=391530.ms user time=390620.ms
at java.lang.String.intern(Native Method)
at
org.apache.lucene.util.SimpleStringInterner.intern(SimpleStringInterner.java:74)
at org.apac
I am taking a snapshot after every commit. From looking at the snapshots,
it does not look like the delay in caused by segments merging because I am
not seeing any large new segments after a commit.
I still can't figure out why there is a 2 minutes gap between "start commit"
and "SolrDelectionPol
You can use the postCommit event listener as an callback mechanism to let
you know that a commit has happened.
Bill
On Sun, May 22, 2011 at 9:31 PM, Jeff Crump wrote:
> I don't have an answer to this but only another question: I don't think I
> can use auto-commit in my application, as I have
I don't have an answer to this but only another question: I don't think I
can use auto-commit in my application, as I have to "checkpoint" my index
submissions and I don't know of any callback mechanism that would let me
know a commit has happened. Is there one?
2011/5/21 Erick Erickson
> Well
Well, committing less offside a possibilty . Here's what's probably
happening. When you pass certain thresholds, segments are merged which can
take quite some time. His are you triggering commits? If it's external,
think about using auto commit instead.
Best
Erick
On May 20, 2011 6:04 PM, "Bill
On my Solr 1.4.1 master I am doing commits regularly at a fixed interval. I
noticed that from time to time commit will take longer than the commit
interval, causing commits to overlap. Then things will get worse as commit
will take longer and longer. Here is the logs for a long commit:
[2011-0
nm, _al7i_2g.del, _ailk.fnm, _al8x.tii, _al8x.tis, _ala8.fnm,
>>> _akzu.frq, _9wzn.frq, _7zfh.nrm, _akuu.fdt, _alad.tii, _akuu.fdx, _aku
>>> u_cd.del, _a61p_b77.del, _alad.tis, _al2o_67.del, _add1.fdx, _9wzn.prx,
>>> _al9w.fdt, _add1.fdt, _al9w.fdx, _akuu.prx, _962y.frq,
a7.fdt, _alaf.fdx, _alaf.fdt, _al2o
>> .fdx, _7zfh.frq, _akzu.fdt, _alaf.nrm, _akzu.fdx, _alab.nrm, _al2o.fdt,
>> _alaa.prx, _alaa.nrm, _ala7.prx, _ailk.fdt, _akzu.prx, _alac.frq,
>> _ailk.prx, _alaf.tii, _alaf_1.del, _alae.frq, _add1.fnm, _alaf.tis,
>> _7zfh.prx, _al9w.fnm, _ala
lae.fdt, _a61p.fnm, _alaf.frq,
> _ala8.tii, _a61p.tis, _9wzn.tii, _alad.frq, _ala8.tis, _alab.tii,
> _akh1.frq, _alab.tis, _al7i.fnm, _alab.fdx, _alac.nrm, _ala8_5.del,
> _ala7.tii, _alab.fdt, _alaa.fdx, _al2o.frq, _akh1.fdx, _alac.prx, _ala
> a.fdt, _al2o.prx, _akh1.fdt, _alad.prx, _ala7.
ext:
http://www.nabble.com/Slow-Commits-tp26097538p26097538.html
Sent from the Solr - User mailing list archive at Nabble.com.
This was answered yesterday on the list:
http://www.nabble.com/Re%3A-exceeded-limit-of-maxWarmingSearchers-p17165631.html
regards,
-Mike
On 12-May-08, at 6:12 PM, David Stevenson wrote:
We have a table that has roughly 1M rows.
If we run a query against the table and order by a string field
We have a table that has roughly 1M rows.
If we run a query against the table and order by a string field that has a
large number of unique values then subsequent commits of any other document
takes much longer.
If we don't run the query or if we order on a string field with very few
unique value
33 matches
Mail list logo
