Re: Question about autocommit

Wed, 19 Nov 2008 08:46:52 -0800

Could also go with the thread safety issues with pending and the deadlock that was reported the other day. All could pretty easily be related. Do we have a JIRA issue on it yet? Suppose I'll look... 

Mark Miller wrote:
Interesting...could go along with the earlier guys post about slow indexing... 

Nickolai Toupikov wrote:

Hello,

I would like some details on the autocommit mechanism. I tried to 
search the wiki, but found only the

standard maxDoc/time settings.

i have set the autocommit parameters in solrconfig.xml to 8000 docs  
and 300000milis.
Indexing at around  200 docs per second (from multiple processes, 
using the CommonsHttpSolrServer class),

i would have expected autocommits to occur around every  40 seconds,

however the jvm log shows the following -  sometimes more than two 
calls per second:


$ tail -f jvm-default.log | grep "commit"

[16:18:15.862] {pool-2-thread-1} start 
commit(optimize=false,waitFlush=true,waitSearcher=true)

[16:18:16.788] {pool-2-thread-1} end_commit_flush

[16:18:21.721] {pool-2-thread-1} start 
commit(optimize=false,waitFlush=true,waitSearcher=true)

[16:18:22.073] {pool-2-thread-1} end_commit_flush

[16:18:36.047] {pool-2-thread-1} start 
commit(optimize=false,waitFlush=true,waitSearcher=true)

[16:18:36.468] {pool-2-thread-1} end_commit_flush

[16:18:36.886] {pool-2-thread-1} start 
commit(optimize=false,waitFlush=true,waitSearcher=true)

[16:18:37.017] {pool-2-thread-1} end_commit_flush

[16:18:37.867] {pool-2-thread-1} start 
commit(optimize=false,waitFlush=true,waitSearcher=true)

[16:18:38.448] {pool-2-thread-1} end_commit_flush

[16:18:44.375] {pool-2-thread-1} start 
commit(optimize=false,waitFlush=true,waitSearcher=true)

[16:18:47.016] {pool-2-thread-1} end_commit_flush

[16:18:47.154] {pool-2-thread-1} start 
commit(optimize=false,waitFlush=true,waitSearcher=true)

[16:18:47.287] {pool-2-thread-1} end_commit_flush

[16:18:50.399] {pool-2-thread-1} start 
commit(optimize=false,waitFlush=true,waitSearcher=true)

[16:18:51.283] {pool-2-thread-1} end_commit_flush

[16:19:13.782] {pool-2-thread-1} start 
commit(optimize=false,waitFlush=true,waitSearcher=true)

[16:19:14.664] {pool-2-thread-1} end_commit_flush

[16:19:15.081] {pool-2-thread-1} start 
commit(optimize=false,waitFlush=true,waitSearcher=true)

[16:19:15.215] {pool-2-thread-1} end_commit_flush

[16:19:15.357] {pool-2-thread-1} start 
commit(optimize=false,waitFlush=true,waitSearcher=true)

[16:19:15.955] {pool-2-thread-1} end_commit_flush

[16:19:16.421] {pool-2-thread-1} start 
commit(optimize=false,waitFlush=true,waitSearcher=true)

[16:19:19.791] {pool-2-thread-1} end_commit_flush

[16:19:50.594] {pool-2-thread-1} start 
commit(optimize=false,waitFlush=true,waitSearcher=true)

[16:19:52.098] {pool-2-thread-1} end_commit_flush

[16:19:52.236] {pool-2-thread-1} start 
commit(optimize=false,waitFlush=true,waitSearcher=true)

[16:19:52.368] {pool-2-thread-1} end_commit_flush

[16:19:52.917] {pool-2-thread-1} start 
commit(optimize=false,waitFlush=true,waitSearcher=true)

[16:19:53.479] {pool-2-thread-1} end_commit_flush

[16:19:54.920] {pool-2-thread-1} start 
commit(optimize=false,waitFlush=true,waitSearcher=true)

[16:19:55.079] {pool-2-thread-1} end_commit_flush



additionally, in the solr admin page , the update handler reports as 
many autocommits as commits -

so i assume it is not some commit(); line lost in my code.


I actually get the feeling that the commits are triggered more and 
more often - with not-so-nice
influence on indexing speed over time. Restarting resin seems to get 
the commit rate to the original level.

Optimizing has no effect.
Is there some other parameter influencing autocommit?

Thank you very much.

Nickolai


























					Reply via email to