Re: Autocommit in SolrCloud with many shards

2020-06-18 Thread Bram Van Dam
he leader will be able to search doc2, but the >>> follower won’t until 60 seconds later. >>> >>> Best, >>> Erick >>> >>>> On Jun 17, 2020, at 5:36 AM, Bram Van Dam wrote: >>>> >>>> 'morning :-) >>>&g

Re: Autocommit in SolrCloud with many shards

2020-06-17 Thread Erick Erickson
der will be able to search doc2, but the >> follower won’t until 60 seconds later. >> >> Best, >> Erick >> >>> On Jun 17, 2020, at 5:36 AM, Bram Van Dam wrote: >>> >>> 'morning :-) >>> >>> I'm wondering how a

Re: Autocommit in SolrCloud with many shards

2020-06-17 Thread Bram Van Dam
the > follower won’t until 60 seconds later. > > Best, > Erick > >> On Jun 17, 2020, at 5:36 AM, Bram Van Dam wrote: >> >> 'morning :-) >> >> I'm wondering how autocommits work in Solr. >> >> Say I have a cluster with many nodes

Re: Autocommit in SolrCloud with many shards

2020-06-17 Thread Erick Erickson
lr. > > Say I have a cluster with many nodes and many colections with many > shards. If each collection's config has a hard autocommit configured > every minute, does that mean that SolrCloud (presumably the leader?) > will dish out commit requests to each node on that schedule? O

Autocommit in SolrCloud with many shards

2020-06-17 Thread Bram Van Dam
'morning :-) I'm wondering how autocommits work in Solr. Say I have a cluster with many nodes and many colections with many shards. If each collection's config has a hard autocommit configured every minute, does that mean that SolrCloud (presumably the leader?) will dish out com

Re: Rule of thumb for determining maxTime of AutoCommit

2020-02-27 Thread Dwane Hall
commit-in-sorlcloud/) Good luck, Dwane From: Emir Arnautović Sent: Thursday, 27 February 2020 9:23 PM To: solr-user@lucene.apache.org Subject: Re: Rule of thumb for determining maxTime of AutoCommit Hi Kaya, Since you do not have soft commits, you must have explicit commits somewhere since

Re: Rule of thumb for determining maxTime of AutoCommit

2020-02-27 Thread Emir Arnautović
>>> Another day, I had an error "exceeded limit of maxWarmingSearchers=2." >>> I know this error causes when multiple commits(which opens a new >> searcher) >>> are requested too frequently. >>> >>> As far as I read Solr wiki, it

Re: Rule of thumb for determining maxTime of AutoCommit

2020-02-26 Thread Kayak28
requested too frequently. > > > > As far as I read Solr wiki, it recommends for me to have more interval > > between each commit, and make commit frequency less. > > Using autoCommit, I would like to decrease the commit frequency, but I > am > > not sure how much I

Re: Rule of thumb for determining maxTime of AutoCommit

2020-02-26 Thread Emir Arnautović
t; I know this error causes when multiple commits(which opens a new searcher) > are requested too frequently. > > As far as I read Solr wiki, it recommends for me to have more interval > between each commit, and make commit frequency less. > Using autoCommit, I would like to decrease the c

Rule of thumb for determining maxTime of AutoCommit

2020-02-25 Thread Kayak28
ommit, and make commit frequency less. Using autoCommit, I would like to decrease the commit frequency, but I am not sure how much I should increase the value of maxTime in autoCommit? My current configuration is the following: ${solr.autoCommit.maxTime:15000} false

Re: merge policy & autocommit

2019-10-28 Thread Shawn Heisey
On 10/28/2019 7:23 AM, Danilo Tomasoni wrote: We have a solr instance with around 40MLN docs. In the bulk import phase we noticed a high IO and CPU load and it looks like it's related to autocommit because if I disable autocommit the load of the system is very low. I know that disa

merge policy & autocommit

2019-10-28 Thread Danilo Tomasoni
Hello all, We have a solr instance with around 40MLN docs. In the bulk import phase we noticed a high IO and CPU load and it looks like it's related to autocommit because if I disable autocommit the load of the system is very low. I know that disabling autocommit is not recommended, bu

Re: how to improve indexing using autocommit

2019-10-03 Thread Erick Erickson
that work, your should include the batch number to keep things separate. Another alternative is to not open new searchers. Your autocommit configuration has false. Be sure to set your autoSoftCommit to -1 in that case too. Then, at the end of your indexing run, to a hard commit externally (i.e

how to improve indexing using autocommit

2019-10-03 Thread babloorawat
how to make full use of auto-commit. Since during full import we are removing previously indexed documents we have set the auto-commit property as 630 . //commit after full indexing true How can I make use of autocommit without removing previously indexed documents till

Re: check softCommit , autocommit and hard commit count

2017-12-04 Thread Shawn Heisey
there no pending /update written to disk, does autocommit really fires at it's interval or is it going to be idle if nothing to write to disk..? In other way, suppose, I made a softcommit on 5th second and I made a hardcommit explicitly on 10th second, is it really going to happen an autocommi

Re: check softCommit , autocommit and hard commit count

2017-12-04 Thread Erick Erickson
chive isn't an issue here. Once I get a hold of > unique phrases in each logs that appends with each type of > commits(softcommit, autohardcommit,hardcommit), I can frame some metrics of > current traffic. > > Our current stack still maintains default autocommit config like

Re: check softCommit , autocommit and hard commit count

2017-12-04 Thread Puppy Linux Distros
with each type of commits(softcommit, autohardcommit,hardcommit), I can frame some metrics of current traffic. Our current stack still maintains default autocommit config like opensearcher=false and 15s period. Currently dont have softcommits enabled, however softcommits and hardcommits invokes

Re: check softCommit , autocommit and hard commit count

2017-12-03 Thread Puppy Linux Distros
Hello, Thanks Shawn. Can you provide command to find the total number of autocommits in the solr.log? On Thu, Nov 30, 2017 at 7:20 PM, Shawn Heisey wrote: > On 11/30/2017 4:36 AM, Puppy Linux Distros wrote: > >> I am trying to calculate the total number of softCommit , autocommit

Re: check softCommit , autocommit and hard commit count

2017-11-30 Thread Shawn Heisey
On 11/30/2017 4:36 AM, Puppy Linux Distros wrote: I am trying to calculate the total number of softCommit , autocommit and hard commit from the solr logs. Can you please check whether the below commands are correct ? Let me know how to find the total softcommit, hardcommit and autocommit from

check softCommit , autocommit and hard commit count

2017-11-30 Thread Puppy Linux Distros
Hi, I am trying to calculate the total number of softCommit , autocommit and hard commit from the solr logs. Can you please check whether the below commands are correct ? Let me know how to find the total softcommit, hardcommit and autocommit from the logs. *1. totalcommit=`cat $solrlogfile

Re: Data Import Handler - autoSoftCommit and autoCommit

2016-02-08 Thread Susheel Kumar
the data import handler to retrieve about 10 million records > during work hours every day of the week. We are using Clean = true, Commit > = true and Optimize = true. The entire process takes about 1 hour. > > What would be a good setting for autoCommit and autoSoftCommit? > > Thanks >

Re: Data Import Handler - autoSoftCommit and autoCommit

2016-02-08 Thread Rajesh Hazari
would be a good setting for autoCommit and autoSoftCommit? > > Thanks >

Data Import Handler - autoSoftCommit and autoCommit

2016-02-08 Thread Troy Edwards
We are running the data import handler to retrieve about 10 million records during work hours every day of the week. We are using Clean = true, Commit = true and Optimize = true. The entire process takes about 1 hour. What would be a good setting for autoCommit and autoSoftCommit? Thanks

Re: Autocommit, opensearchers and ingestion

2014-07-31 Thread rulinma
good -- View this message in context: http://lucene.472066.n3.nabble.com/Autocommit-opensearchers-and-ingestion-tp4119604p4150558.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: The way Autocommit works in solr - Wierd

2014-03-10 Thread Erick Erickson
ogram which reads a flat file, and adds document to solr >> using cloudsolrserver. >> And we index for every 1000 documents(bulk indexing). >> >> And the Autocommit setting of my application is: >> >> 10 >> false >> >> >&g

Re: The way Autocommit works in solr - Wierd

2014-03-10 Thread Furkan KAMACI
at file, and adds document to solr > using cloudsolrserver. > And we index for every 1000 documents(bulk indexing). > > And the Autocommit setting of my application is: > > 10 > false > > > So after every 100,000 documents are indexed, engine sho

The way Autocommit works in solr - Wierd

2014-03-10 Thread RadhaJayalakshmi
Hi, Brief Description of my application: We have a java program which reads a flat file, and adds document to solr using cloudsolrserver. And we index for every 1000 documents(bulk indexing). And the Autocommit setting of my application is: 10 false So after every 100,000

Re: Autocommit, opensearchers and ingestion

2014-02-26 Thread Mark Miller
On Feb 26, 2014, at 5:24 PM, Joel Cohen wrote: > he's told me that he's doing commits in his SolrJ code > every 1000 items (configurable). Does that override my Solr server settings? Yes. Even if you have configured autocommit - explicit commits are explicit commits that

Re: Autocommit, opensearchers and ingestion

2014-02-26 Thread Joel Cohen
Erickson wrote: > Gopal: I'm glad somebody noticed that blog! > > Joel: > For bulk loads it's a Good Thing to lengthen out > your soft autocommit interval. A lot. Every second > poor Solr is trying to open up a new searcher while > you're throwing lots of docume

Re: Autocommit, opensearchers and ingestion

2014-02-25 Thread Erick Erickson
Gopal: I'm glad somebody noticed that blog! Joel: For bulk loads it's a Good Thing to lengthen out your soft autocommit interval. A lot. Every second poor Solr is trying to open up a new searcher while you're throwing lots of documents at it. That's what's generatin

Re: Autocommit, opensearchers and ingestion

2014-02-25 Thread Gopal Patwa
This blog by Eric will help you to understand different commit option and transaction logs and it does provide some recommendation for ingestion process. http://searchhub.org/2013/08/23/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/ On Tue, Feb 25, 2014 at 11:40 AM, Furkan KA

Re: Autocommit, opensearchers and ingestion

2014-02-25 Thread Furkan KAMACI
Hi; You should read here: http://wiki.apache.org/solr/FAQ#What_does_.22exceeded_limit_of_maxWarmingSearchers.3DX.22_mean.3F On the other hand do you have 4 Zookeeper instances as a quorum? Thanks; Furkan KAMACI 2014-02-25 20:31 GMT+02:00 Joel Cohen : > Hi all, > > I'm working with Solr 4.6.1

Autocommit, opensearchers and ingestion

2014-02-25 Thread Joel Cohen
Hi all, I'm working with Solr 4.6.1 and I'm trying to tune my ingestion process. The ingestion runs a big DB query and then does some ETL on it and inserts via SolrJ. I have a 4 node cluster with 1 shard per node running in Tomcat with -Xmx=4096M. Each node has a separate instance of Zookeeper on

autoCommit and autoSoftCommit

2013-08-26 Thread Bryan Bende
, but none of the results have the stored fields present. As a test I also tried modifying the autoCommit to add maxDocs like this: 100 6 false It seems like with this configuration something different happens... if I send in 150 docs then the first 100 will show up correctly through

Re: solr 4.3, autocommit, maxdocs

2013-07-15 Thread Jonathan Rochkind
Ah, thanks for this explanation. Although I don't entirely understand it, I am glad there is an expected explanation! This Solr instance is actually set up to be a replication master. It never gets searched itself, it just replicates to slaves that get searched. Perhaps some time in the past

Re: solr 4.3, autocommit, maxdocs

2013-07-15 Thread Jason Hellman
Jonathan, Please note the openSearcher=false part of your configuration. This is why you don't see documents. The commits are occurring, and being written to segments on disk, but they are not visible to the search engine because a Solr searcher class has not opened them for visibility. You

solr 4.3, autocommit, maxdocs

2013-07-15 Thread Jonathan Rochkind
I have a solr 4.3 instance I am in the process of standing up. It started out with an empty index. I have in it's solrconfig.xml, 10 false I have an index process running, that has currently added around 400k documents to Solr. I had expected that a 'commit'

Re: Apparent odd interaction between autoCommit values and indexing ram buffer

2013-06-19 Thread Shawn Heisey
On 6/19/2013 10:38 AM, Shawn Heisey wrote: Looking at the numDocs for each segment, here's what I think is happening: The autoCommit kicks in after the first 25000 docs (25002 to be precise), but the ram buffer isn't emptied. The next 3339 documents get indexed, at which point the

Apparent odd interaction between autoCommit values and indexing ram buffer

2013-06-19 Thread Shawn Heisey
I've run into something a little odd that's been happening for a while. The apparent symptoms: Two index segments are created every time an autoCommit (hard, not soft) happens during a DIH full-import. Here's the directory listing from the first few minutes of importing,

Re: upgrading 1hr autoCommit behavior

2013-06-12 Thread Erick Erickson
On Tue, Jun 11, 2013 at 8:39 PM, Chris Hostetter wrote: > > : > However, we are wondering how to best setup autoCommit/autoSoftCommit on > : > masters to preserve the old behavior. It seems that setting autoCommit to > : > 1hr (openSearcher=true) without any autoSoftCommit pre

Re: upgrading 1hr autoCommit behavior

2013-06-11 Thread Chris Hostetter
: > However, we are wondering how to best setup autoCommit/autoSoftCommit on : > masters to preserve the old behavior. It seems that setting autoCommit to : > 1hr (openSearcher=true) without any autoSoftCommit preserves our previous : > setup - is this correct? Wil the transact

Re: upgrading 1hr autoCommit behavior

2013-06-11 Thread Shawn Heisey
On 6/11/2013 4:07 PM, Michael Tsadikov wrote: Thanks for the quick reply, Shawn. I am less worried about long transaction-log replays after crashes because in normal life (long uptimes, rare orderly restarts) this should never happen, and even if it does, uses will not be affected. I am more wo

Re: upgrading 1hr autoCommit behavior

2013-06-11 Thread Michael Tsadikov
pgrading our Solr 3.1 distributed masters/slaves setup to 4.3. >> >> In 3.1 we used autoCommit every hour on masters, each commit is replicated >> to slaves, and all searches are done on slaves. 1hr visibility is ok - we >> don't need NRT. >> >> In 4.3 we

Re: upgrading 1hr autoCommit behavior

2013-06-11 Thread Shawn Heisey
On 6/11/2013 3:45 PM, Michael Tsadikov wrote: We're upgrading our Solr 3.1 distributed masters/slaves setup to 4.3. In 3.1 we used autoCommit every hour on masters, each commit is replicated to slaves, and all searches are done on slaves. 1hr visibility is ok - we don't need NRT.

upgrading 1hr autoCommit behavior

2013-06-11 Thread Michael Tsadikov
We're upgrading our Solr 3.1 distributed masters/slaves setup to 4.3. In 3.1 we used autoCommit every hour on masters, each commit is replicated to slaves, and all searches are done on slaves. 1hr visibility is ok - we don't need NRT. In 4.3 we enable transaction logs, for durability

Re: Ho to adjust maxDocs and maxTime for autoCommit?

2013-05-16 Thread Jack Krupansky
Ho to adjust maxDocs and maxTime for autoCommit? I will start my pre-production step soon. How can I adjust maxDocs and maxTime for autoCommit? What do you suggest for me to adjust that parameters?

Re: Ho to adjust maxDocs and maxTime for autoCommit?

2013-05-16 Thread Shawn Heisey
On 5/16/2013 9:36 AM, Furkan KAMACI wrote: I will start my pre-production step soon. How can I adjust maxDocs and maxTime for autoCommit? What do you suggest for me to adjust that parameters? Change the numbers for those settings in your solrconfig.xml. Look at the example solrconfig.xml

Ho to adjust maxDocs and maxTime for autoCommit?

2013-05-16 Thread Furkan KAMACI
I will start my pre-production step soon. How can I adjust maxDocs and maxTime for autoCommit? What do you suggest for me to adjust that parameters?

Re: Autocommit and replication have been slowing down

2013-04-24 Thread gustavonasu
Hi Shawn, Thanks for the lesson! I really appreciate your help. I'll figure out a way to use that knowledge to solve my problem. Best Regards -- View this message in context: http://lucene.472066.n3.nabble.com/Autocommit-and-replication-have-been-slowing-down-tp4058361p4058584.html

Re: Autocommit and replication have been slowing down

2013-04-23 Thread Shawn Heisey
On 4/23/2013 3:44 PM, gustavonasu wrote: If I understand well the autoWarmCount is the number of elements used from the cache for new searches. I guess that this isn't the problem because after the commit property increases on the "UPDATE HANDLERS" (admin UI) I can see the new docs in the searche

Re: Autocommit and replication have been slowing down

2013-04-23 Thread gustavonasu
ge in context: http://lucene.472066.n3.nabble.com/Autocommit-and-replication-have-been-slowing-down-tp4058361p4058459.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: Autocommit and replication have been slowing down

2013-04-23 Thread Shawn Heisey
On 4/23/2013 11:27 AM, gustavonasu wrote: We migrated recently from Solr 1.4 to 3.6.1. In the new version we have noticed that after some hours (around 8) the autocommit is taking more time to be executed. In the new version we have noticed that after some hours the autocommit is

Autocommit and replication have been slowing down

2013-04-23 Thread gustavonasu
Hi, We migrated recently from Solr 1.4 to 3.6.1. In the new version we have noticed that after some hours (around 8) the autocommit is taking more time to be executed. In the new version we have noticed that after some hours the autocommit is taking more time to be executed. We

Re: Transient commit errors during autocommit

2012-10-19 Thread Casey Callendrello
error happens during an autocommit, the error does not cause a safe rollback or notify the user there was a problem. Instead, there is a write lock failure and Solr has to be restarted. It run fine after restart. Is this a known problem? Is it fixable? Is it unit-test-able?

Re: Solr Replication and Autocommit

2012-09-27 Thread Aleksey Vorona
orona wrote: Hi, I remember having some issues with replication and autocommit previously. But now we are using Solr 3.6.1. Are there any known issues or any other reasons to avoid autocommit while using replication? I guess not, just want confirmation from someone confident and competent. -- Aleksey

Re: Solr Replication and Autocommit

2012-09-27 Thread Erick Erickson
I'll echo Otis, nothing comes to mind... Unless you were indexing stuff to the _slaves_, which you should never do, now or in the past Erick On Thu, Sep 27, 2012 at 12:00 AM, Aleksey Vorona wrote: > Hi, > > I remember having some issues with replication and autocommit previou

Re: Solr Replication and Autocommit

2012-09-26 Thread Otis Gospodnetic
No issues that I know of, never encountered such an issue with any if our clients using Solr. Otis -- Performance Monitoring - http://sematext.com/spm On Sep 27, 2012 12:00 AM, "Aleksey Vorona" wrote: > Hi, > > I remember having some issues with replication and autocommit pr

Solr Replication and Autocommit

2012-09-26 Thread Aleksey Vorona
Hi, I remember having some issues with replication and autocommit previously. But now we are using Solr 3.6.1. Are there any known issues or any other reasons to avoid autocommit while using replication? I guess not, just want confirmation from someone confident and competent. -- Aleksey

Re: SOLR 4 autocommit - is it working as I think it should?

2012-04-11 Thread vybe3142
Thanks, makes perfect sense -- View this message in context: http://lucene.472066.n3.nabble.com/SOLR-4-autocommit-is-it-working-as-I-think-it-should-tp3903135p3903353.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: SOLR 4 autocommit - is it working as I think it should?

2012-04-11 Thread Yonik Seeley
On Wed, Apr 11, 2012 at 12:58 PM, vybe3142 wrote: > This morning, I've been looking at the autocommit functionality as defined > in solrconfig.xml. By default, it appears that it should kick in 15 seconds > after a new document has been added. I do see this event triggered via the

SOLR 4 autocommit - is it working as I think it should?

2012-04-11 Thread vybe3142
I've gotten past most of my initial hurdles with SOLR, with some useful suggestions from this group. Thank You. On to tweaking. This morning, I've been looking at the autocommit functionality as defined in solrconfig.xml. By default, it appears that it should kick in 15 seconds a

Autocommit woes

2011-12-08 Thread Isan Fulia
Hi All, My autocommit settings are max docs - 1000 max time - 86 secs We have put newrelic agent so as to monitor our solr performance. In that we see a continous curve for autocommit. It is as good as autocommit is continuously being fired. Is it that if Autocommit for certain documents takes

RE: Autocommit & Index Size

2011-12-06 Thread Husain, Yavar
, 2011 12:00 AM To: solr-user@lucene.apache.org Subject: Re: Autocommit & Index Size On 12/6/2011 1:01 AM, Husain, Yavar wrote: > In solrconfig.xml I was experimenting with Indexing Performance. When I set > the maxDocs (in autoCommit) to say 1 documents the index size is double >

Re: Autocommit & Index Size

2011-12-06 Thread Shawn Heisey
On 12/6/2011 1:01 AM, Husain, Yavar wrote: In solrconfig.xml I was experimenting with Indexing Performance. When I set the maxDocs (in autoCommit) to say 1 documents the index size is double to if I just dont use autoCommit (i.e. keep it commented, i.e commit at the end only after adding

Autocommit & Index Size

2011-12-06 Thread Husain, Yavar
In solrconfig.xml I was experimenting with Indexing Performance. When I set the maxDocs (in autoCommit) to say 1 documents the index size is double to if I just dont use autoCommit (i.e. keep it commented, i.e commit at the end only after adding documents). Does autoCommit affect the index

Re: questions about autocommit & committing documents

2011-10-27 Thread darul
ext: http://lucene.472066.n3.nabble.com/questions-about-autocommit-committing-documents-tp1582487p3459089.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: questions about autocommit & committing documents

2011-10-26 Thread Erick Erickson
472066.n3.nabble.com/questions-about-autocommit-committing-documents-tp1582487p3450794.html > Sent from the Solr - User mailing list archive at Nabble.com. >

Re: questions about autocommit & committing documents

2011-10-25 Thread darul
I was not sure thank you. -- View this message in context: http://lucene.472066.n3.nabble.com/questions-about-autocommit-committing-documents-tp1582487p3450794.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: questions about autocommit & committing documents

2011-10-25 Thread Mark Miller
It's not 'mandatory', but it makes no sense to keep it. Even without autocommit, committing after every doc add is horribly inefficient. On Oct 25, 2011, at 9:45 AM, darul wrote: > Well until now I was using SolrJ API to commit() (for each document added...) > changes bu

Re: questions about autocommit & committing documents

2011-10-25 Thread darul
Well until now I was using SolrJ API to commit() (for each document added...) changes but wonder in case of a production deployment it was not a best solution to use AutoCommit feature instead. With AutoCommit parameters, is it mandatory to remove commit() instruction called on

Re: questions about autocommit & committing documents

2011-10-23 Thread Erick Erickson
nothing for a minute. One commit of 999 docs happens because of maxtime... But I have to ask, "why do you care"? What high level problem are you trying to handle? Best Erick On Sun, Oct 23, 2011 at 3:03 PM, darul wrote: > May someone explain me different use case when both or only o

Re: questions about autocommit & committing documents

2011-10-23 Thread darul
May someone explain me different use case when both or only one AutoCommit parameters is filled ? I really need to understand it. For example with these configurations : 1 or 1000 or 1 1000 Thanks to everyone -- View this message in context: http

Re: questions about autocommit & committing documents

2011-10-22 Thread darul
However, you can do a commit explicitly, if you want to do so. -- View this message in context: http://lucene.472066.n3.nabble.com/questions-about-autocommit-committing-documents-tp1582487p3443838.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: RamBufferSize and AutoCommit

2011-03-29 Thread Isan Fulia
RAMBufferSizeMB. > > should set value less than it. if you setRAMBufferSizeMB to 2700m and > > the other 3 conditions are not > > triggered, I think it will hit OOM exception. > > > > 2011/3/28 Isan Fulia : > >> Hi all , > >> > >> I woul

Re: RamBufferSize and AutoCommit

2011-03-28 Thread Erick Erickson
than it. if you setRAMBufferSizeMB to 2700m and > the other 3 conditions are not > triggered, I think it will hit OOM exception. > > 2011/3/28 Isan Fulia : >> Hi all , >> >> I would like to know is there any relation between autocommit and >> rambuffersize.

Re: RamBufferSize and AutoCommit

2011-03-28 Thread Li Li
will hit OOM exception. 2011/3/28 Isan Fulia : > Hi all , > > I would like to know is there any relation between autocommit and > rambuffersize. > My solr config does not  contain rambuffersize which mean its > deault(32mb).Autocommit setting are after 500 docs or 80 sec > which

RamBufferSize and AutoCommit

2011-03-28 Thread Isan Fulia
Hi all , I would like to know is there any relation between autocommit and rambuffersize. My solr config does not contain rambuffersize which mean its deault(32mb).Autocommit setting are after 500 docs or 80 sec whichever is first. Solr starts with Xmx 2700M .Total Ram is 4 GB. Does the

RE: autocommit commented out -- what is the default?

2010-12-04 Thread Jonathan Rochkind
It means they never happen automatically, added documents wont' be committed until you send a commit to solr. Jonathan From: Brian Whitman [br...@echonest.com] Sent: Saturday, December 04, 2010 10:36 AM To: solr-user@lucene.apache.org Subject: autoc

Re: autocommit commented out -- what is the default?

2010-12-04 Thread Yonik Seeley
On Sat, Dec 4, 2010 at 10:36 AM, Brian Whitman wrote: > Hi, if you comment out the block in solrconfig.xml > > > > Does this mean that (a) commits never happen automatically or (b) some > default autocommit is applied? Commented out means they never happen automatically (i.e

autocommit commented out -- what is the default?

2010-12-04 Thread Brian Whitman
Hi, if you comment out the block in solrconfig.xml Does this mean that (a) commits never happen automatically or (b) some default autocommit is applied?

Re: questions about autocommit & committing documents

2010-09-26 Thread MitchK
mit explicitly. With the post.jar you need to set either dcommit=yes or to append "", I think. Hope this helps. - Mitch -- View this message in context: http://lucene.472066.n3.nabble.com/questions-about-autocommit-committing-documents-tp1582487p1582846.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: questions about autocommit & committing documents

2010-09-26 Thread Andy
Thanks Mitch. How do I do an explicit commit? Andy --- On Sun, 9/26/10, MitchK wrote: > From: MitchK > Subject: Re: questions about autocommit & committing documents > To: solr-user@lucene.apache.org > Date: Sunday, September 26, 2010, 4:13 AM > > Hi Andy,

Re: questions about autocommit & committing documents

2010-09-26 Thread MitchK
if there was no commit to it. Kind regards, Mitch -- View this message in context: http://lucene.472066.n3.nabble.com/questions-about-autocommit-committing-documents-tp1582487p1582676.html Sent from the Solr - User mailing list archive at Nabble.com.

questions about autocommit & committing documents

2010-09-25 Thread Andy
In the example solrconfig.xml that comes with Solr, the autocommit section: 1 1000 has been commented out. - With commented out, does it mean that every new document indexed to Solr is being auto-committed individually? Or that they are not being auto-committed at all? - If I

Re: Multiple updatehandlers in solr, different autocommit settings

2010-08-12 Thread Chris Hostetter
: : I'm trying to set different autocommit settings to 2 separate request : handlers...I would like a requesthandler to use an update handler and a : second requesthandler use another update handler... : : can I have more than one update handler in the same solrconfig? : how can I config

Multiple updatehandlers in solr, different autocommit settings

2010-08-06 Thread Turbo
Hi all, I'm trying to set different autocommit settings to 2 separate request handlers...I would like a requesthandler to use an update handler and a second requesthandler use another update handler... can I have more than one update handler in the same solrconfig? how can I config

Re: Autocommit not happening

2010-07-23 Thread John DeRosa
I'll see you, and raise. My solrconfig.xml wasn't being copied to the server by the deployment script. On Jul 23, 2010, at 3:26 PM, Jay Luker wrote: > For the sake of any future googlers I'll report my own clueless but > thankfully brief struggle with autocommit. > >

Re: Autocommit not happening

2010-07-23 Thread Jay Luker
For the sake of any future googlers I'll report my own clueless but thankfully brief struggle with autocommit. There are two parts to the story: Part One is where I realize my config was not contained within my . In Part Two I realized I had typed "" rather than "". --

Re: Autocommit not happening

2010-07-23 Thread John DeRosa
On Jul 23, 2010, at 9:37 AM, John DeRosa wrote: > Hi! I'm a Solr newbie, and I don't understand why autocommits aren't > happening in my Solr installation. > [snip] "Never mind"... I have discovered my boneheaded mistake. It's so silly, I wish I could retract my question from the archives.

Re: Autocommit not happening

2010-07-23 Thread John DeRosa
010 9:36:32 PM org.apache.solr.update.DirectUpdateHandler2$CommitTracker INFO: AutoCommit: disabled What's stepping in and disabling autocommit? John

Autocommit not happening

2010-07-23 Thread John DeRosa
r-common solr-tomcat tomcat6-admin" My solrconfig.xml has: 1 1 My code can add documents just fine. But after 12 hours, autocommit has never happened! Here's what I see on my Solr Admin pages: CORE: name: core class: version:1.0 de

Questions on autocommit and optimize operations

2010-04-23 Thread dipti khullar
Hi Solr Gurus We are thinking about optimizing our production master slave solr setup, just wanted to poll the group on following questions: 1. Currently we are using autocommit feature with setting of 50 docs and 5 mins. Now the requirement is to reduce this time. So we are analyzing the

Re: DIH import "out of memory" problem (batchSize and autoCommit not working)

2009-09-22 Thread Shalin Shekhar Mangar
On Tue, Sep 22, 2009 at 3:00 PM, Steve Sun wrote: > Done. > http://issues.apache.org/jira/browse/SOLR-1450 > > This is fixed in trunk now. Thanks Steve! -- Regards, Shalin Shekhar Mangar.

Re: DIH import "out of memory" problem (batchSize and autoCommit not working)

2009-09-22 Thread Steve Sun
took a look into > > DIH's source code and found that there's actually a bug in there. When > JDBC > > driver is placed in /lib (as instructed by DIHQuickStart page > of > > Solr wiki), but not in tomcat's lib directory, the JDBC connection will > not &g

Re: DIH import "out of memory" problem (batchSize and autoCommit not working)

2009-09-22 Thread Shalin Shekhar Mangar
re's actually a bug in there. When JDBC > driver is placed in /lib (as instructed by DIHQuickStart page of > Solr wiki), but not in tomcat's lib directory, the JDBC connection will not > be configured as specified in the DIH configuration at all. Attributes like > autoCommit, re

DIH import "out of memory" problem (batchSize and autoCommit not working)

2009-09-22 Thread Steve Sun
ucted by DIHQuickStart page of Solr wiki), but not in tomcat's lib directory, the JDBC connection will not be configured as specified in the DIH configuration at all. Attributes like autoCommit, readOnly and batchSize will be ignored. The fix is simple, have attached my patch. (contrib/dataimp

Re: multi-core, autocommit and resource use

2009-06-18 Thread Yonik Seeley
On Thu, Jun 18, 2009 at 8:30 PM, Peter Wolanin wrote: > So for now would it make sense to spread out the autocommit times for > the different cores? Sure. You might also consider using commitWithin (solr 1.4) when updating the index - then you could either send the updates at slightly dif

Re: multi-core, autocommit and resource use

2009-06-18 Thread Peter Wolanin
So for now would it make sense to spread out the autocommit times for the different cores? Thanks. -Peter On Thu, Jun 18, 2009 at 7:07 PM, Yonik Seeley wrote: > On Thu, Jun 18, 2009 at 4:27 PM, Peter Wolanin > wrote: >>  I think I understand >> that all the pending changes a

Re: multi-core, autocommit and resource use

2009-06-18 Thread Yonik Seeley
On Thu, Jun 18, 2009 at 4:27 PM, Peter Wolanin wrote: >  I think I understand > that all the pending changes are on disk already, so the "commit" that > happens when the time is up is really just opening new searchers that > include the added documents. Only some of the pending changes may be on d

multi-core, autocommit and resource use

2009-06-18 Thread Peter Wolanin
A question for anyone familiar with the details of the time-based autocommit mechanism in Solr: if I am running several core on the same server and send updates to each core at the same time, what happens? If all the cores have their autocommit time run out at the same time, will every core try

  1   2   >