Re: Solr 4 near real time commit

2011-12-08 Thread yu shen
[?], thanks. Try in a minute. Spark 2011/12/9 Mark Miller > pseudo code: > >SolrServer client = ... > >UpdateRequest ureq = new UpdateRequest(); >ureq.add(doc); >ureq.setParam("param", "value"); >ureq.setAction( UpdateRequest.ACTION.COMMIT, waitFlush, waitSearcher); >ure

Re: Solr 4 near real time commit

2011-12-08 Thread Mark Miller
pseudo code: SolrServer client = ... UpdateRequest ureq = new UpdateRequest(); ureq.add(doc); ureq.setParam("param", "value"); ureq.setAction( UpdateRequest.ACTION.COMMIT, waitFlush, waitSearcher); ureq.process(client); It will say setAction is deprecated, but internal So

Re: Solr 4 near real time commit

2011-12-08 Thread yu shen
I will try to use your methods. 2011/12/9 Sharath Jagannath > Kinda off topic why not directly use /solr/update/json?commitWithin=3000, > ofcourse with autocommit.maxtime and autoSoftCommit in configs. > I am not using the java client but send the docs using the http client. > It is doing fairly

Re: Solr 4 near real time commit

2011-12-08 Thread yu shen
Hi Mark, My question inline. 2011/12/9 Mark Miller > I don't know about DataImportHandler. > > But the whole point of AutoCommit and AutoSoftCommit is that they happen > 'Auto'matically. You couldn't measure how long they take from solrj, and if > you are using solrj to do the commit, it has no

Re: Solr 4 near real time commit

2011-12-08 Thread Sharath Jagannath
Kinda off topic why not directly use /solr/update/json?commitWithin=3000, ofcourse with autocommit.maxtime and autoSoftCommit in configs. I am not using the java client but send the docs using the http client. It is doing fairly good for me. Zoie is another option worth looking at. Had very good e

Re: Solr 4 near real time commit

2011-12-08 Thread Mark Miller
I don't know about DataImportHandler. But the whole point of AutoCommit and AutoSoftCommit is that they happen 'Auto'matically. You couldn't measure how long they take from solrj, and if you are using solrj to do the commit, it has nothing to do with auto commit. If you want to commit from solrj

Re: Solr 4 near real time commit

2011-12-08 Thread yu shen
That sounds like a good idea. I will check my schema configuration, and see what I can do. Thanks Mark and Siva for all the information, and see what I can do, and update the result here. Spark 2011/12/9 Siva Kommuri > Hi Spark, > > Is it possible to store a modified time on the database recor

Re: Solr 4 near real time commit

2011-12-08 Thread Siva Kommuri
Hi Spark, Is it possible to store a modified time on the database record which would help you to get the updated documents? That way you can avoid reindexing the documents that have not been updated. Best wishes, Siva on 3GS On Dec 8, 2011, at 15:40, yu shen wrote: > Hi Mark, > > I did not

Re: Solr 4 near real time commit

2011-12-08 Thread yu shen
Hi Mark, I did not fully catch you. I have a dataImportHandler which import data from a database. Are you suggesting autoCommit/autoSoftCommit can automaticall pull data from database using dataImportHandler? Otherwise, how can I commit user modification of search result from my web page back to

Re: Solr 4 near real time commit

2011-12-08 Thread Mark Miller
Lets please keep the conversation on list. If you are using auto commit and soft auto commit, it makes no sense to be committing with solrj. The commits happen automatically. You couldn't measure them with solrj. - mark On Dec 8, 2011, at 12:44 AM, yu shen wrote: > Thanks for the help. > > F

Re: Solr 4 near real time commit

2011-12-07 Thread Mark Miller
Hmmm...that sounds pretty odd... How are you measuring the commit time? You likely want to turn off any caches, as they will be expired every second, but that should not cause this... I can try and duplicate your setup tomorrow and see what i can spot. - Mark On Dec 7, 2011, at 8:13 PM, yu sh

Re: Solr 4 near real time commit

2011-12-07 Thread yu shen
Hi Mark, and all I now use commit configuration exactly as below: 10 1000 But the commit time takes about 60 seconds. I have around 120 - 130 documents in my server. And each day, the number will increase about 6000. My symptom is if solr server is just s

Re: Solr 4 near real time commit

2011-12-07 Thread yu shen
Thanks for the correction, I did not notice that [?] Spark 2011/12/7 Mark Miller > Well, if that is exactly what you put, it's wrong. That second one should > be softAutoCommit. > > On Wednesday, December 7, 2011, yu shen wrote: > > Hi All, > > > > I tried using solr 4 nightly build: apache-s

Re: Solr 4 near real time commit

2011-12-07 Thread Mark Miller
Well, if that is exactly what you put, it's wrong. That second one should be softAutoCommit. On Wednesday, December 7, 2011, yu shen wrote: > Hi All, > > I tried using solr 4 nightly build: apache-solr-4.0-2011-12-06_08-52-46. > And try to enable autoSoftCommit like below in solrconfig.xml > >

Solr 4 near real time commit

2011-12-07 Thread yu shen
Hi All, I tried using solr 4 nightly build: apache-solr-4.0-2011-12-06_08-52-46. And try to enable autoSoftCommit like below in solrconfig.xml 10 1000 I try to add a document to this solr instance using solrj client in the nightly build. I do saw a commit time boost. Single docume