Re: Doing url search in solr is slow

2012-01-10 Thread yu shen
DebugComponent. The query I use did not have any prepended asterisk. (Excuse me if the context description is still not complete enought) Is there any other margin to boost the query performance? Spark 2012/1/10 yu shen > Hi Erick, > > I only added debugyQuery=on to the url, and did n

Re: Doing url search in solr is slow

2012-01-09 Thread yu shen
s what you > want, but that is determined by your problem space. You might > spend some time with admin/analysis to see the effects of > various analysis chains. "string" is used when you want no > tokenization, no case transformations etc. > > Best > Erick > > On Mon,

Re: Doing url search in solr is slow

2012-01-09 Thread yu shen
Erickson > Yu Shen & Arian: > > We can't help much without more information. In particular, how are > the fields in question analyzed? What is the result of looking > at the admin/analysis page? What do you get when you > attach &debugQuery=on to the quer

Doing url search in solr is slow

2012-01-08 Thread yu shen
Hi, My solr document has up to 20 fields, containing data from product name, date, url etc. The volume of documents is around 1.5m. My symptom is when doing url search like [ url:*www.someurl.com* referal_url:*www.someurl.com* page_url:*www.someurl.com*] will get a extraordinary long response ti

Re: Solr 4 near real time commit

2011-12-08 Thread yu shen
quest.ACTION.COMMIT, waitFlush, waitSearcher); >ureq.process(client); > > It will say setAction is deprecated, but internal SolrServer code still > uses it, so meh. > > On Dec 8, 2011, at 7:49 PM, yu shen wrote: > > > Hi Mark, > > > > My question inline. > > &

Re: Solr 4 near real time commit

2011-12-08 Thread yu shen
It is doing fairly good for me. > > Zoie is another option worth looking at. Had very good experience with it > as well. > > Thanks, > Sharath > > > > On Thu, Dec 8, 2011 at 4:23 PM, yu shen wrote: > > > That sounds like a good idea. I will check my schema configu

Re: Solr 4 near real time commit

2011-12-08 Thread yu shen
SOLR_SERVER.add(solrDocs); SOLR_SERVER.commit(); > On Thu, Dec 8, 2011 at 7:23 PM, yu shen wrote: > > > That sounds like a good idea. I will check my schema configuration, and > see > > what I can do. > > > > Thanks Mark and Siva for all the information, and see

Re: Solr 4 near real time commit

2011-12-08 Thread yu shen
database record which would > help you to get the updated documents? > > That way you can avoid reindexing the documents that have not been updated. > > Best wishes, > Siva on 3GS > > On Dec 8, 2011, at 15:40, yu shen wrote: > > > Hi Mark, > > > > I did not

Re: Solr 4 near real time commit

2011-12-08 Thread yu shen
> > On Dec 8, 2011, at 12:44 AM, yu shen wrote: > > > Thanks for the help. > > > > For the measurement part, I record system time before commit using > solrj, and also after. And calculate the delta. > > > > >>> System.currentTimeMillis() - startTim

Re: Solr 4 near real time commit

2011-12-07 Thread yu shen
started, the commit time is about 3-5 seconds. But after one day time, the commit time increase substantially to about 1 min. Do I missed anything or had any mis-configuration? Thanks very much for any help in advance. Spark 2011/12/7 yu shen > Thanks for the correction, I did not notice t

Re: Solr 4 near real time commit

2011-12-07 Thread yu shen
Thanks for the correction, I did not notice that [?] Spark 2011/12/7 Mark Miller > Well, if that is exactly what you put, it's wrong. That second one should > be softAutoCommit. > > On Wednesday, December 7, 2011, yu shen wrote: > > Hi All, > > > > I tried

Solr 4 near real time commit

2011-12-07 Thread yu shen
Hi All, I tried using solr 4 nightly build: apache-solr-4.0-2011-12-06_08-52-46. And try to enable autoSoftCommit like below in solrconfig.xml 10 1000 I try to add a document to this solr instance using solrj client in the nightly build. I do saw a commit time boost. Single docume

Re: [Announce] Solr-RA, Solr with RankingAlgorithm

2011-12-06 Thread yu shen
ithm.tgels.**org <http://rankingalgorithm.tgels.org> > > > On 12/6/2011 1:20 AM, yu shen wrote: > >> Hi Nagendra, >> >> I tried to use solr-nrt-ra-3.4, while the dataimporthandler does not work. >> The error message is: >> >> INFO:

Re: [Announce] Solr-RA, Solr with RankingAlgorithm

2011-12-06 Thread yu shen
Hi Nagendra, I tried to use solr-nrt-ra-3.4, while the dataimporthandler does not work. The error message is: INFO: created /dataimport: org.apache.solr.handler.dataimport.DataImportHandler Dec 6, 2011 1:16:18 AM org.apache.solr.common.SolrException log SEVERE: java.lang.IncompatibleClassChangeEr

Re: Solr real time update

2011-11-23 Thread yu shen
solrconfig.xml: > > http://solr-ra.tgels.org/wiki/**en/Near_Real_Time_Search<http://solr-ra.tgels.org/wiki/en/Near_Real_Time_Search> > > Regards, > > - Nagendra Nagrajayya > http://solr-ra.tgels.org/ > http://rankingalgorithm.tgels.**org/ <http://rankingalgorithm.tgels.

Re: Solr real time update

2011-11-22 Thread yu shen
hm.tgels.org/> > > > On 11/21/2011 9:47 PM, yu shen wrote: > >> Hi All, >> >> After some study, I used below snippet. Seems the documents is updated, >> while still takes a long time. Feels like the parameter does not take >> effect. An

Re: Solr real time update

2011-11-21 Thread yu shen
); req.setParam("commitWithin", "5000"); req.setAction(AbstractUpdateRequest.ACTION.COMMIT, true, true); req.process(SOLR_SERVER); 2011/11/22 yu shen > Hi All, > > I try to do a 'nearly real time update' to solr. My solr version is >

Solr real time update

2011-11-21 Thread yu shen
Hi All, I try to do a 'nearly real time update' to solr. My solr version is 1.4.1. I read this solr CommentWithin wiki, and a related threadmostly on the difficulty to do

Re: [Profiling] How to profile/tune Solr server

2011-11-04 Thread yu shen
Thank you for the information. 2011/11/5 yu shen > Really helpful, thanks so much. > > Spark > > 2011/11/4 > > Hi Spark, >> >> 2009 there was a monitor from lucidimagination: >> >> http://www.lucidimagination.com/about/news/releases/lucid-imagin

Re: [Profiling] How to profile/tune Solr server

2011-11-04 Thread yu shen
and-lucidworks-with-zabbix/ > > In your case any profiler would do, but if you find out a Profiler with > solr-specific default-filter let me know. > > > > Best regrads > Karsten > > P.S. eMail in context > > http://lucene.472066.n3.nabble.com/Profiling-How-to-p

Re: [Profiling] How to profile/tune Solr server

2011-10-31 Thread yu shen
ow? > > -- > Jan Høydahl, search solution architect > Cominvent AS - www.cominvent.com > Solr Training - www.solrtraining.com > > On 31. okt. 2011, at 04:59, yu shen wrote: > > > Hi All, > > > > I am a solr newbie. I find solr documents easy to access an

[Profiling] How to profile/tune Solr server

2011-10-30 Thread yu shen
Hi All, I am a solr newbie. I find solr documents easy to access and use, which is really good thing. While my problem is I did not find a solr home grown profiling/monitoring tool. I set up the server as a multi-core server, each core has approximately 2GB index. And I need to update solr and re

Re: DataImportHanlder - Multiple entities will step into each other

2011-01-04 Thread yu shen
ument with a that is already in the index, > the old version of the document is removed and the new one added. > > But is not required at all and no field in solr is required > unless > required="true" is specified. > > Best > Erick > > On Tue, Jan 4, 2011

Re: DataImportHanlder - Multiple entities will step into each other

2011-01-04 Thread yu shen
> This SQL syntax should do it: "select id, field as table_id, field". > > On Tue, Jan 4, 2011 at 5:59 AM, yu shen wrote: > > Thanks for the prompt reply. Let me try. Delete is not a big deal for the > > moment. > > > > 2011/1/4 Matti Oi

Re: DataImportHanlder - Multiple entities will step into each other

2011-01-04 Thread yu shen
delete to the > index. It seems that TemplateTransformer is not used when performing > delete so delete by id doesn't work. > > > > 2011/1/4 yu shen : > > Hi All, > > > > I have a dataimporthandler config file as below. It contains multiple > > entities

DataImportHanlder - Multiple entities will step into each other

2011-01-04 Thread yu shen
Hi All, I have a dataimporthandler config file as below. It contains multiple entities: All data are from a database. Problem is item/company and other entity all have the field 'id', with value start from 1 to n.