Solr for noSQL

Hi, Do we have data import handler to fast read in data from noSQL database, specifically, MongoDB I am thinking to use? Or a more general question, how does Solr work with noSQL database? Thanks. Jianbin

embeded solrj doesn't refresh index

Hi, I am using embedded solrj. After I add new doc to the index, I can see the changes through solr web, but not from embedded solrj. But after I restart the embedded solrj, I do see the changes. It works as if there was a cache. Anyone knows the problem? Thanks. Jianbin

RE: embeded solrj doesn't refresh index

x27;t refresh index You should send a commit to you embedded solr Marco Martínez Bautista http://www.paradigmatecnologico.com Avenida de Europa, 26. Ática 5. 3ª Planta 28224 Pozuelo de Alarcón Tel.: 91 352 59 42 2011/7/20 Jianbin Dai > Hi, > > > > I am using embedded solrj. After

RE: embeded solrj doesn't refresh index

x27;t refresh index You should send a commit to you embedded solr Marco Martínez Bautista http://www.paradigmatecnologico.com Avenida de Europa, 26. Ática 5. 3ª Planta 28224 Pozuelo de Alarcón Tel.: 91 352 59 42 2011/7/20 Jianbin Dai > Hi, > > > > I am using embedded solrj. After

RE: embeded solrj doesn't refresh index

Thanks Marc. Guess I was not clear about my previous statement. So let me rephrase. I use DIH to import data into solr and do indexing. Everything works fine. I have another embedded solr server setting to the same index files. I use embedded solrj to search the index file. So the first solr i

Help needed on DataImportHandler to index xml files

Hi All, I am new here. Thanks for reading my question. I want to use DataImportHandler to index my tons of xml files (7GB total) stored in my local disk. My data-config.xml is attached below. It works fine with one file (abc.xml), but how can I index all xml files at one time? Thanks!

How to index large set data

Hi, I have about 45GB xml files to be indexed. I am using DataImportHandler. I started the full import 4 hours ago, and it's still running My computer has 4GB memory. Any suggestion on the solutions? Thanks! JB

Re: How to index large set data

u, May 21, 2009 at 11:48 AM, Jianbin Dai > wrote: > > > > Hi, > > > > I have about 45GB xml files to be indexed. I am using > DataImportHandler. I started the full import 4 hours ago, > and it's still running > > My computer ha

Re: How to index large set data

rg > Date: Thursday, May 21, 2009, 10:39 PM > what is the total no:of docs created > ?  I guess it may not be memory > bound. indexing is mostly amn IO bound operation. You may > be able to > get a better perf if a SSD is used (solid state disk) > > On Fri, May 22, 2009

Re: How to index large set data

ck... > > > Otis > -- > Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch > > > > - Original Message > > From: Jianbin Dai > > To: solr-user@lucene.apache.org; > noble.p...@gmail.com > > Sent: Friday, May 22, 2009 3:42:04 AM > >

Re: How to index large set data

ush into Solr. > > > On May 22, 2009, at 3:42 AM, Jianbin Dai wrote: > > > > > about 2.8 m total docs were created. only the first > run finishes. In  > > my 2nd try, it hangs there forever at the end of > indexing, (I guess  > > right before commit)

How to use DIH to index attributes in xml file

I have an xml file like this 301.46 In the data-config.xml, I use but how can I index "id", "mid"? Thanks.

Re: How to index large set data

o need to use embedded Solrserver. > you can use SolrJ with streaming > in multiple threads > > On Fri, May 22, 2009 at 8:36 PM, Jianbin Dai > wrote: > > > > If I do the xml parsing by myself and use embedded > client to do the push, would it be more efficient than DIH? >

Re: How to use DIH to index attributes in xml file

iday, May 22, 2009, 9:03 PM > wild cards are not supported . u must > use full xpath > > On Sat, May 23, 2009 at 4:55 AM, Jianbin Dai > wrote: > > > > I have an xml file like this > > > > > >                     type="stock-4" /> > &g

Re: How to index large set data

index large set data > To: "Jianbin Dai" > Date: Friday, May 22, 2009, 10:04 PM > On Sat, May 23, 2009 at 10:27 AM, > Jianbin Dai > wrote: > > > > Hi Pual, but in your previous post, you said "there is > already an issue for writing to Solr in multiple thr

Is it memory leaking in solr?

I am using DIH to do indexing. After I indexed about 8M documents (took about 1hr40m), it used up almost all memory (4GB), and the indexing becomes extremely slow. If I delete all indexing and shutdown tomcat, it still shows over 3gb memory was used. Is it memory leaking? if it is, then the lea

Re: Is it memory leaking in solr?

VIRT RES SHR S %CPU %MEMTIME+ COMMAND 3322 root 21 0 1357m 1.0g 11m S 100 27.0 397:51.74 java --- On Mon, 5/25/09, Jianbin Dai wrote: > From: Jianbin Dai > Subject: Is it memory leaking in solr? > To: solr-user@lucene.apache.org

Re: Is it memory leaking in solr?

leak?  The > slowness? > > Otis > -- > Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch > > > > - Original Message > > From: Jianbin Dai > > To: solr-user@lucene.apache.org > > Sent: Monday, May 25, 2009 1:05:43 PM > > Subject: Re

how to do exact serch with solrj

Hi, I want to search "hello the world" in the "title" field using solrj. I set the query filter query.addFilterQuery("title"); query.setQuery("hello the world"); but it returns not exact match results as well. I know one way to do it is to set "title" field to string instead of text. But is

Re: how to do exact serch with solrj

gt; world") is what you need. > > Cheers > Avlesh > > On Sun, May 31, 2009 at 6:23 AM, Jianbin Dai > wrote: > > > > > Hi, > > > > I want to search "hello the world" in the "title" > field using solrj. I set > > the query

Re: how to do exact serch with solrj

e tokens? > If yes, try query.setQuery("title:\"hello the world\""); > > Cheers > Avlesh > > On Sun, May 31, 2009 at 12:12 PM, Jianbin Dai > wrote: > > > > > I tried, but seems it's not working right. > > > > --- On Sat, 5/30/09, Av

Index Comma Separated numbers

Hi, One of the fields to be indexed is price which is comma separated, e.g., 12,034.00. How can I indexed it as a number? I am using DIH to pull the data. Thanks.

Re: how to do exact serch with solrj

ow to do exact serch with solrj > > To: solr-user@lucene.apache.org > > Date: Saturday, May 30, 2009, 11:45 PM > > You need exact match for all the > > three tokens? > > If yes, try query.setQuery("title:\"hello the > world\""); > > &g

Re: Index Comma Separated numbers

did you try the > NumberFormatTransformer ? > > On Fri, Jun 5, 2009 at 12:09 AM, Jianbin Dai > wrote: > > > > Hi, One of the fields to be indexed is price which is > comma separated, e.g., 12,034.00.  How can I indexed it as > a number

Re: Index Comma Separated numbers

I forgot to put formatStyle="number" on the field. It works now. Thanks!! --- On Fri, 6/5/09, Jianbin Dai wrote: > From: Jianbin Dai > Subject: Re: Index Comma Separated numbers > To: solr-user@lucene.apache.org, noble.p...@gmail.com > Date: Friday, June 5, 2009, 12:3

Use DIH with large xml file

Hi, I have about 50GB of data to be indexed each day using DIH. Some of the files are as large as 6GB. I set the JVM Xmx to be 3GB, but the DIH crashes on those big files. Is there any way to handle it? Thanks. JB

Re: Use DIH with large xml file

;     Erik > > On Jun 20, 2009, at 9:23 PM, Jianbin Dai wrote: > > > > > Hi, > > > > I have about 50GB of data to be indexed each day using > DIH. Some of the files are as large as 6GB. I set the JVM > Xmx to be 3GB, but the DIH crashes on those big files. Is > there any way to handle it? > > > > Thanks. > > > > JB > > > > > > > >

weighted search and index

Hi, I am trying to use solr for a content match application. A content is described by a set of keywords with weights associated, eg., C1: fruit 0.8, apple 0.4, banana 0.2 C2: music 0.9, pop song 0.6, Britney Spears 0.4 Those contents would be indexed in solr. In the search, I also have a set

RE: weighted search and index

oosting will *tend* to move a hit up or down in the ranking, not position it absolutely. HTH Erick On Wed, Mar 3, 2010 at 8:13 PM, Jianbin Dai wrote: > Hi, > > I am trying to use solr for a content match application. > > A content is described by a set of keywords with weights ass

RE: weighted search and index

t index time. Why are they even there? Erick On Wed, Mar 3, 2010 at 8:54 PM, Jianbin Dai wrote: > Thank you very much Erick! > > 1. I used boost in search, but I don't know exactly what's the best way to > boost, for such as Sports 0.8, golf 0.5 in my example, would it be &

RE: weighted search and index

://www.lucidimagination.com/blog/2009/08/05/getting-started-with-payload s/ <http://www.lucidimagination.com/blog/2009/08/05/getting-started-with-payloa ds/>for your index-time term boosting. Query time boosting is as you indicated HTH Erick On Wed, Mar 3, 2010 at 9:34 PM, Jianbin Dai wrote: