solr indexing on same set of records with different value of unique field...not working...

hi, I need to run around 10 million records to index, by solr. I has nearly 2lakh records, so i made a program to looping it till 10 million. Here, i specified 20 fields in schema.xml file. the unoque field i set was, currentTimeStamp field. So, when i run the loader program (which loads xml da

Re: How to configure Solr in Glassfish ?

Yes, I don't know how set solr.home in glassfish with centOS. I tried to configure the solr.home, but the error log is:"looking for solr.xml: /var/deploy/solr/solr.xml" markrmiller wrote: > > What have you tried? Deploying the Solr war should be pretty > straightforward. The main issue is l

DocList Pagination

Hi, I am try to get the next DocList "page" in my custom search component. Could I get a code example of this? Cheers. -- View this message in context: http://www.nabble.com/DocList-Pagination-tp24581850p24581850.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: method inform of SolrCoreAware callled 2 times

it is not normal to get the inform() called twice for a single object. which version of solr are you using? On Mon, Jul 20, 2009 at 7:17 PM, Marc Sturlese wrote: > > Hey there, > I have implemented a custom component wich extends SearchComponent and > implements SolrCoreAware. > I have decalred it

Re: Exception searching PhoneticFilterFactory field with number

Robert, Can you narrow things down by simplifying the query? For example, I see allDoublemetaphone:"2226", which looks suspicious in the "give me phonetic version of the input" context, but if you could narrow it down, we could probably be able to help more. Otis -- Sematext -- http://semat

Re: Obtaining SOLR index size on disk

Actually, if you have a server enabled as a replication master, the stats.jsp page reports the index size, so that information is available in some cases. -Peter On Sat, Jul 18, 2009 at 8:14 AM, Erik Hatcher wrote: > > On Jul 17, 2009, at 8:45 PM, J G wrote: >> >> Is it possible to obtain the SOL

Re: Recommended Articles

I personally love this book: http://www.amazon.com/Building-Search-Applications-Lucene-LingPipe/dp/0615204252 It intermixes search with analysis: sentiment, named entity recognition, NLP Pipelines and so on... There's a little Nutch cameo too... On Mon, Jul 20, 2009 at 4:56 PM, Mark Miller wro

Re: hierarchical faceting discussion

I was particularly surprised by the SOLR-64 numbers. What makes it's response so huge (and thus slow) to return the entire tree of facet counts? Erik On Jul 19, 2009, at 5:35 PM, Erik Hatcher wrote: I've posted the details of some experiments I just did comparing/ contrasting two

Re: unable to run the solr in tomcat 5.0

try this: java -Durl=http://localhost:8080/solr/update -jar post.jar filename.xml it should work. HH uday kumar maddigatla wrote: > > hi > > you mis understood my question. > > When i try to use the command java -post.jar *.*. It is trying to Post > files in Solr which is there in 8983 por

index version on slave

If you ask for the index version of a slave instance, you always get version number being 0. Is it expected behavior? I am using this url http://slave_host:8983/solr/replication?command=indexversion This request returns correct version on master. If you use the 'details' command, you get the ri

Re: Implementing related tags

Faceting on tags will give you all the related tags, including the original tag (tagA in your case). You will have to filter out the original tag on the client side if you don't want to show that. With Solar 1.4, you will be able to use localParam to exclude the original tag in the results. If y

Re: Implementing related tags

If I understood your problem correctly, faceting on "tags" field is what you need. Try this - http://localhost:8983/solr/ memoir/select?fq=tag:tagA&q=( tags%3Aba*)&facet=true&facet.field=tags&facet.mincount=1

Re: Implementing related tags

That does not seem to work fine. To further simplify the issue, assuming there is a multi valued tag field and number of docs is 1 million. By constrainting on a given tag, I need to search on the related tags. So Doc 1: tags: tagA, tagB, tagC, ball Doc 2: tags: tagA, bat Now constrainting

RE: multi-word synonyms with multiple matches

> You haven't given us the full details on how you are using the > SynonymFilterFactory ("expand" true or false?) but in general: yes the > SynonymFilter finds the longest match it can. Sorry - doing expansion at index time: > > if every svp is also a vp, then being explict in your synonyms (whe

Re: Truncated XML responses from CoreAdminHandler

Hi James, That is very odd behavior! I'm not sure what causing it at the moment, but that is not a great way to get all of the core names anyway. It also gathers a *lot* of information for each core that you don't need, including index statistic from Luke. Its very heavy weight for what you want.

Re: Solr JMX and Cacti

On Mon, Jul 20, 2009 at 12:31 PM, Ryan McKinley wrote: > > On Jul 20, 2009, at 9:16 AM, Edward Capriolo wrote: > >> On Mon, Jul 20, 2009 at 11:53 AM, Ryan McKinley wrote: >>> >>> On Jul 20, 2009, at 8:47 AM, Edward Capriolo wrote: >>> Hey all, We have several deployments of Solr acro

Re: How to configure Solr in Glassfish ?

What have you tried? Deploying the Solr war should be pretty straightforward. The main issue is likely setting solr.home. You likely have a lot of options there though. You can set a system property in the startup script, set a system property in the webapp context xml (if you can locate it), or I

Indexing issue with XML control characters

During indexing I will often get this error: SEVERE: com.ctc.wstx.exc.WstxUnexpectedCharException: Illegal character ((CTRL-CHAR, code 3)) at [row,col {unknown-source}]: [2,1] at com.ctc.wstx.sr.StreamScanner.throwInvalidSpace(StreamScanner.java:675) By looking at this list and elsewhe

Re: SolrJ embedded server : error while adding document

On Mon, Jul 20, 2009 at 18:35, Ryan McKinley wrote: > you send a bunch of requests with add( doc/collection ) and they are not > visible until you send commit() That's what I meant thanks. -- Gérard Dupont Information Processing Control and Cognition (IPCC) - EADS DS http://weblab-project.org

Re: SolrJ embedded server : error while adding document

not sure what you mean... yes, i guess... you send a bunch of requests with add( doc/collection ) and they are not visible until you send commit() On Jul 20, 2009, at 9:07 AM, Gérard Dupont wrote: my mistake, pb with the buffer I added. But it raises a question : does solr (using embedde

Re: Solr JMX and Cacti

On Jul 20, 2009, at 9:16 AM, Edward Capriolo wrote: On Mon, Jul 20, 2009 at 11:53 AM, Ryan McKinley wrote: On Jul 20, 2009, at 8:47 AM, Edward Capriolo wrote: Hey all, We have several deployments of Solr across our enterprise. Our largest one is a several GB and when enough documents a

Re: Implementing related tags

Have a look at the MoreLikeThis component - http://wiki.apache.org/solr/MoreLikeThis Cheers Avlesh On Mon, Jul 20, 2009 at 8:05 PM, James T wrote: > Hi, > > I have a specific requirement for searching and looking for some help from > the community on how to achieve it using solr: > > I need to i

Re: Solr JMX and Cacti

On Mon, Jul 20, 2009 at 11:53 AM, Ryan McKinley wrote: > > On Jul 20, 2009, at 8:47 AM, Edward Capriolo wrote: > >> Hey all, >> >> We have several deployments of Solr across our enterprise. Our largest >> one is a several GB and when enough documents are added an OOM >> exception is occurring. >> >

Exception searching PhoneticFilterFactory field with number

Reposting in hopes of an answer... Hello all, I am getting the following exception whenever a user includes a numeric term in their search, and the search includes a field defined with a PhoneticFilterFactory and further it occurs whether I use the DoubleMetaphone encoder or any other. Ha

Re: SolrJ embedded server : error while adding document

my mistake, pb with the buffer I added. But it raises a question : does solr (using embedded server) has its own buffer mechanism in indexing or not ? I guess not but I might be wrong. 2009/7/20 Gérard Dupont > Hi SolR guys, > > I'm starting to play with SolR after few years with classic Lucene.

SolrJ embedded server : error while adding document

Hi SolR guys, I'm starting to play with SolR after few years with classic Lucene. I'm trying to index a single document using the embedded server, but I got a strange error which looks like XML parsing problem (see trace hereafter). To add details, this is a simple Junit which create single docume

Re: Solr JMX and Cacti

On Jul 20, 2009, at 8:47 AM, Edward Capriolo wrote: Hey all, We have several deployments of Solr across our enterprise. Our largest one is a several GB and when enough documents are added an OOM exception is occurring. To debug this problem I have enable JMX. My goal is to write some cacti te

Solr JMX and Cacti

Hey all, We have several deployments of Solr across our enterprise. Our largest one is a several GB and when enough documents are added an OOM exception is occurring. To debug this problem I have enable JMX. My goal is to write some cacti templates similar to the ones I have done for hadoop. http

Re: Recommended Articles

dar...@ontrenet.com wrote: http://www.amazon.com/s/ref=nb_ss_gw?url=search-alias%3Dstripbooks&field-keywords=search&x=0&y=0 Does anyone have links or books to recommended reading on search in general. Would like to see some literature on larger search concepts and ideas. -- Jeff Newburn Soft

Re: Recommended Articles

http://www.amazon.com/s/ref=nb_ss_gw?url=search-alias%3Dstripbooks&field-keywords=search&x=0&y=0 > Does anyone have links or books to recommended reading on search in > general. > Would like to see some literature on larger search concepts and ideas. > -- > Jeff Newburn > Software Engineer, Zappos

Recommended Articles

Does anyone have links or books to recommended reading on search in general. Would like to see some literature on larger search concepts and ideas. -- Jeff Newburn Software Engineer, Zappos.com jnewb...@zappos.com - 702-943-7562

Implementing related tags

Hi, I have a specific requirement for searching and looking for some help from the community on how to achieve it using solr: I need to index 1million + documents. Each document contains ( among other fields ) 3 fields representing the category which that doc belongs to. For example ( a very simp

Re: Posting multiple documents at once - clarification

2009/7/20 Noble Paul നോബിള്‍ नोब्ळ् > if the error is an xml parsing error there is no way of continuing > from that point. even otherwise , solr assumes that if the whole > payload is not correct it is to be discarded > > Thank you for your response -- Thanks, Vanniarajan

RE: Wildcards at the Beginning of a Search.

Depending on how you are sending docs in for indexing, you could also add an additional field who's value was a string reverse of the primary value. Then search that field with a trialing wildcard. -Original Message- From: Jeff Newburn [mailto:jnewb...@zappos.com] Sent: Monday, July

Solr tika and posting .pst files

Hi, I am using Solr-Tika to post various files.When i try to post .pst file(outlook express), the file is being posted but it does not contain any data.I could not found anything useful after googling. Regarding solrschema , i use 1) id 2) content(this is the default field) Do i need to con

Re: Wildcards at the Beginning of a Search.

Add setAllowLeadingWildcard(true); to the constructor of org.apache.solr.search.SolrQueryParser.java Gr, Reza On Jul 20, 2009, at 4:00 PM, Jeff Newburn wrote: There is a hacky way to do it if you can pull it off. You can prepend some known prefix to the field then strip it off when you get

Re: Wildcards at the Beginning of a Search.

There is a hacky way to do it if you can pull it off. You can prepend some known prefix to the field then strip it off when you get the results back. An example would be putting Phone: in front of every value in a phone number field then instead of searching like this *-111- (which won't work)

method inform of SolrCoreAware callled 2 times

Hey there, I have implemented a custom component wich extends SearchComponent and implements SolrCoreAware. I have decalred it in solrconfig.xml as: And added it in my Searchhandler as: mycomp I am using multicore with two cores. I have noticed (doing some logging) that the

RE: Word frequency count in the index

Hi Wunder, Thanks for your reply! I take your point. It has to be appropriate to your content... In the cases I deal with, using stop words wouldn't be a big deal because the documents we handle are usually a proper article (although titles could still be impacted by it). I based my stop words o

Re: Posting multiple documents at once - clarification

if the error is an xml parsing error there is no way of continuing from that point. even otherwise , solr assumes that if the whole payload is not correct it is to be discarded On Mon, Jul 20, 2009 at 6:32 PM, Vannia Rajan wrote: > Hi, > >  When we post a file with a number of documents of the for

Posting multiple documents at once - clarification

Hi, When we post a file with a number of documents of the format shown below to solr, if there is some 'error' in one of the , then all the s in the file are error-ed out and not added to the Solr-index. ...

Re: Wildcards at the Beginning of a Search.

See http://issues.apache.org/jira/browse/SOLR-218 - Solr currently does not have leading wildcard support enabled. Erik On Jul 20, 2009, at 8:09 AM, Jörg Agatz wrote: Hallo Solr Users... I tryed to search with a Wildcard at the beginning from a search. for example, i will search fo

Re: Confusion around Binary/XML in SolrJ

On Jul 20, 2009, at 6:11 AM, Code Tester wrote: I am even unable to delete documents using the EmbeddedSolrServer ( on a specific core ) Steps: 1) I have 2 cores ( core0 , core1 ) Each of them have ~10 records. 2) System.setProperty("solr.solr.home", "/home/user/projects/solr/example/multi"

Re: post error - ERROR:unknown field 'title'

On Jul 20, 2009, at 6:23 AM, rossputin wrote: I have two different solr versions as I am evaluating nightly builds. On a more recent one.. I think 15th July I am getting the following error : ERROR:unknown field 'title' I am posting to 'solr/update/extract' with the following: curl "http:/

Re: Help needed with Solr

If yours is a JAVA stack of application, I would recommend moving to SolrJ. It is a client API which lets you talk to Solr. Know more about it here - http://wiki.apache.org/solr/Solrj Clients API's for other languages can be found here - http://wiki.apache.org/solr/#head-ab1768efa59b26cbd30f1acd03b

Wildcards at the Beginning of a Search.

Hallo Solr Users... I tryed to search with a Wildcard at the beginning from a search. for example, i will search for "*est" and get "test, vogelnest, fest, " But it dosent work, i alsways get an error... Now my Big brother GOOGLE tolds me, that it can work but a search with a Wildcad at the

Solr and UIMA

We are starting to use UIMA as a platform to analyze the text. The result of analyzing a document is a UIMA CAS. A Cas is a generic data structure that can contain different data. UIMA processes single documents, They get the documents from a CAS producer, process them using a PIPE that the user

Re: Confusion around Binary/XML in SolrJ

Sorry everyone. Found the issue. It was because of a very stupid assumption. My code and solr were running as 2 different processes! ( Weird part is that when I run the code using EmbeddedSolrServer, it did not throw any exception that there was already a server running on that port. ) Thanks! O

post error - ERROR:unknown field 'title'

Hi guys. I have two different solr versions as I am evaluating nightly builds. On a more recent one.. I think 15th July I am getting the following error : ERROR:unknown field 'title' I am posting to 'solr/update/extract' with the following: curl "http://localhost:8983/solr/update/extract?ext.

Re: Help needed with Solr

Hi Shalin, We just found that there is no limit on Solr side about the maximum boolean condition. We have set the 2048 and we are able to send about 1574 OR conditions. Over that limit, we are getting "HTTP/1.1 400 Bad Request". You are correct, it's not a Solr issue, its due to HTTP GET is not

Re: Confusion around Binary/XML in SolrJ

Another observation: I am even unable to delete documents using the EmbeddedSolrServer ( on a specific core ) Steps: 1) I have 2 cores ( core0 , core1 ) Each of them have ~10 records. 2) System.setProperty("solr.solr.home", "/home/user/projects/solr/example/multi"); File home = new File

Confusion around Binary/XML in SolrJ

I am using solr 1.4 dev in a multicore way. Each of my core's solrconfig.xml has the following lines I am using SolrJ as EmbeddedSolrServer. When I try to add a POJO ( with @Field annotations ), the data does not get indexed. Where as, if I use SolrInputDocument way, the data gets indexed. PS

Re: Help needed with Solr

On Mon, Jul 20, 2009 at 2:12 PM, dipanjan_pramanick < dipanjan_praman...@infosys.com> wrote: > > Its true that there is a design flaw, because of what we need to support a > huge list of OR conditions through Solr. > But still I would like to know if there is any other configuration other > than t

Re: Help needed with Solr

Hi Shalin, Thanks for your time to respond to this issue. Its true that there is a design flaw, because of what we need to support a huge list of OR conditions through Solr. But still I would like to know if there is any other configuration other than the one in solrConfig.xml, through which we

Re: Help needed with Solr

On Mon, Jul 20, 2009 at 1:37 PM, dipanjan_pramanick < dipanjan_praman...@infosys.com> wrote: > Hi, > We have scenario where we need to send more than 1024 ids in the Solr url > as "OR" condition. > I have changed the value of maxBooleanClauses in solrconfig.xml, to 2048, > but it is failing after

How to configure Solr in Glassfish ?

I want use glassfish as the solr search server, but I don't know how to configure. Anybody knows? enzhao...@gmail.com Thanks! -- View this message in context: http://www.nabble.com/How-to-configure-Solr--in-Glassfish---tp24565758p24565758.html Sent from the Solr - User mailing list archive a

Help needed with Solr

Hi, We have scenario where we need to send more than 1024 ids in the Solr url as "OR" condition. I have changed the value of maxBooleanClauses in solrconfig.xml, to 2048, but it is failing after handling 1024 "OR" conditions. Solr is throwing "SEVERE: org.apache.solr.common.SolrException: Bad Req