Re: [Announce[ White paper describing Near Real Time Implementation with Solr and RankingAlgorithm

2011-05-23 Thread Andy
Thanks Nagendra. 2 questions: 1) How does NRT update affect the performance of facet search? My understanding is that facet in Solr relies on caching. With NRT update, wouldn't facet cache be invalidated all the time? And if that's the case, would the performance of facet be significantly redu

Stats help needed on price field using different currencies

2011-05-23 Thread insigh...@gmail.com
Hi all, I took a look at: http://lucene.472066.n3.nabble.com/Tuning-StatsComponent-td2225809.html which is a similar problem. I have a "price" field, on which I'm faceting (stats.field=price). But the currency of those docs is different, so the returned value is useless without converting th

Re: Too many Boolean Clause and Filter Query

2011-05-23 Thread Sujatha Arun
A filter query does not affect the scoring of the document ,so is filter query like constantScore query and hence should not be affected by the maxBoolean clause? Regards Sujatha On Mon, May 23, 2011 at 4:56 PM, Sujatha Arun wrote: > I got the info from this artice > > > http://www.lucidimagi

Re: SOLR Install

2011-05-23 Thread Yuhan Zhang
Hi Raj, To index files using java, use solrj: http://www.google.com/search?q=solrj&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:en-US:official&client=firefox-a To index files by a post request, follow this tutorial: http://www.xml.com/pub/a/2006/08/09/solr-indexing-xml-with-lucene-andrest.html Yuhan

Improving PayloadTermQuery Performance

2011-05-23 Thread Neil Hooey
What are some ways that one can increase the performance of PayloadTermQuery's? I'm currently getting a max of 22 QPS after 90k unique queries from a payload-enhanced keyword field on a dataset of 18 million documents, where a simple term search on the equivalent multivalue string field gives a ma

RE: Spatial Solr 3.1: filter by viewport

2011-05-23 Thread Zac Smith
It looks like someone asked this question a few months ago and didn't get an answer either ... http://lucene.472066.n3.nabble.com/Spatial-Solr-Representing-a-bounding-box-and-searching-for-it-tc2447262.html#none I really thought this would be a pretty simple question to answer? Is there no way

Re: dates with unknown parts

2011-05-23 Thread Markus Jelsma
I'm not sure i understand what you're looking for but if you write about unknown values i think of lack of precission. Solr allows you to reduce precission of dates using the flexible DateMathParser. http://lucene.apache.org/solr/api/org/apache/solr/util/DateMathParser.html > Hello, > > I am

dates with unknown parts

2011-05-23 Thread Mari Masuda
Hello, I am just getting started with Solr and have a question about date/time indexing. I know from the javadoc documentation ( http://lucene.apache.org/solr/api/org/apache/solr/schema/DateField.html ) that the format should be a restricted form of http://www.w3.org/TR/xmlschema-2/#dateTime-

Re: Similarity

2011-05-23 Thread Markus Jelsma
Hmm. I don't add code to Apache packages but create my own packages and namespaces, build a jar and add it to the lib directory as specified in solrconfig. Then you can use the FQCN to in the similarity config to point to the class. May be it can work when messing inside the apache namespace bu

Re: Similarity

2011-05-23 Thread Brian Lamb
Okay well this is encouraging. I changed SweetSpotSimilarity to MyClassSimilarity. I created this class in: lucene/contrib/misc/src/java/org/apache/lucene/misc/ I am getting a ClassNotFoundException when I try to start solr. Here is the contents of the MyClassSimilarity file: package org.apache

Re: Including phonetic search in text field

2011-05-23 Thread Jamie Johnson
Paul, Do you have an example of how to enable this in the solr config on the default request handler? Is it as simple as adding edismax text^0.5 features^1.0 name^1.2 sku^1.5 id^10.0 manu^1.1 cat^1.4 to the requestHandler named search? On Mon, May 23, 2011 at 4:1

Re: Including phonetic search in text field

2011-05-23 Thread Jamie Johnson
Ah, yes very helpful thanks Paul. I knew there would be something that I broke :). I will need to go back and consider the use cases and see which will and will not require exact matches. Thanks again! I have never heard of DisMax so this is new to me as well but have found some posts about it

Re: Including phonetic search in text field

2011-05-23 Thread Paul Libbrecht
Jamie, the problem with that is that you cannot do exact matching anymore. For this reason, it is good style to have two fields, to use a query expander such as dismax (prefer exact matches, and less phonetic matches), and to only use that when you sort by score. hope it helps paul Le 23 mai

Including phonetic search in text field

2011-05-23 Thread Jamie Johnson
I am new to solr and am trying to determine the best way to take the text field type (the one in the example) and add phonetic searches to it. Currently I have done the following:

Re: Similarity

2011-05-23 Thread Markus Jelsma
As far as i know, SweetSpotSimilarty needs be configured. I did use it once but wrapped a factory around it to configure the sweet spot. It worked just as expected and explained in that paper about the subject. If you use a custom similarity that , for example, caps tf to 1. Does it then work?

Similarity

2011-05-23 Thread Brian Lamb
Hi all, I'm having trouble getting the basic similarity example to work. If you notice at the bottom of the schema.xml file, there is a line there that is commented out: I uncomment that line and replace it with the following: Which comes natively with lucene. However, the scores before and

RE: spellcheck.collate returning all results

2011-05-23 Thread Dyer, James
Richard, To enable the guarantee you need to specify "spellcheck.maxCollationTries" with a value other than zero (which is default). There is cost involved with verifying beforehand if the collations will return hits so this feature is "off" by default. Also, you may want to enable extended c

Re: Including Score in Solr POJO

2011-05-23 Thread Anuj Kumar
Hi, On Mon, May 23, 2011 at 8:52 PM, Kissue Kissue wrote: > Thanks Anuj for your reply. Would it then include it as a field in my POJO? > I meant the score given by Solr in response to the search query. Is it an application specific score that you want to include? > How do i define such field

correctlySpelled and onlyMorePopular in 3.1

2011-05-23 Thread Markus Jelsma
Hi, I know about the behaviour of the onlyMorePopular setting. It can return suggestions while the actual query is correctly spelled. There is, in my opinion, some bad behaviour, consider the following query that is correctly spelled and yields results and never suggestions: q=test&spellcheck.

Re: Indexing documents with "complex multivalued fields"

2011-05-23 Thread anass talby
Thank you very much On Mon, May 23, 2011 at 4:27 PM, Stefan Matheis < matheis.ste...@googlemail.com> wrote: > Anass, > > what about combining them both into one? so to say: > 1|red > 2|green > > "synchronized" multivalued fields are not possible, afaik. > > Regards > Stefan > > On Mon, May 23, 20

Re: Including Score in Solr POJO

2011-05-23 Thread Kissue Kissue
Thanks Anuj for your reply. Would it then include it as a field in my POJO? How do i define such field? I have a POJO with the @Field annotation which is mapped to fields in my schema. Thanks. On Mon, May 23, 2011 at 4:10 PM, Anuj Kumar wrote: > Hi, > > If you mean SolrJ (as I understand by you

Re: Indexing documents with "complex multivalued fields"

2011-05-23 Thread anass talby
Thank you Renaud. I appreciate your help On Mon, May 23, 2011 at 4:47 PM, Renaud Delbru wrote: > Hi, > > you could look at this recent thread [1], it is similar to your problem. > > [1] > http://search.lucidimagination.com/search/document/33ec1a98d3f93217/search_across_related_correlated_multiva

Re: Including Score in Solr POJO

2011-05-23 Thread Anuj Kumar
Hi, If you mean SolrJ (as I understand by your description of POJOs), you can add the score by setting the property IncludeScore to true. For example- SolrQuery query = new SolrQuery(). setQuery(keyword). *setIncludeScore(true);* Regards, Anuj On Mon, May 23, 2011 at 8:31 PM, Kissue K

Including Score in Solr POJO

2011-05-23 Thread Kissue Kissue
Hi, I am currently using Solr and indexing/reading my documents as POJO. The question i have is how can i include the score in the POJO for each document found in the index? Thanks.

Re: SOLR Install

2011-05-23 Thread Gora Mohanty
On Mon, May 23, 2011 at 7:40 PM, Roger Shah wrote: > Hi, > > I am a new user and I have installed SOLR 3.1.0 and running Tomcat 7.0. > I was able to run the example which shows the SOLR Admin screen.  Also posted > an XML file by this command from dos prompt:  java -jar post.jar solr.xml. Great.

spellcheck.collate returning all results

2011-05-23 Thread Richard Hodsdon
Hi, I have been trying to set up spellchecking on our system using the SpellCheckComponent. According to the wiki by using spellcheck.collate any fq parameters that are passed through to the original query while doing spellcheck will return results if the collation is re-run. So far this has not

Re: how to convert YYYY-MM-DD to YYY-MM-DD hh:mm:ss - DIH

2011-05-23 Thread Gora Mohanty
On Mon, May 23, 2011 at 7:03 PM, stockii wrote: > yes. i put the &zeroDateTimeBehavior=convertToNull to my url like: > url="jdbc:mysql://localhost/databaseName?zeroDateTimeBehavoir=convertToNull" I presume that this is just a typo here, i.e., "Behavoir" instead of "Behavior"

Re: I only want to return a fields value in certain cases, how is this done

2011-05-23 Thread Anuj Kumar
Hi, On Mon, May 23, 2011 at 7:27 PM, bryan rasmussen wrote: > Let us say I have 3 fields I index > f1, f2, f3. > > f1 and f2 are copied to f4. > f4 is the default searched field. > > > There is a value that is found in f2 and f3. > > When I am searching in f3 I want to return only f3 and none ot

Re: Indexing documents with "complex multivalued fields"

2011-05-23 Thread Renaud Delbru
Hi, you could look at this recent thread [1], it is similar to your problem. [1] http://search.lucidimagination.com/search/document/33ec1a98d3f93217/search_across_related_correlated_multivalue_fields_in_solr#1f66876c782c78d5 -- Renaud Delbru On 23/05/11 14:40, anass talby wrote: Hi, I'm new

Re: Indexing documents with "complex multivalued fields"

2011-05-23 Thread Stefan Matheis
Anass, what about combining them both into one? so to say: 1|red 2|green "synchronized" multivalued fields are not possible, afaik. Regards Stefan On Mon, May 23, 2011 at 3:40 PM, anass talby wrote: > Hi, > > I'm new in solr and would like to index documents that have complex > multivalued fie

SOLR Install

2011-05-23 Thread Roger Shah
Hi, I am a new user and I have installed SOLR 3.1.0 and running Tomcat 7.0. I was able to run the example which shows the SOLR Admin screen. Also posted an XML file by this command from dos prompt: java -jar post.jar solr.xml. How can I get SOLR to search web sites and also search through othe

I only want to return a fields value in certain cases, how is this done

2011-05-23 Thread bryan rasmussen
Let us say I have 3 fields I index f1, f2, f3. f1 and f2 are copied to f4. f4 is the default searched field. There is a value that is found in f2 and f3. When I am searching in f3 I want to return only f3 and none other. when I am searching in f4 I do not want to return f3. I only want to retu

Re: Pivot with Stats (or Stats with Pivot)

2011-05-23 Thread Eduardo
I would like to request this feature. How can I do that? Thanks. -- View this message in context: http://lucene.472066.n3.nabble.com/Pivot-with-Stats-or-Stats-with-Pivot-tp2953789p2975341.html Sent from the Solr - User mailing list archive at Nabble.com.

Indexing documents with "complex multivalued fields"

2011-05-23 Thread anass talby
Hi, I'm new in solr and would like to index documents that have complex multivalued fields. I do want to do something like: 1 1 red 2 green ... ... How can i do this with solr thanks in advance. -- Anass

Re: how to convert YYYY-MM-DD to YYY-MM-DD hh:mm:ss - DIH

2011-05-23 Thread stockii
the problems are not the empty -00-00 values. the problem is the missing timestamp at the end of the string ! - --- System One Server, 12 GB RAM, 2 Solr Instances, 7 Cores, 1 Core with 31 Million Documents other Cor

Re: how to convert YYYY-MM-DD to YYY-MM-DD hh:mm:ss - DIH

2011-05-23 Thread stockii
yes. i put the &zeroDateTimeBehavior=convertToNull to my url like: url="jdbc:mysql://localhost/databaseName?zeroDateTimeBehavoir=convertToNull" ExceptoiN: May 23, 2011 3:30:22 PM org.apache.solr.handler.dataimport.DataImporter doFullImport SEVERE: Full Import failed org.apache.solr.handler.datai

Re: how to convert YYYY-MM-DD to YYY-MM-DD hh:mm:ss - DIH

2011-05-23 Thread Gora Mohanty
On Mon, May 23, 2011 at 6:51 PM, stockii wrote: > okay. i didn find the problem =( > it sstill the same shit. > > i cannot conver with DateTimeFormater dates form -00-00 => > "-MM-dd'T'hh:mm:ss'Z'" [...] Please post the exact Solr exception. Have you tried the last suggestion in the DIH

Re: how to convert YYYY-MM-DD to YYY-MM-DD hh:mm:ss - DIH

2011-05-23 Thread stockii
okay. i didn find the problem =( it sstill the same shit. i cannot conver with DateTimeFormater dates form -00-00 => "-MM-dd'T'hh:mm:ss'Z'" i put my date fields into another entity: solr throws exceptopn like above. WHY cannot transform

Re: very slow commits and overlapping commits

2011-05-23 Thread Bill Au
You can use the postCommit event listener as an callback mechanism to let you know that a commit has happened. Bill On Sun, May 22, 2011 at 9:31 PM, Jeff Crump wrote: > I don't have an answer to this but only another question: I don't think I > can use auto-commit in my application, as I have

Re: chinese SOLR query parser

2011-05-23 Thread Michael McCandless
Not that I know of... and I'm no expert on it!! I know there are at least two possibilities -- ChineseAnalyzer / CJKAnalyzer (from trunk's modules/analysis), but I don't know the tradeoffs of each. Hopefully others will chime in here? However, once you do figure out a good schema, could you plea

Re: Too many Boolean Clause and Filter Query

2011-05-23 Thread Sujatha Arun
I got the info from this artice http://www.lucidimagination.com/blog/2009/06/08/bringing-the-highlighter-back-to-wildcard-queries-in-solr-14/ Yes ,I know this can be configured ,but this will not scale to "n" ,what is the performance implication of this with "n" Boolean Clauses . Regards Sujat

Re: Too many Boolean Clause and Filter Query

2011-05-23 Thread Ahmet Arslan
> But this also gives the same TOO many Boolean Caluses > Exception You can adjust that parameter in solrconfig.xml 3024

Re: Too many Boolean Clause and Filter Query

2011-05-23 Thread Stefan Matheis
Sujatha, On Mon, May 23, 2011 at 1:05 PM, Sujatha Arun wrote: > Then we changed this to a filter query as follows , as filter Query is not > supposed to result in MaxBoolean Clause Exception Where did you read this? I'm pretty sure, that both is based on the same check for the maximum number of

Too many Boolean Clause and Filter Query

2011-05-23 Thread Sujatha Arun
Hello, We were initially doing a query with multiple "OR" as follows AND id:(1 2 3 4 5 ...n number ) Which gave maxBoolean Exception on exceeding 1024 boolean operators Then we changed this to a filter query as follows , as filter Query is not supposed to result in MaxBoolean Clause

missing content stream on csv upload

2011-05-23 Thread Robin Palotai
Hi, When uploading csv using curl http://localhost:8983/solr/update/csv --data-binary @books.csv -H 'Content-type:text/plain; charset=utf-8' as mentioned at the Wiki, you can get a "missing content stream" error on the jetty console. The problem is that on Windows, the -H parameter should be put

Re: Extracting contents of zipped files with Tika and Solr 1.4.1 (now Solr 3.1)

2011-05-23 Thread Gary Taylor
Jayendra, I cleared out my local repository, and replayed all of my steps from Friday and it now it works. The only difference (or the only one that's obvious to me) was that I applied the patch before doing a full compile/test/dist. But I assumed that given I was seeing my new log entries

Re: Did anyone rewrite the solr indexing section?

2011-05-23 Thread LeoYuan88
Hi Gora, Thanks for you quick response, I will take a look of that page. Thanks again! -- View this message in context: http://lucene.472066.n3.nabble.com/Did-anyone-rewrite-the-solr-indexing-section-tp2974539p2974568.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: Solr index as multiple separate index directories

2011-05-23 Thread LeoYuan88
Hi Jason, I got the same question with you, I'm wondering that have you figured that out? Plz take a look of the topic below: http://lucene.472066.n3.nabble.com/Did-anyone-rewrite-the-solr-indexing-section-td2974539.html Thanks in advance! -- View this message in context: http://lucene

Re: Sorting function with condition

2011-05-23 Thread Jan Høydahl
Hi, Could be possible if we implement https://issues.apache.org/jira/browse/SOLR-2136 Until then, perhaps you can try to pre-calculate a sort field during indexing, based on input data? You can also emulate an if() function using map() function http://wiki.apache.org/solr/FunctionQuery#map so

Re: Did anyone rewrite the solr indexing section?

2011-05-23 Thread Gora Mohanty
On Mon, May 23, 2011 at 2:39 PM, LeoYuan88 wrote: > Hi all, >     As we all have known, solr put all index files in a single directory, > namely ${datadir}/index, > but the perfomance's getting slower when the size of index dir's getting > bigger and bigger, > so I wanna split the single dir into

Did anyone rewrite the solr indexing section?

2011-05-23 Thread LeoYuan88
Hi all, As we all have known, solr put all index files in a single directory, namely ${datadir}/index, but the perfomance's getting slower when the size of index dir's getting bigger and bigger, so I wanna split the single dir into serveral dirs, e.g. ${datadir}/index1 and ${datadir}/index2,

query parser and other query filters

2011-05-23 Thread Dmitry Kan
Hi all, Can someone tell wether the query parser must always be the first step in the processing chain, before any other filter / tokenizer hits the query? Problem is that the user query can contain punctuation and we would like to clean it similarly to what we do on the index side. Is there any

What's the need for a complicated SolrTestCaseJ4.getClassName() ?

2011-05-23 Thread Gabriele Kahlout
Hello, As long as I subclass SolrTestCaseJ4 I cannot do this.getClass().getSimpleName(), I don't understand why. I wonder if the following complicated methods in SolrTestCaseJ4 have anything to do with it? protected static String getClassName() { StackTraceElement[] stack = new RuntimeExce