Re: jconsole over jmx - should threads be visible?

2012-12-19 Thread Bernd Fehling
Hi Shawn, actually I use munin for monitoring but just checked with jvisualvm which also runs fine for remote monitoring. You might try the following: http://www.codefactorycr.com/java-visualvm-to-profile-a-remote-server.html You have to: - generate a policy file on the server to be monitored -

Re: Faceting on Dynamic fields

2012-12-19 Thread Erick Erickson
The base problem is you don't know what they are in advance. You could simply attach &facet.field=name_0&facet.field=name_1 Or is "name" variable too? If it is, you could query the index up-front and get all of the dynamic field names (the admin page does this) and assemble the above from the

Re: Solr Cloud 4.0 Production Ready?

2012-12-19 Thread Anirudha Jadhav
I am curious to know what issues you are facing with 3.5 and large indices. We use 3.5 with 200g indices with no issues until now -Ani On Tuesday, December 18, 2012, Luis Cappa Banda wrote: > Any idea about when Solr 4.1 will be released? > > 2012/12/18 Otis Gospodnetic > > > > Hi, > > > > If yo

Re: Solr atomic update of multi-valued field

2012-12-19 Thread Erick Erickson
FIrst question: Is the "name" field stored (stored="true")? If it isn't, that would explain your problems with that field. _all_ relevant fields (i.e. everything not a destination of a copyField) need to be stored for atomic updates to work. Your second problem I'm not sure about. I remember some

Re: Strange data-loss problem on one of our cores

2012-12-19 Thread Erick Erickson
Thanks for letting us know, and do bring let us know if you see the problem again. Erick On Tue, Dec 18, 2012 at 7:39 AM, John Nielsen wrote: > I build a solr version from the solr-4x branch yesterday and so far am > unable to replicate the problems i had before. > > I am cautiously optimistic

Re: jconsole over jmx - should threads be visible?

2012-12-19 Thread Shawn Heisey
On 12/19/2012 1:39 AM, Bernd Fehling wrote: Hi Shawn, actually I use munin for monitoring but just checked with jvisualvm which also runs fine for remote monitoring. You might try the following: http://www.codefactorycr.com/java-visualvm-to-profile-a-remote-server.html You have to: - generate

Re: Solr atomic update of multi-valued field

2012-12-19 Thread Dikchant Sahi
Hi Erick, The name field is stored. I experience problem only when I update multiValued field with multiple values like, * solr* * lucene* * * It works perfect, when I set single value for multiValued field like, *solr* Thanks, Dikchant On Wed, Dec 19, 2012 at 6:25 PM, Erick Erickson wrote: > F

Re: java.lang.OutOfMemoryError and shard can‘t work

2012-12-19 Thread Jam Luo
OK, I will try to upgrade Oracle java and the lastest solr. thanks 2012/12/19 Shawn Heisey > On 12/18/2012 8:18 PM, Jam Luo wrote: > > I deployed a solr-4.0-beta cluster, 4 shard, 2 peers in a shard. A peer > > catch exception: > > 十二月 18, 2012 7:56:31 下午 org.apache.solr.common.SolrException l

RE: "order" question on solr multi value field

2012-12-19 Thread Dyer, James
I was noticing when I was looking this up in LIA yesterday a mention was made that when using a field list to return just some of the stored fields, that it still takes time to seek over the fields you don't want. So, if it has this strict guarantee about order, could a user potentially optimiz

Re: "order" question on solr multi value field

2012-12-19 Thread Yonik Seeley
On Tue, Dec 18, 2012 at 8:24 PM, Robert Muir wrote: > I agree with James. Actually lucene tests will fail if a codec violates this. > > Actually it goes much deeper than this. > > From the lucene apis, when you call IndexReader.document() with your > storedfieldVisitor, it must visit the fields in

Invalid version (expected 2, but 60) or the data in not in 'javabin'

2012-12-19 Thread Shahar Davidson
Hi, I'm encountering this error randomly when running a distributed facet. (i.e. I'm sending the exact same request, yet this does not reproduce consistently) I have about 180 shards that are being queried. It seems that when Solr distributes the request to the shards one , or perhaps more, sh

Re: Sort Results on Popularity

2012-12-19 Thread Mikhail Khludnev
nope. The closest features is http://www.findwise.com/blog/externalfilefield-in-solr/ there is a prebuild framework from the vendor http://lucidworks.lucidimagination.com/display/lweug/Click+Scoring+Relevance+Framework On Wed, Dec 19, 2012 at 5:13 PM, Pratyul Kapoor wrote: > Dear all, > > I a

RE: Ensuring SpellChecker returns corrections which satisfy fq params for default OR query

2012-12-19 Thread Dyer, James
Let me try and get a better idea of what you're after. Is it that your users might query a combination of irrelevant terms and misspelled terms, so you want the ability to ignore the irrelevant terms but still get suggestions for the misspelled terms? For instance if someone wanted "q=christma

Re: Solr 4.0 with Solr Cloud 404

2012-12-19 Thread Mark Miller
The default host for urls is /solr. Looks like you want to run on the root context. In Solr 4.1 this is best done by setting the host to / (either in solr.xml or with the hostContext sys prop). For 4.0 you have to use a host context of "." to run on the root context. - Mark On Dec 19, 2012, at

Solr Cloud Architecture and DIH

2012-12-19 Thread Mark
We're currently running Solr 3.5 and our indexing process works as follows: We have a master that has a cron job to run a delta import via DIH every 5 minutes. The delta-import takes around 75 minutes to full complete, most of that is due to optimization after each delta and then the slaves s

Re: if I only need exact search, does frequency/score matter?

2012-12-19 Thread Jie Sun
Hi Otis, I customized the Similarity class and add it through the end of schema.xml: ... ... and mypackage.NoTfSimilarity.java is like: public class NoTfSimilarity extends DefaultSimilarity { public float tf(float freq) { return freq > 0 ? 1.0f : 0.0f; } public flo

Solrj- QueryResponse's getbeans throwing bindingexception

2012-12-19 Thread Sadhana
Hi, I have a state column defined in schema.xml as The getbeans method of Queryrespoinse using solj API is throwing binding exception as follows..Probably the special characters like [ : / - might be throwing this error.. org.apache.solr.client.solrj.beans.Binding

Re: if I only need exact search, does frequency/score matter?

2012-12-19 Thread Jie Sun
Hi Otis, here is the debug output on the query... seems all tf and idf indeed return 1.0f as I customized... I did not overwrite queryNorm or weight etc... see below. but the bottom line is that if my purpose is to reduce the frq file size, customize similarity seems wont help on that. I guess th

Re: Ensuring SpellChecker returns corrections which satisfy fq params for default OR query

2012-12-19 Thread Nalini Kartha
Hi James, Yup the example you gave about sums it up. Reason we use an OR query is that we want the flexibility of every term not having to match but when it comes to corrections we want to be sure that the ones we pick will actually return results (we message the user with the corrected query so i

Re: Solrj- QueryResponse's getbeans throwing bindingexception

2012-12-19 Thread Sadhana
Though the type is String its storing the datatype as List because the mutlitvalued=true in the schema.xml. So the issue got resolved after I change my datatype of State variable in the POJO from String to List. Thanks Sadhana -- View this message in context: http://lucene.472066.n3.nabble

RE: dataimport.properties not created/updated with solrcloud

2012-12-19 Thread adm1n
Well, I saw, that when I ran the full/delta i,port process on the 2nd, 3rd etc times I didn't so this exception any more. So I checked in my mysql queries log what's going on in mysql when I was running delta import process and I saw, that the queries got correct times on each delta-import executio

RE: Ensuring SpellChecker returns corrections which satisfy fq params for default OR query

2012-12-19 Thread Dyer, James
Instead of using spellcheck.collateParam.mm, try just setting spellcheck.maxResultsForSuggest to a very high value (you can use up to Integer.MAX_VALUE here). So long as the user gets fewer results that whatever this is set for, you will get suggestions (and collations if desired). I was just

RE: dataimport.properties not created/updated with solrcloud

2012-12-19 Thread Dyer, James
Someone with more zookeeper knowledge than I have can better answer this, but there is code in place specifically for using DIH with SolrCloud to save the "dataimport.properties" file in an appropriate place. The default path is: /configs/{collection}/dataimport.properties I'm not sure which

ICUTokenizer labels number as Han character?

2012-12-19 Thread Tom Burton-West
Hello, Don't know if the Solr admin panel is lying, or if this is a wierd bug. The string: "1986年" gets analyzed by the ICUTokenizer with "1986" being identified as type:NUM and script:Han. Then the CJKBigram filter identifies "1986" as type:Num and script:Han and "年" as type:Single and script:

Solr stats.facet on TrieField doesn't work

2012-12-19 Thread jefferyyuan
This seems an known issue: http://wiki.apache.org/solr/StatsComponent TrieFields has to use a precisionStep of -1 to avoid using UnInvertedField.java. Consider using one field for doing stats, and one for doing range facetting on. To fix this problem. and support dacet search on this field, I hav

Re: ICUTokenizer labels number as Han character?

2012-12-19 Thread Robert Muir
Your attachment didnt come through: I think the list strips them. Maybe just open a JIRA and attach your screenshots? or put them elsewhere and just include a link? As far as the ultimate behavior, I think its correct. Keep in mind tokens don't really get a script value: runs of untokenized text d

Re: Solr Cloud Architecture and DIH

2012-12-19 Thread Shawn Heisey
On 12/19/2012 11:50 AM, Mark wrote: We have a master that has a cron job to run a delta import via DIH every 5 minutes. The delta-import takes around 75 minutes to full complete, most of that is due to optimization after each delta and then the slaves sync up. Our index is around 30 gigs so a

Re: Intersect Circle is matching points way outside the radius ( Solr 4 Spatial)

2012-12-19 Thread Javi Molina
Hi David, Your latest response was lost in my inbox, I just realised it was there. You are right, I am using Open Layers, and even though I use the mercator projection, there are elements that not adhere to that projection, in particular the polygon that generates the circle and the scale contro

Putting more weight on particular column.

2012-12-19 Thread Prachi Phatak
We configured Enhanced search in our environment and while testing noticed below behavior: Just searching for "P" returns Enterprise ID which has "P" (Pankaj) as well as Resource Entity which has "P" (Sowmya), and the result lists Sowmya above Pankaj. * We are looking to limit the search

Using SolrJ to update index with array fields

2012-12-19 Thread leonildo
Hello, I´m using solr 1.4 How can I update solr index creating fields that must be updated with arrays ? My code is: long arrayOfLong[] = new long [] {1,2,3} ; SolrInputDocument doc = new SolrInputDocument(); doc.setField(field.getFieldName(), arrayOfLong); when updating the following error i

edismax request handler doesn't use defined qf

2012-12-19 Thread Jose Yadao
Hi, I wanted to put the fields I want to be searched in qf but it looks like edismax is not using it. I have the ff request handler made for testing edismax, used the sample xml to populate my solr 4.0 explicit edismax features^1.0 1

SolrCloud: only partial results returned

2012-12-19 Thread Lili
We set up SolrCloud with 2 shards and separate multiple zookeepers. The data added using http post with json in tutorial sample are not completely returned in query.However, if you send the same http post request again or shutdown solr instance and restart, the complete results will be retur

Re: Putting more weight on particular column.

2012-12-19 Thread Otis Gospodnetic
Hi, If I understand correctly, you want to search against a specific field - enterprise id. To do that just use something like enterpriseid:(keywords). Yes, you can sort using sort URL parameter. This stuff id on the Wiki and you can search it, too. :) Otis -- SOLR Performance Monitoring - http:

Re: edismax request handler doesn't use defined qf

2012-12-19 Thread Jack Krupansky
The white space is probably messing things up. Change: features^1.0 to features^1.0 It SHOULDN'T be a problem, but... -- Jack Krupansky -Original Message- From: Jose Yadao Sent: Wednesday, December 19, 2012 6:32 PM To: solr-user@lucene.apache.org Subject: edismax request handle

Re: edismax request handler doesn't use defined qf

2012-12-19 Thread Jose Yadao
Thanks but that didn't work. Parsedquery doesn't show DisjunctionMaxQuery it's using the default field of select text:car On Thu, Dec 20, 2012 at 9:38 AM, Jack Krupansky wrote: > The white space is probably messing things up. > > Change: > > > > features^1.0 > > > to > > > features^1.0 > > I

Re: edismax request handler doesn't use defined qf

2012-12-19 Thread Jack Krupansky
It may not be using that request handler. You are using the "qt" parameter, but have you changed the compatibility setting to enable it? See the comments in solrconfig. -- Jack Krupansky -Original Message- From: Jose Yadao Sent: Wednesday, December 19, 2012 8:44 PM To: solr-user@luce

Re: edismax request handler doesn't use defined qf

2012-12-19 Thread Jose Yadao
Thanks! so using select was a old approach and was not the default since 3.6. On Thu, Dec 20, 2012 at 9:47 AM, Jack Krupansky wrote: > It may not be using that request handler. You are using the "qt" > parameter, but have you changed the compatibility setting to enable it? See > the comments in s

Improving the speed of Solr query over 16 million tweets

2012-12-19 Thread naryad
I use Solr (SolrCloud) to index and search my tweets. There are about 16 million tweets and the index size is approximately 3 GB. The tweets are indexed in real time as they come so that real time search is enabled. Currently I use lowercase field type for my tweet body field. For a single search t

Re: Solr Cloud Architecture and DIH

2012-12-19 Thread Mikhail Khludnev
Hello Mark some of these questions has been touched recently, see below. On Wed, Dec 19, 2012 at 10:50 PM, Mark wrote: > We're currently running Solr 3.5 and our indexing process works as follows: > > . > > I also have the following questions. > Does DIH work with Solr Cloud? > Yes. it seem

Re: Improving the speed of Solr query over 16 million tweets

2012-12-19 Thread Alexandre Rafalovitch
KeywordTokenizerFactory does not actually break the text into tokens. It makes the whole field one whole token. Is that what you actually want? I would have thought that at least tweet body would be broken into words/tokens. Regards, Alex. Personal blog: http://blog.outerthoughts.com/ LinkedI

Re: Improving the speed of Solr query over 16 million tweets

2012-12-19 Thread naryad
You are completely right. I realized this, so is the only way to fix this is to redeclare the field as text_en or text_en_splitting and then delete all the documents and recreate the index? Or is there any other easy way? -- View this message in context: http://lucene.472066.n3.nabble.com/Impro

Dynamic modification of field value

2012-12-19 Thread AlexeyK
Hi, Suppose the document stored in the index has fields A and B. What would be the best way to alter the value of B after the result set is available? The modified value of B is influenced by the value of A and also by some custom logic based on (custom) SolrCache. Can it be a custom function query

Finding the last committed record in SOLR 4

2012-12-19 Thread Joe
I'm using SOLR 4 for an application, where I need to search the index soon after inserting records. I'm using the solrj code below to get the last ID in the index. However, I noticed that the last id I see when I execute a query through the solr web admin is often lagging behind this. And that my

SolrTestCaseJ4 and searcher initialization

2012-12-19 Thread AlexeyK
Hi, I've written a unit test for a custom search component, which naturally extends the SolrTestCaseJ4. beforeClass() has initCore(), assertU(adoc()) and assertU(commit()) inside. The test creates a SolrQueryRequest via req() and runs h.query(request). In other words, nothing special. I see a rathe