Re: Handling growth

2014-11-19 Thread Toke Eskildsen
On Thu, 2014-11-20 at 01:42 +0100, Patrick Henry wrote: > Good eye, that should have been gigabytes. When adding to the new shard, > is the shard already part of the the collection? What mechanism have you > found useful in accomplishing this (i.e. routing)? Currently (and for the foreseeable fu

NIO connector and maxIdleTime

2014-11-19 Thread sachinpkale
We are using SOLR 4.10.1 cloud setup and having following configuration in etc/jetty.xml 5 16 3000 false 8443 32768 In Jetty documentation, it is mentioned t

Include Solr score into a ranking algorithm

2014-11-19 Thread Nicholas Ding
Hi, Currently, I'm trying to implement a ranking algorithm on Solr to include TFIDFSimilarity score into a formula. Ranking = TFIDFSimilarity Score * X1 + V1 * X2 + V2 * X3 + . + Vn-1 * Xn Basically, the values of Vn are stored in DocValues, I can access them in customized Function Query. Th

Re: sort and group.sort

2014-11-19 Thread Erick Erickson
Please provide us with more information, including: 1> the query you send 2> some sample results 3> what you think is wrong. As it is, there's not much to go on here, sort and group.sort should do what you want. Best, Erick On Wed, Nov 19, 2014 at 2:33 PM, Frank li wrote: > We have a query whic

Re: Handling growth

2014-11-19 Thread Patrick Henry
Michael, Interesting, I'm still unfamiliar with limitations (if any) of aliasing. Does architecture utilize realtime get? On Nov 18, 2014 11:49 AM, "Michael Della Bitta" < michael.della.bi...@appinions.com> wrote: > We're achieving some success by treating aliases as collections and > collections

RE: Handling growth

2014-11-19 Thread Patrick Henry
Good eye, that should have been gigabytes. When adding to the new shard, is the shard already part of the the collection? What mechanism have you found useful in accomplishing this (i.e. routing)? On Nov 14, 2014 7:07 AM, "Toke Eskildsen" wrote: > Patrick Henry [patricktheawesomeg...@gmail.com]

Re: Handling intersection facets of many values

2014-11-19 Thread Peter Sturge
Hi Toke, Yes, the 'lots-of-booleans' thing is a bit prohibitive as it won't realistically scale to large value sets. I've been wrestling with joins this evening and have managed to get these working - and it works very nicely - and across cores (although not shards yet afaik)! For anyone looking

sort and group.sort

2014-11-19 Thread Frank li
We have a query which has both sort and group.sort. What we are expecting is that we can use sort to sort groups but inside the group we have a different sort. However, looks like sort is over-writting the sorting order inside groups. Can any one of you help us on this? Basically we want to sort

RE: Handling intersection facets of many values

2014-11-19 Thread Toke Eskildsen
Peter Sturge [peter.stu...@gmail.com] wrote: > I guess you mean take the 1k or so values and build a boolean query from > them? Not really. Let me try again: 1) Perform a facet call with facet.limit=-1 on dest to get the relevant dest values. The result will always be 1000 values or less. Take t

Re: Handling intersection facets of many values

2014-11-19 Thread Peter Sturge
Hi Toke, Thanks for your input. I guess you mean take the 1k or so values and build a boolean query from them? If that's not what you mean, my apologies.. I'd thought of doing that - the trouble I had was the unique values could be 20k, or 15,167 or any arbirary and potentially high-ish number - i

Resource leak(?) when closing CloudSolrServer

2014-11-19 Thread Phanindra R
Hi, Our indexing job and expiration job run every ~60 minutes (for about 10 minutes) in the test environment, within same JVM. Every job creates a new CloudSolrServer (decision was taken keeping other parts of system design in mind) and invokes shutdown() after it's complete. We have been seeing

RE: Handling intersection facets of many values

2014-11-19 Thread Toke Eskildsen
Peter Sturge [peter.stu...@gmail.com] wrote: [addr 7M unique, dest 1K unique] > What is the best/only/most efficient way to consutruct a search where by I > get back an (ideally faceted) list of values for 'dest' that occur in > 'addr'? I assume the actual values are defined by a query? As the n

Re: Solrcloud and remote Zookeeper ensemble

2014-11-19 Thread Jeon Woosung
did you set solr.solrxml.location in catalina.sh and how did you specify zkhost? -Dsolr.solrxml.location=zookeeper On Wed, Nov 19, 2014 at 9:54 PM, Robert Kent wrote: > Hi, > > I'm experiencing some odd behaviour with Solrcloud and Zookeeper. I am > running Solrcloud on one host and am running

Re: Solrcloud and remote Zookeeper ensemble

2014-11-19 Thread Erick Erickson
https://issues.apache.org/jira/browse/SOLR-6765 On Wed, Nov 19, 2014 at 10:07 AM, Mark Miller wrote: > If someone wants to file a JIRA, we really should detect and help the user > on that. > > - Mark > > On Wed Nov 19 2014 at 10:39:56 AM Robert Kent > wrote: > >> Yes, Alan's comment was correct

Handling intersection facets of many values

2014-11-19 Thread Peter Sturge
Hi Solr Group, Got an interesting use case (to me, at least), perhaps someone could give some insight on how best to achieve this? I've got a core that has about 7million entries, with a field call 'addr'. By definition, every entry has a unique 'addr' value, so there are 7million unique values f

Re: Solrcloud and remote Zookeeper ensemble

2014-11-19 Thread Mark Miller
If someone wants to file a JIRA, we really should detect and help the user on that. - Mark On Wed Nov 19 2014 at 10:39:56 AM Robert Kent wrote: > Yes, Alan's comment was correct. Using the correct Zookeeper string made > things work correctly, e.g.: > > SOLR_ZK_ENSEMBLE=zookeeper1:2181,zookee

Re: Solr JOIN: keeping permission data out of primary documents

2014-11-19 Thread Philip Durbin
On Wed, Nov 19, 2014 at 11:56 AM, Yonik Seeley wrote: > On Wed, Nov 19, 2014 at 9:22 AM, Philip Durbin > wrote: >> On Wed, Nov 19, 2014 at 5:45 AM, Yonik Seeley wrote: >>> On Tue, Nov 18, 2014 at 3:47 PM, Philip Durbin >>> wrote: Solr JOINs are a way to enforce simple document security, as

Re: Solr JOIN: keeping permission data out of primary documents

2014-11-19 Thread Yonik Seeley
On Wed, Nov 19, 2014 at 9:22 AM, Philip Durbin wrote: > On Wed, Nov 19, 2014 at 5:45 AM, Yonik Seeley wrote: >> On Tue, Nov 18, 2014 at 3:47 PM, Philip Durbin >> wrote: >>> Solr JOINs are a way to enforce simple document security, as explained >>> by Yonik Seeley at >>> http://lucene.472066.n3.n

Re: Using Zookeeper with REST URL

2014-11-19 Thread Nazik Huq
Look at the \contrib\rest directory for zookeeper-3.4.5-rest.jar. It may give you REST access to ZK - @nazik_huq On Wed, Nov 19, 2014 at 10:27 AM, Ian Rose wrote: > I don't think zookeeper has a REST api. You'll need to use a Zookeeper > client library in your language (or roll one yourself). >

RE: Solrcloud and remote Zookeeper ensemble

2014-11-19 Thread Robert Kent
Yes, Alan's comment was correct. Using the correct Zookeeper string made things work correctly, e.g.: SOLR_ZK_ENSEMBLE=zookeeper1:2181,zookeeper2:2181,zookeeper3:2181/solr From: Erick Erickson [erickerick...@gmail.com] Sent: 19 November 2014 14:32 To: so

Re: Using Zookeeper with REST URL

2014-11-19 Thread Ian Rose
I don't think zookeeper has a REST api. You'll need to use a Zookeeper client library in your language (or roll one yourself). On Wed, Nov 19, 2014 at 9:48 AM, nabil Kouici wrote: > Hi All, > > I'm connecting to solr using REST API (No library like SolJ). As my solr > configuration is in cloud

Using Zookeeper with REST URL

2014-11-19 Thread nabil Kouici
Hi All, I'm connecting to solr using REST API (No library like SolJ). As my solr configuration is in cloud using Zookeeper ensemble, I don't know how to get available Solr server from ZooKeeper to be used in my URL Call. With SolrJ I can do: String zkHostString = "10.0.1.8:2181"; CloudSolrServ

Re: Solrcloud and remote Zookeeper ensemble

2014-11-19 Thread Erick Erickson
Alan's comment is spot on, and it's the first thing I'd try. Beyond that, though, this forum really doesn't have much knowledge about various company's bundling for Solr and associated support tools so you might get more knowledgeable responses from the Cloudera support forums... Just in case the

Re: SOLR not starting after restart 2 node cloud setup

2014-11-19 Thread Erick Erickson
You've really got to provide details for us to say much of anything. There are about a zillion things that it could be. In particular, have you looked at the solr logs? Are there any interesting things in them? How big are the cores? How much memory are you allocating the JVM? How many docs in the

Re: Solr JOIN: keeping permission data out of primary documents

2014-11-19 Thread Philip Durbin
On Wed, Nov 19, 2014 at 5:45 AM, Yonik Seeley wrote: > On Tue, Nov 18, 2014 at 3:47 PM, Philip Durbin > wrote: >> Solr JOINs are a way to enforce simple document security, as explained >> by Yonik Seeley at >> http://lucene.472066.n3.nabble.com/document-level-security-filter-solution-for-Solr-tp4

Re: solrcloud collection info

2014-11-19 Thread Shawn Heisey
On 11/19/2014 6:38 AM, Bernd Fehling wrote: > with my first cloud (Solr 4.10.2) up and running (4 shards, 1 replica) > I can't find any info with Solr Admin about my collection, like summary > number of docs or summary index size of my collection. > > Any idea where to find it? The admin UI does

solrcloud collection info

2014-11-19 Thread Bernd Fehling
Hi list, with my first cloud (Solr 4.10.2) up and running (4 shards, 1 replica) I can't find any info with Solr Admin about my collection, like summary number of docs or summary index size of my collection. Any idea where to find it? Regards, Bernd

IndexSearcher not being closed

2014-11-19 Thread Priya Rodrigues
Hi, I am running into OOM errors on Solr 4.6. Our index size is quite small at 5000 documents and 75MB index size. While debugging this OOM, I found that the IndexSearcher Refcount is not coming down to 0 and hence the index searcher is not released on commits. We are using custom request handle

Re: Solrcloud and remote Zookeeper ensemble

2014-11-19 Thread Alan Woodward
> SOLR_ZK_ENSEMBLE=zookeeper1:2181/solr,zookeeper2:2181/solr,zookeeper3:2181/solr This is the incorrect part, it should be: > SOLR_ZK_ENSEMBLE=zookeeper1:2181,zookeeper2:2181,zookeeper3:2181/solr The chroot is only appended at the end of the connection string. Not the way I would have done it,

Solrcloud and remote Zookeeper ensemble

2014-11-19 Thread Robert Kent
Hi, I'm experiencing some odd behaviour with Solrcloud and Zookeeper. I am running Solrcloud on one host and am running three Zookeepers on another three hosts. The Zookeeper part of things works correctly, I can add/remove/etc nodes from Zookeeper. I am running, or rather trying to run, Sol

Re: Solr JOIN: keeping permission data out of primary documents

2014-11-19 Thread Yonik Seeley
On Tue, Nov 18, 2014 at 3:47 PM, Philip Durbin wrote: > Solr JOINs are a way to enforce simple document security, as explained > by Yonik Seeley at > http://lucene.472066.n3.nabble.com/document-level-security-filter-solution-for-Solr-tp4126992p4126994.html > > I'm trying to tweak this pattern so t

SOLR not starting after restart 2 node cloud setup

2014-11-19 Thread Doss
I have two node SOLR (4.9.0) cloud with Tomcat (8), Zookeeper. At times SOLR in Node 1 stops responding, to fix the issue I am restarting tomcat in Node 1, but SOLR not starting up, but if I remove the solr cores in both nodes and try restarting it starts working, and then I have to reindex the who