Re: SolrJ 5.5 won't work with any of my servers

2016-02-29 Thread Shai Erera
Shawn, not sure if it's the same case as yours, but I've hit NPEs upgrading to 5.5 too. In my case though, SolrJ talks to a proxy servlets before the request gets routed to Solr, and that servlet didn't handle binary content stream well. I had to add another resource method to the servlet which ha

Re: SolrJ 5.5 won't work with any of my servers

2016-03-01 Thread Shai Erera
Ah ok, in my case even 5.4.1 didn't work with binary request writer, so probably we don't face the same issue. Shai On Tue, Mar 1, 2016, 17:07 Shawn Heisey wrote: > On 2/29/2016 9:14 PM, Shai Erera wrote: > > Shawn, not sure if it's the same case as yours, but I'v

Re: NoSuchFileException errors common on version 5.5.0

2016-03-11 Thread Shai Erera
Hey Shawn, I added segments file information (name and size) to Core admin status API. Turns out that you might get into NoSuchFileException if indexing happens and the commit point has changed, but the IndexReader LukeRequestHandler receives hasn't picked up the new commit yet, in which case the

Re: Problem with CoreAdmin API CREATE command

2015-09-17 Thread Shai Erera
Solr 5.3 can read Solr 4.10.4 indexes as-is. Why are you trying to upgrade the indexes in the first place? Shai On Thu, Sep 17, 2015 at 3:05 PM, Yago Riveiro wrote: > I have a very old index with more than 12T (re-index data is not an option > ...) that I want upgrade to 5.3, I’m using lucene-c

Re: Problem with CoreAdmin API CREATE command

2015-09-17 Thread Shai Erera
pgraded > to 4.10.4 with source code of 5.3 worked as expected with any issue. > > > —/Yago Riveiro > > On Thu, Sep 17, 2015 at 1:08 PM, Shai Erera wrote: > > > Solr 5.3 can read Solr 4.10.4 indexes as-is. Why are you trying to > upgrade > > the indexes in the first pla

Re: solr4.7: leader core does not elected to other active core after sorl OS shutdown, known issue?

2015-09-21 Thread Shai Erera
I don't think the process Shalin describes applies to clusterstate.json. That JSON object reflects the status Solr "knows" about, or "last known status". When Solr is properly shutdown, I believe those attributes are cleared from clusterstate.json, as well the leaders give up their lease. However,

Auto naming replicas via ADDREPLICA

2015-03-23 Thread Shai Erera
Hi I have a Solr cluster started (all programmatically) with one Solr node, one collection and one shard. I set the replicationFactor to 1. The name of the result core was set to mycollection_shard1_replica1. I then start a second Solr node and issue an ADDREPLICA command as described in the refe

Re: Auto naming replicas via ADDREPLICA

2015-03-23 Thread Shai Erera
be nice if someone wanted to control the name of the added replica, but otherwise should not be included in the request Or maybe we fix the bug by doing #1 and consider #2 as a new feature "allow naming replicas"? Shai On Mon, Mar 23, 2015 at 6:14 PM, Shawn Heisey wrote: > On 3/23

maxReplicasPerNode

2015-03-24 Thread Shai Erera
Hi I saw that we can define maxShardsPerNode when creating a collection, but I don't see that I can set something similar for replicas. My scenario is the following: - I setup one Solr node - Create collection with numShards=1 and replicationFactor=2 - Hopefully, one replica is created o

Re: maxReplicasPerNode

2015-03-24 Thread Shai Erera
; https://cwiki.apache.org/confluence/display/solr/Running+Solr+on+HDFS > > About #3, i line with my answer to the previous question, Solr wouldn't > auto-add a Replica to meet the replication factor when a node goes down. > > > On Tue, Mar 24, 2015 at 12:36 AM, Shai Erera wro

How to verify a document is indexed by all replicas

2015-03-24 Thread Shai Erera
Hi Is there a recommended, preferably fast, way to check that a document is indexed by all replicas? I currently do that by issuing a search request to each replica, but was wondering if there's a faster way. Even better, is there a way to verify all replicas of a shard are "up-to-date", e.g. by

Re: How to verify a document is indexed by all replicas

2015-03-24 Thread Shai Erera
to > the tlogs, and since the tlogs will be replayed in the event of a > problem your data should be fine. Of course if you're indexing at a > very fast rate and your tlog is huge, it'll take a while.... > > FWIW, > Erick > > On Tue, Mar 24, 2015 at 4:59 AM, Shai Erera

Re: maxReplicasPerNode

2015-03-24 Thread Shai Erera
. Solr wouldn't auto-add replicas in either of those > cases (or any other case) to meet the rf specified at create time. > > On Tue, Mar 24, 2015 at 2:22 AM, Shai Erera wrote: > > > Thanks Anshum, > > > > About #3, i line with my answer to the previous ques

Re: Auto naming replicas via ADDREPLICA

2015-03-24 Thread Shai Erera
gt; Can you confirm what version of Solr are you using? > > If it turns out to be a bug in 5x/trunk I'll create a JIRA and fix it to > both #1 and #2. > > On Mon, Mar 23, 2015 at 9:48 AM, Shai Erera wrote: > > > Shawn, that was a great tip! > &g

Re: How to verify a document is indexed by all replicas

2015-03-24 Thread Shai Erera
-- You can add a min_rf=true parameter to your indexing > request and then Solr will add information to the response about how many > replicas gave an ack' to the leader. So if the returned number is equal to > the number of replicas, you can be sure that the doc has been indexed &

Re: rough maximum cores (shards) per machine?

2015-03-24 Thread Shai Erera
While it's hard to answer this question because as others have said, "it depends", I think it will be good of we can quantify or assess the cost of running a SolrCore. For instance, let's say that a server can handle a load of 10M indexed documents (I omit search load on purpose for now) in a sing

Replica and node states

2015-03-25 Thread Shai Erera
Hi Is it possible for a replica to be DOWN, while the node it resides on is under /live_nodes? If so, what can lead to it, aside from someone unloading a core. I don't know if each SolrCore reports status to ZK independently, or it's done by the Solr process as a whole. Also, is it possible for

Re: Replica and node states

2015-03-25 Thread Shai Erera
> > On Wed, Mar 25, 2015 at 8:30 AM, Shai Erera wrote: > > > Hi > > > > Is it possible for a replica to be DOWN, while the node it resides on is > > under /live_nodes? If so, what can lead to it, aside from someone > unloading > > a core. > > &

Re: Replica and node states

2015-03-25 Thread Shai Erera
015 at 11:39 PM, Shalin Shekhar Mangar < shalinman...@gmail.com> wrote: > On Wed, Mar 25, 2015 at 12:51 PM, Shai Erera wrote: > > > Thanks. > > > > Does Solr ever clean up those states? I.e. does it ever remove "down" > > replicas, or replicas belonging to

Re: Unable to update config file using zkcli or RELOAD

2015-04-04 Thread Shai Erera
I was looking into a similar issue when I saw this thread. I wrote a simple test which uploads a configuration to ZK, creates a collection and indexes a document. Then I delete the configuration and I see these in the logs, which is expected: 6299 [Thread-13] INFO org.apache.solr.cloud.ZkControll

Re: Information regarding "This conf directory is not valid" SolrException.

2015-04-16 Thread Shai Erera
I opened SOLR-7408 to track that. Shai On Mon, Apr 13, 2015 at 3:31 PM, Bar Weiner wrote: > After some additional debugging, I think that this issue is caused by a > possible race condition introduced to ZkController in Solr-5.0.0. > > My concerns are around unregister(...) function in ZkContro

Verify a certain Replica contains a document

2015-05-15 Thread Shai Erera
Hi Is there a REST API in Solr that allows me to query a certain Replica/core? I am writing some custom replica-recovery code and I'd like to verify that it works well. I wanted to use the /collection/select API, passing shards=host.under.test:ip/solr/collection, but that also works even if 'host

Re: Verify a certain Replica contains a document

2015-05-15 Thread Shai Erera
8:08 AM, Anshum Gupta wrote: > Did you also try querying /core.name/select with distrib=false ? > > On Fri, May 15, 2015 at 9:22 PM, Shai Erera wrote: > > > Hi > > > > Is there a REST API in Solr that allows me to query a certain > Replica/core? > > I am wri

Re: Limitation on Collections Number

2015-06-14 Thread Shai Erera
Thanks Jack for your response. But I think Arnon's question was different. If you need to index 10,000 different collection of documents in Solr (say a collection denotes someone's Dropbox files), then you have two options: index all collections in one Solr collection, and add a field like collect

Re: Limitation on Collections Number

2015-06-14 Thread Shai Erera
wrote: > Yes, there are some known problems while scaling to large number of > collections, say 1000 or above. See > https://issues.apache.org/jira/browse/SOLR-7191 > > On Sun, Jun 14, 2015 at 8:30 PM, Shai Erera wrote: > > > Thanks Jack for your response. But I th

Re: fq versus q

2015-06-24 Thread Shai Erera
Thanks Shawn, What's Solr equivalence to ConstantScoreQuery? I.e., what if you want to run a query that does not score, but only filter. The rationale behind using a non-cached 'fq' was just that. Shai On Wed, Jun 24, 2015 at 4:29 PM, Shawn Heisey wrote: > On 6/24/2015 5:28 AM, Esther Goldbrai

Re: fq versus q

2015-06-24 Thread Shai Erera
equest a constant score query in Solr with the ^= > operator. > > For example: +color:blue^=1 text:shoes > > See: > https://issues.apache.org/jira/browse/SOLR-7218 > > -- Jack Krupansky > > On Wed, Jun 24, 2015 at 1:41 PM, Shai Erera wrote: > > > T

Re: fq versus q

2015-06-25 Thread Shai Erera
wrong) if it means > 200ms .vs. 1,000ms. > > Best, > Erick > > On Wed, Jun 24, 2015 at 5:30 PM, Upayavira wrote: > > Are you wanting to do no scoring at all, or just have a portion of the > > query not contribute to the score? > > > > If you don'

Re: Programmatically find out if node is overseer

2015-07-16 Thread Shai Erera
An easier way (IMO) and more 'official' is to use the CLUSTERSTATUS ( https://cwiki.apache.org/confluence/display/solr/Collections+API#CollectionsAPI-api18) or OVERSEERSTATUS ( https://cwiki.apache.org/confluence/display/solr/Collections+API#CollectionsAPI-api17) API. The OVERSEERSTATUS returns a

Re: Programmatically find out if node is overseer

2015-07-17 Thread Shai Erera
> > Also, ideally, there shouldn't be a point where you have multiple active > Overseers in a single cluster. > In the reference guide, CLUSTERSTATUS shows as if the overseer role can return more than one node. Does it mean that these nodes were designated potential 'overseers', but OVERSEERSTATUS

Re: File Descriptor/Memory Leak

2016-07-07 Thread Shai Erera
Shalin, we're seeing that issue too (and actually actively debugging it these days). So far I can confirm the following (on a 2-node cluster): 1) It consistently reproduces on 5.5.1, but *does not* reproduce on 5.4.1 2) It does not reproduce when SSL is disabled 3) Restarting the Solr process (som

Re: File Descriptor/Memory Leak

2016-07-10 Thread Shai Erera
uli 2016 18.14 > To: solr-user@lucene.apache.org > Subject: Re: File Descriptor/Memory Leak > > I've created a JIRA to track this: > https://issues.apache.org/jira/browse/SOLR-9290 > > On Thu, Jul 7, 2016 at 8:00 AM, Shai Erera wrote: > > > Shalin, we're see