Re: Performance testing on SOLR cloud

2015-11-18 Thread Emir Arnautovic
Hi Aswath, It is not common to test only QPS unless it is static index most of the time. Usually you have to test and tune worst case scenario - max expected indexing rate + queries. You can get more QPS by reducing query latency or by increasing number of replicas. You manage latency by tunin

Re: Security Problems

2015-11-18 Thread Upayavira
Not sure I quite understand. You're saying that the cost for the UI is not large, but then suggesting we protect just one resource (/admin/security-check)? Why couldn't we create the permission called 'admin-ui' and protect everything under /admin/ui/ for example? Along with the root HTML link to

Re: search for documents where all words of field present in the query

2015-11-18 Thread Ahmet Arslan
Hi Jim, I think you could do some magic with function queries. https://cwiki.apache.org/confluence/display/solr/Function+Queries Index number of unique words in the product title e.g. title = john smith length = 2 return products if the number of matching terms equals to the number of words

Upgrading from 4.x to 5.x

2015-11-18 Thread Daniel Miller
Hi! I'm a very inexperienced user with Solr. I've been using Solr to provide indexes for my Dovecot IMAP server. Using version 3.x, and later 4.x, I have been able to do so without too much of a challenge. However, version 5.x has certainly changed quite a bit and I'm very uncertain how to

Re: CloudSolrClient Connect To Zookeeper with ACL Protected files

2015-11-18 Thread Alan Woodward
At the moment it seems that it's only settable via System properties - see https://cwiki.apache.org/confluence/display/solr/ZooKeeper+Access+Control. But it would be nice to do this programmatically as well, maybe worth opening a JIRA ticket? Alan Woodward www.flax.co.uk On 17 Nov 2015, at 1

Re: Solr Search: Access Control / Role based security

2015-11-18 Thread Charlie Hull
On 18/11/2015 07:55, Noble Paul wrote: I haven't evaluated manifoldCF for this . However , my preference would be to have a generic mechanism in built into Solr to restrict user access to certain docs based on some field values. Relying on external tools make life complex for users who do not lik

Configure it on server

2015-11-18 Thread Prateek Sharma
Hi, Can you help me out how I can configure it on a server? It was configured on one of our servers but I am unable to replicate it. Can you please help. Thanks, Prateek This message and the information contained herein is proprietary and confidential and subject to the Amdocs policy statement

Re: Configure it on server

2015-11-18 Thread Aman Tandon
Hi Prateek, Your question is little ambiguous. Could you please describe it more precisely what you want to configure on server and what is your requirement and problem. This will be more helpful to understand your problem. With Regards Aman Tandon On Wed, Nov 18, 2015 at 4:29 PM, Prateek Sharma

Synchronization Problems

2015-11-18 Thread 马柏樟
Hi, I have encountered some problems with solr-5.3.1. After I initialized the solrcloud and set up BasicAuthPlugin and RuleBasedAuthorizationPlugin, something wrong happened to my solrcloud. I can't Synchronization as usual. The server log as follows: master log Invalid key PKIAuthenticationPlug

Re: Upgrading from 4.x to 5.x

2015-11-18 Thread Jan Høydahl
Hi You could try this Instead of example/, use the server/ folder (it has Jetty in it) Start Solr using bin/solr start script instead of java -jar start.jar … Leave your solrconfig and schema as is to keep back-compat with 4.x. You may need to remove use of 3.x classes that were deprecated in 4.x

Re: Security Problems

2015-11-18 Thread Noble Paul
As of now the admin-ui calls are not protected. The static calls are served by jetty and it bypasses the authentication mechanism completely. If the admin UI relies on some API call which is served by Solr. The other option is to revamp the framework to take care of admin UI (static content) as wel

Re: search for documents where all words of field present in the query

2015-11-18 Thread Alessandro Benedetti
Assuming this is the only, specific kind of search you want, what about using shingles of tokens at query time and keyword tokenizer at indexing time ? Ideally you don't tokenise at indexing time. At query time you build your shingles ( apparently you need not only adiacent token shingles, so play

Re: Security Problems

2015-11-18 Thread Jan Høydahl
I tried out BasicAuthPlugin today. Surprised that not admin UI is protected. But even more surprised that only /select seems to be protected for not logged in users. I can create collections and /update documents without being prompted for pw. My security.json is https://gist.github.com/janhoy/d1

Re: add and then delete same document before commit,

2015-11-18 Thread Matteo Grolla
On Solr 4.10.3 I'm noting a different (desired) behaviour 1) add document x 2) delete document x 3) commit document x doesn't get indexed. The question now is: Can I count on this behaviour or is it just incidental? 2014-11-05 22:21 GMT+01:00 Matteo Grolla : > Perfectly clear, > thanks

Re: add and then delete same document before commit,

2015-11-18 Thread Shawn Heisey
On 11/18/2015 8:21 AM, Matteo Grolla wrote: > On Solr 4.10.3 I'm noting a different (desired) behaviour > > 1) add document x > 2) delete document x > 3) commit > > document x doesn't get indexed. If the last operation for document X is to delete it, then it will be gone after the commit and not

Re: Security Problems

2015-11-18 Thread Noble Paul
Everything requires explicit rules, if you wish to protect "/update/*" create a permission with name "update" and assign a role for the same. If you don't have an explicit rule, those paths are accessible by all On Wed, Nov 18, 2015 at 8:10 PM, Jan Høydahl wrote: > I tried out BasicAuthPlugin tod

Re: add and then delete same document before commit,

2015-11-18 Thread Matteo Grolla
Thanks Shawn, I'm aware that solr isn't transactional and I don't need this property: a single application is indexing. With solr 4.6 I was noting a different behaviour, with 4.10 I'm observing the desired one. I'd like to know If I can count on this behaviour to be maintained by successive solr

Re: CloudSolrClient Connect To Zookeeper with ACL Protected files

2015-11-18 Thread Kevin Lee
Thanks Alan! That works! I was looking for a programatic way to do it, but this will work for now as it doesn’t seem to be supported. - Kevin > On Nov 18, 2015, at 1:24 AM, Alan Woodward wrote: > > At the moment it seems that it's only settable via System properties - see > https://cwiki.ap

Re: add and then delete same document before commit,

2015-11-18 Thread Erick Erickson
Then that was probably a bug in 4.6. There's a lot of work that's been done since then, and distributed updates that are mixed like this are particularly "interesting". So you should be able to count on this. One other possibility: Is it possible that this was a false failure in 4.6 and a commit

Re: add and then delete same document before commit,

2015-11-18 Thread Matteo Grolla
Thanks Erik, I observed the wrong behaviour on 4.6 in a controlled environment with a very simple test case, so It's was probably a bug (or I was drunk ;-) ) Really thanks again!!! 2015-11-18 17:40 GMT+01:00 Erick Erickson : > Then that was probably a bug in 4.6. There's a lot > of work that

Re: Error in log after upgrading Solr

2015-11-18 Thread Shawn Heisey
On 11/17/2015 12:42 AM, Shawn Heisey wrote: > I have upgraded from 5.2.1 to a 5.3.2 snapshot -- the lucene_solr_5_3 > branch plus the patch for SOLR-6188. > > I'm getting errors in my log every time I make a commit on a core. > > 2015-11-16 20:28:11.554 ERROR > (searcherExecutor-82-thread-1-process

unsubscribe me.

2015-11-18 Thread Pramod
please unsubscribe me. Regards, YP

Re: unsubscribe me.

2015-11-18 Thread davidphilip cherian
You should probably send an email to solr-user-unsubscr...@lucene.apache.org Reference links http://lucene.apache.org/solr/resources.html#community https://wiki.apache.org/solr/Unsubscribing%20from%20mailing%20lists On Wed, Nov 18, 2015 at 1:04 PM, Pramod wrote: > please unsubscribe me. > >

Re: Limiting number of parallel queries per user

2015-11-18 Thread deansg
Just an update: my problem turned out to be that in the search-component, I decremented the entry for the user running a query in the first call to finishStage, and didn't realize that most of the query processing and time occurs only in later stages. Because the entry was decremented so quickly,

Implementing security.json is breaking ADDREPLICA

2015-11-18 Thread Oakley, Craig (NIH/NLM/NCBI) [C]
Implementing security.json is breaking ADDREPLICA I have been able to reproduce this issue with minimal changes from an out-of-the-box Zookeeper (3.4.6) and Solr (5.3.1): loading configsets/basic_configs/conf into Zookeeper, creating the security.json listed below, creating two nodes (one with

Re: Security Problems

2015-11-18 Thread Upayavira
I'm very happy for the admin UI to be served another way - i.e. not direct from Jetty, if that makes the task of securing it easier. Perhaps a request handler specifically for UI resources which would make it possible to secure it all in a more straight-forward way? Upayavira On Wed, Nov 18, 201

Boost non stemmed keywords (KStem filter)

2015-11-18 Thread bbarani
Hi, I am using KStem factory for stemming. This stemmer converts 'france to french', 'chinese to china' etc.. I am good with this stemming but I am trying to boost the results that contain the original term compared to the stemmed terms. Is this possible? Thanks, Learner -- View this message

Re: Error in log after upgrading Solr

2015-11-18 Thread Chris Hostetter
: > I'm getting errors in my log every time I make a commit on a core. Do you have any custom plugins? what is the definition of the /lbcheck handler? : > 2015-11-16 20:28:11.554 ERROR : > (searcherExecutor-82-thread-1-processing-x:sparkinclive) [ : > x:sparkinclive] o.a.s.c.SolrCore Previous

RE: Boost non stemmed keywords (KStem filter)

2015-11-18 Thread Markus Jelsma
Hi - easiest approach is to use KeywordRepeatFilter and RemoveDuplicatesTokenFilter. This creates a slightly higher IDF for unstemmed words which might be just enough in your case. We found it not to be enough, so we also attach payloads to signify stemmed words amongst others. This allows you

Re: CloudSolrCloud - Commit returns but not all data is visible (occasionally)

2015-11-18 Thread Mark Miller
If you see "WARNING: too many searchers on deck" or something like that in the logs, that could cause this behavior and would indicate you are opening searchers faster than Solr can keep up. - Mark On Tue, Nov 17, 2015 at 2:05 PM Erick Erickson wrote: > That's what was behind my earlier comment

Re: Error in log after upgrading Solr

2015-11-18 Thread Shawn Heisey
On 11/18/2015 2:20 PM, Chris Hostetter wrote: > : > I'm getting errors in my log every time I make a commit on a core. > > Do you have any custom plugins? > what is the definition of the /lbcheck handler? I have one simple update processor in use that I wrote myself, and we have a third-party plu

RE: CloudSolrCloud - Commit returns but not all data is visible (occasionally)

2015-11-18 Thread Markus Jelsma
Hi - i sometimes see the too many searcher warning to since some 5.x version. The warning cloud has no autoCommit and there is only a single process ever sending a commit, only once every 10-15 minutes orso. The cores are quite small, commits finish quickly and new docs are quickly searchable. I

adding document with nested document require to set id

2015-11-18 Thread CrazyDiamond
i'm trying to add document with the nested objects but don't want id to be generated automatically. When i add document without nesting it's ok.But if i add _childDocuments_ there is an error [doc=null] missing required field: id -- View this message in context: http://lucene.472066.n3.nabble.

Re: CloudSolrCloud - Commit returns but not all data is visible (occasionally)

2015-11-18 Thread Shawn Heisey
On 11/17/2015 1:11 AM, adfel70 Could it be that the parameter waitSearcher=true doesn't really work? or maybe I don't understand something here... I am just guessing with this, but I think this is likely how it works: I believe that if maxWarmingSearchers is exceeded, a commit call will retur

Re: adding document with nested document require to set id

2015-11-18 Thread Alexandre Rafalovitch
If you have id listed as a required field (which I believe you need to anyway), what do you actually get when you add a document without nesting? What does the document echo back? Because if you are getting a document back without id field when it is declared required in the schema, that would be

Problem with Synchronization

2015-11-18 Thread Byzen Ma
Hi, I encountered some problems with solr-5.3.1. After I initialized the solrcloud and set up BasicAuthPlugin and RuleBasedAuthorizationPlugin, something wrong happened to my solrcloud. I can't Synchronization as usual. The server log as follows: master log Invalid key PKIAuthenticationPlugin si

Re: CloudSolrCloud - Commit returns but not all data is visible (occasionally)

2015-11-18 Thread Erick Erickson
bq: Hi - i sometimes see the too many searcher warning to since some 5.x version. The warning cloud has no autoCommit and there is only a single process ever sending a commit, only once every 10-15 minutes orso This is very surprising unless your autowarming is taking 10-15 minutes, almost assured

Shards and Replicas

2015-11-18 Thread Troy Edwards
I am looking for some good articles/guidance on how to determine number of shards and replicas for an index? Thanks

Re: Shards and Replicas

2015-11-18 Thread Shawn Heisey
On 11/18/2015 9:02 PM, Troy Edwards wrote: > I am looking for some good articles/guidance on how to determine number of > shards and replicas for an index? The long version: https://lucidworks.com/blog/sizing-hardware-in-the-abstract-why-we-dont-have-a-definitive-answer/ The short version: Ther

Re: Shards and Replicas

2015-11-18 Thread Jack Krupansky
1. No more than 100 million documents per shard. 2. Number of replicas to meet your query load and to allow for the possibility that a replica might go down. 2 or 3, maybe 4. 3. Proof of concept implementation to validate the number of documents that will query well for a given number of documents

Re: Implementing security.json is breaking ADDREPLICA

2015-11-18 Thread Anshum Gupta
Hi Craig, Just to be sure that you're using the feature as it should be used, can you outline what is it that you're trying to do here? There are a few things that aren't clear to me here, e.g. I see permissions for the /admin handler for a particular collection. What are the kind of permissions