Re: Solr 5.2.1 deadlock on commit

I really appreciate if somebody can help me to solve this problem. Regards. On Tue, Dec 8, 2015 at 9:22 PM, Ali Nazemian wrote: > I did that already. The situation was worse. The autocommit part makes > solr unavailable. > On Dec 8, 2015 7:13 PM, "Emir Arnautovic" > wrote: > >> Hi Ali, >> Can y

Re: Solr 5.2.1 deadlock on commit

Hi Ali, Is Solr busy at that time and eventually recover or it is deadlocked? Can you provide full thread dump when it happened? Do you run only indexing at that time? Is "unavailable" only from indexing perspective, or you cannot do anything with Solr? Is there any indexing scenario that does n

Can Solr5.3 support multiple Geographical enveloppe

Dear All, do you know if Solr5.3 is supposed to support multiple Geographical enveloppe search ? Indeed, we have a bbox index defined as follow : multiValued="true"/> distanceUnits="degrees" numberType="_bbox_coord" /> precisionStep="8" docValues="true" stored="false"/> When we

NRT vs Redis for Dynamic Data in SOLR (like counts, viewcounts, etc) -

We are creating a web application which would contain posts (something like FB or say Youtube). For the stable part of the data (i.e.the facets, search results & its content), we plan to use SOLR. What should we use for the unstable part of the data (i.e. dynamic and volatile content such as Like

Re: Indexing of annotated corpora

Let me answer in line : On 10 December 2015 at 06:11, Emmanuel CARTIER < emmanuel.cart...@lipn.univ-paris13.fr> wrote: > Hi, > > I am a newbie in Solr and I would like to know > > 1. The most efficient way(s?) to index annotated corpora with Linguistic > information at the token and chunk levels.

Nested document query with wrong numFound value

Hi, I'm playing with the nested documents feature and after run this query: http://localhost:8983/solr/ecommerce-15/query?q=id:3181426982318142698228* The documents has the IDs: - Parent : 3181426982318142698228 - Child_1 : 31814269823181426982280 - Child_2 : 31814269823181426982281 I have t

Re: capacity of storage a single core

Susheel, this is a very good idea. I am a little bit busy this period, so I doubt I can contribute with a blog post, but it would be great if anyone has time. If not I will add it to my backlog and sooner or later I will do it :) Furthermore latest observations from Erick are pure gold, and I agre

Re: NRT vs Redis for Dynamic Data in SOLR (like counts, viewcounts, etc) -

Hi Vikram, sounds like you're using those "dynamic" fields only for visualization (i.e. you don't need to have them "indexed")...this is the big point that could make the difference. If the answer is yes, about the first option (NOTE: I don't know Redis and that plugin), a custom SearchComponent w

Schema API, change the defaultoperator

Hi, How can I change the defaultoperator parameter through the schema API? Thanks. - Best regards -- View this message in context: http://lucene.472066.n3.nabble.com/Schema-API-change-the-defaultoperator-tp4244857.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: capacity of storage a single core

On Thu, 2015-12-10 at 14:43 -0500, Susheel Kumar wrote: > Like the details here Eric how you broke memory into different parts. I > feel if we can combine lot of this knowledge from your various posts, above > sizing blog, Solr wiki pages, Uwe article on MMap/heap, consolidate and > present in at

how to secure standalone solr

Hello, Anyone told me how to secure standalone solr . 1.)using Kerberos Plugin is a good practice or any other else. -- View this message in context: http://lucene.472066.n3.nabble.com/how-to-secure-standalone-solr-tp4244866.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: Nested document query with wrong numFound value

what do you see with debugQuery=true ? On Fri, Dec 11, 2015 at 2:02 PM, Yago Riveiro wrote: > Hi, > > I'm playing with the nested documents feature and after run this query: > > http://localhost:8983/solr/ecommerce-15/query?q=id:3181426982318142698228* > > The documents has the IDs: > > - Paren

Re: NRT vs Redis for Dynamic Data in SOLR (like counts, viewcounts, etc) -

You can consider DataStax Enterprise (DSE) which deeply integrates Solr (not just a plugin) with the Cassandra database (DSE Search): http://www.datastax.com/products/datastax-enterprise-search Solr's Join queries are supported across tables in DSE Search, so you could keep dynamic data in a separ

Re: Nested document query with wrong numFound value

This: { responseHeader: { status: 0, QTime: 10, params: { q: "id:3181426982318142698228*", debugQuery: "true" } }, response: {

Re: Schema API, change the defaultoperator

On 12/11/2015 4:23 AM, Yago Riveiro wrote: > How can I change the defaultoperator parameter through the schema API? The default operator and default field settings in the schema have been deprecated for quite some time, so I would imagine that you can't change them with the schema API -- they shou

Re: Authorization API versus zkcli.sh

Shouldn't this be the znode version? Why put a version in security.json? Or is the idea that the user will upload security.json only once and then use the security APIs for all further changes? On Fri, Dec 11, 2015 at 11:51 AM, Noble Paul wrote: > Please do not put any number. That number is used

Re: Authorization API versus zkcli.sh

yes, that's the assumption. The reason why there's a version there is to optimize on reloads i.e. Authentication and Authorization plugins are reloaded only when the version number is changed. e.g. * Start with Ver 1 for both authentication and authorization * Make changes to Authentication, the ve

Re: capacity of storage a single core

Thanks, Alessandro. We can attempt to come up with such a blog and I can volunteer for bullets/headings to start with. I also agree that we can can't come up with some definitive answer as mentioned in other places but can give an attempt to at least consolidate all these knowledge into one place.

Re: Schema API, change the defaultoperator

I uploaded a schema.xml manualy with the defaultoperator configuration and it's working. My problem is that my legacy application is huge and I can't go to all places to add the q.op parameter. The solrconfig.xml option should be an option. The q.op param defined in request handlers work

RE: Use multiple istance simultaneously

Thanks for all of your clarification. I know that solrcloud is a really better configuration than any other, but actually it has a complexity that is really higher. I just want to give you the pain point I've noticed while I was gathering all the info I can got on SolrCloud. 1) zookeeper documenta

Block Join query

Hi Guys, I'm trying block join query, so I have tried +{!parent which="doctype:200"}flow:624 worked fine. But when i tried +{!parent which="doctype:200"}flow:[624 TO 700] Got the below error org.apache.solr.search.SyntaxError: Cannot parse 'flow_l:[624': Encountered \"\" at line 1, c

Re: Solr 6 Distributed Join

Akiel, Without seeing your full url I assume that you're missing the stream=innerJoin(.) part of it. A full sample url would look like this http://localhost:8983/solr/careers/stream?stream=innerJoin(search(careers, fl="personId,companyId,title", q=companyId:*, sort="companyId asc",zkHost="loca

RE: Authorization API versus zkcli.sh

So, when one has finished constructing the desired security.json (by means of Authentication and Authorization commands) and then run "zkcli.sh -cmd getfile" to get this security.json in order for it to be used as a template: one should edit the template to remove this "":{"v":85} clause (and th

Re: Schema API, change the defaultoperator

On 12/11/2015 8:02 AM, Yago Riveiro wrote: > I uploaded a schema.xml manualy with the defaultoperator configuration and > it's working. > > My problem is that my legacy application is huge and I can't go to all places > to add the q.op parameter. > > The solrconfig.xml option should be an optio

Re: Joins with SolrCloud

Mugeesh, You can use Streaming Aggregation to provide various types of cross-collection joins. This is currently available in trunk and will be a part of Solr 6. To follow with your example, let's assume the following setup: Restaurants: avail on machine1:8983 with 3 shards, zk at zk1:2345 Users:

Re: Joins with SolrCloud

Something I forgot to mention - the collection shards can live on any number of machines, anywhere in the world. As long as the clusterstate in zk knows where the shard can be found (ie, a basis of SolrCloud) then everything will work. The example I gave had the shards living on the same machine bu

Re: Use multiple istance simultaneously

On 12/11/2015 8:19 AM, Gian Maria Ricci - aka Alkampfer wrote: > Thanks for all of your clarification. I know that solrcloud is a really > better configuration than any other, but actually it has a complexity that > is really higher. I just want to give you the pain point I've noticed while > I was

Solrcloud 4.8.1 - Solr cores reload

Hi all, in day by day work, often I need to change the solr configurations files. Often adding new synonyms, changing the schema or the solrconfig.xml. Everything is stored in zookeeper. But I have inherited a piece of code that, after every change, reload all the cores using CoreAdmin API. Now

SolrCloud 4.8.1 - commit wait

Hi all, I have a SolrCloud cluster with a collection (2.5M docs) with 3 shards and 15 replicas. There is a solrj application that feeds the collection, updating few documents every hour, I don't understand why, at end of process, the hard commit takes about 8/10 minutes. Even if there are only fe

Unable to create lot of cores -- Failing after 100 cores

Hello all, We need to create around 300 collections with replication factor 2. But after creating 100, we couldn't create more and most of them are in RED state in the solrcloud. Is this issue related to zookeeper jute.maxBuffer issue? If so, how can we increase the size of zookeeper maxbuffer

How Json facet API works with domains and facet functions?

Hi, How the json facet api works with domains and facet functions? I try to google some info and I do not find nothing useful. How can do a query that find all parents that match a clause (a date) and calculate the avg price of all of children that have property X? Following yonik's blog exampl

Re: How Json facet API works with domains and facet functions?

If you search on the parents and want to match child documents, I think you want {!child} and not {!parent} in your queries or filters. fq={!child of=...}date_query_on_parents fq=child_prop:X For this specific example, you don't even need the block-join support in facets since the base domain (qu

Re: how to secure standalone solr

>Anyone told me how to secure standalone solr . Recently there were few discussion on this. In short, it is not tested and there doesn’t seem to a plan to test it. >1.)using Kerberos Plugin is a good practice or any other else. The answer depends how you are using it. Where you are deploying it,

Re: How Json facet API works with domains and facet functions?

One more question. It’s posisble use the domain clause in json facet without a term query? Ex. json.facet={     x:'avg(price)',    domain: { blockChildren : "parent_type:ecommerce”} } This make any sense, or I always should reduce the domain using the query and filters. —/Yag

Re: Joins with SolrCloud

You can also do the innerJoin in parallel across worker nodes using the parallel function: hashJoin( parallel(workerCollection, innerJoin( search(users, q="*:*", fl="userId, full_name, hometown", sort="userId a

SolrCloud page is blank

Hi All, We have set up a solr 5.3.1. Now I realize that in the solr admin UI, the cloud page is blank. What could be the reason behind this? Following are the exceptions that I’m seeing in the logs 12/11/2015, 9:58:37 AM WARN null ClientCnxn Session 0x25111a5595ab885 for server null,​ unexp

Re: Unable to create lot of cores -- Failing after 100 cores

A quick Google search shows the following: "you must set -Djute.maxbuffer in zookeeper and solr..." What have you tried? What were the results? What version of Solr are you using? 5.x defaults to an individual state.json file per collection rather than one big one for all collections, that will

Re: SolrCloud 4.8.1 - commit wait

First of all, your autocommit settings are _very_ aggressive. Committing every second is far to frequent IMO. As an aside, I generally prefer to omit the maxDocs as it's not all that predictable, but that's a personal preference and really doesn't bear on your problem.. My _guess_ is that you are

Re: Solrcloud 4.8.1 - Solr cores reload

You should absolutely always use the Collection API rather than any core admin API if at all possible. If for no other reason than your client will be _lots_ simpler (i.e. you don't have to find all the replicas and issue the core admin RELOAD command for each one). I'm not entirely sure whether t

Re: Nested document query with wrong numFound value

Ok. I got it. SolrCloud relies on uniqueKey (id) for merging shard results, but in your examples it doesn't work, because nested documents disables this. And you have duplicates, which make merge heap mad: false}

Re: Block Join query

Novin, I regret so much. It's my pet peeve in Solr query parsing. Handling s space is dependent from the first symbol of query sting This will work (starts from '{!' ): q={!parent which="doctype:200"}flow:[624 TO 700] These won't due to " ", "+": q= {!parent which="doctype:200"}flow:[624 TO 700] q

Re: Nested document query with wrong numFound value

When do you say that I have duplicates, what do you mean?  If I have duplicate documents is not intentional, each document must be unique. Running a query for each id: - Parent :  3181426982318142698228 - Child_1 : 31814269823181426982280 - Child_2 : 31814269823181426982281 The resul

Re: Unable to create lot of cores -- Failing after 100 cores

Thanks Eric, This is what we did to the zookeeper and solr settings.. Still, we are not seeing the improvement in the collection creation.. it takes lot of time to see the collection on the Solrcloud. added the following line in zkServer.sh export JVMFLAGS="$JVMFLAGS -Xms256m -Xmx1g -Djute.m

Re: Unable to create lot of cores -- Failing after 100 cores

Which version of Solr are you using? As Erick, said, use the latest 5.3.1 release, it is much more better in handling many collections. On Sat, Dec 12, 2015 at 1:39 AM, Venkat Paranjothi wrote: > Thanks Eric, > > This is what we did to the zookeeper and solr settings.. Still, we are not > seeing

Re: Solr memory usage

Hi Steve, Fluctuation is OK. 100% utilization for more than a moment is not :) Not sure what tool(s) you use for monitoring your Solr servers, but look under "JVM Pool Utilization" in SPM if you're using SPM. Or this live demo of a Solr system: * click on https://apps.sematext.com/demo to get in

API accessible without authentication even though Basic Auth Plugin is enabled

Hi, I noticed that it is possible to access the API even if the Basic Auth plugin is enabled. Is that a known issue/done on purpose? I didn’t find anything in JIRA or the docs. What I did: - Started zookeeper on port 2181 and uploaded security.json from https://cwiki.apache.org/confluence/disp

Re: Unable to create lot of cores -- Failing after 100 cores

Thanks shalin.. Sorry i forgot to mention the verison.. 4.6.1 + tomcat6 + zookeeper 3.4.5 Thanks, Venkat Paranjothi Kenexa 2xB Software Engineering Team

Re: Nested document query with wrong numFound value

On Fri, Dec 11, 2015 at 11:05 PM, Yago Riveiro wrote: > When do you say that I have duplicates, what do you mean? > I mean http: //node-01:8983/solr/ecommerce-15_shard1_replica2/: { QTime: "0", ElapsedTime: "2", Req

Re: Unable to create lot of cores -- Failing after 100 cores

You seem to have changed the problem statement from being unable to create cores to it takes a long time for them to show up. What's the current problem you're having? Best, Erick On Fri, Dec 11, 2015 at 12:45 PM, Venkat Paranjothi wrote: > Thanks shalin.. > > Sorry i forgot to mention the ver

Re: Unable to create lot of cores -- Failing after 100 cores

Eric, Sorry for the confusion. Noticed both issues, some times collection was showing in RED in the solrcloud.. if i leave this for a day, next day it shows GREEN. This is not true for all the cases and not seeing the consistent behavior. After restarting the zookeeper & tomcat ser

RE: JSON facets and excluded queries

Answering one question myself after doing some testing on 5.3.1: Yes, facet.threads is still relevant with Json facets. We are seeing significant gains as we are increasing the number of threads from 1 up to 4. Beyond that we only observed marginal improvements -- which makes sense because t

Re: Nested document query with wrong numFound value

Mmmm, In fact, if I running a json facet query the result count is 5 for both of them, this is consistent with the debug query. What I don't understand is from where these documents are. I pre-clean the colection several time with a delete query (id:*) and index always  3181426982318142

Re: SolrCloud 4.8.1 - commit wait

He has waitSearcher as false it looks, so all the time should be in the commit. So that amount of time does sound odd. I would certainly change those commit settings though. I would not use maxDocs, that is an ugly way to control this. And one second is much too aggressive as Erick says. If you w

Re: JSON facets and excluded queries

Do note that the number of threads also won't help much last I knew unless you are faceting over that many fields too. I.e. setting this to 5 while faceting on only 1 field won't help. And it's not implemented for all facet types IIRC. Best, Erick On Fri, Dec 11, 2015 at 1:07 PM, Aigner, Max wr

Re: API accessible without authentication even though Basic Auth Plugin is enabled

Ugh ... no sure WTF is going on here, but that's for reporting it with clear steps to reproduce... https://issues.apache.org/jira/browse/SOLR-8408 : Date: Fri, 11 Dec 2015 20:43:46 +0100 : From: Kristine Jetzke : Reply-To: solr-user@lucene.apache.org : To: solr-user@lucene.apache.org : Subject

Re: Block Join query

No Worries, I was just wondering what did I miss. And thanks for blog link. On 11/12/2015 18:52, Mikhail Khludnev wrote: Novin, I regret so much. It's my pet peeve in Solr query parsing. Handling s space is dependent from the first symbol of query sting This will work (starts from '{!' ): q={!

In Solr 5.3.0, how to load customized analyzer jar file ?

Hi All , I switched from solr 4.x version to solr 5.3.0 . And I am creating a core and run it as standalone mode , not cloud mode . I want to know , how to load those external jar file , for example , my customized analyzer or filter ? I add a in solrconfig.xml , for example : However , it

Re: In Solr 5.3.0, how to load customized analyzer jar file ?

Hi, Apparently best way thing to do is create lib directory under the solr home directory. Jars in this directory loaded automatically. No need a solrconfig.xml entry. thanks, Ahmet On Saturday, December 12, 2015 2:05 AM, Mingzhu Gao wrote: Hi All , I switched from solr 4.x version to solr

RE: JSON facets and excluded queries

Good to know, thank you. From an implementation standpoint that makes a lot of sense. We are only using facets of type 'term' for now and for those it works nicely. Our usual searches carry around 8-12 facets so we are covered from that side :-) -Original Message- From: Erick Erickso

Re: SolrCloud 4.8.1 - commit wait

Thanks Erick, Mark, I'll raise maxTime asap. Just to be sure understand, given that I have openSearcher=false, I suppose it shouldn't trigger autowarming at least until a commit is executed, shouldn't it? Anyway, I don't understand, given that maxTime is very aggressive, why hard commit takes so

Re: API accessible without authentication even though Basic Auth Plugin is enabled

It works as designed. Protect the read path using the following command curl http://localhost:8983/solr/admin/authorization -H 'Content-type:application/json' -d '{ set-permission : {name : read, role : admin}}' Then, you will have the right experience In this case /select is not protected. So a

Re: Authorization API versus zkcli.sh

Oakley, 1) ideally you should only upload the first empty security.json. In that case there is no need to add the version attributes. Thereafter you are supposed to use the API 2) Just in case you need to upload the security.json, please remove that attribute shalin: The version is added inside t

Re: how to secure standalone solr

For standalone Solr , Kerberos is the only option for authentication. If you have a SolrCloud setup, you have other options https://cwiki.apache.org/confluence/display/solr/Basic+Authentication+Plugin https://cwiki.apache.org/confluence/display/solr/Rule-Based+Authorization+Plugin On Fri, Dec 11

Re: In Solr 5.3.0, how to load customized analyzer jar file ?

Thanks Ahmet , You mean solr home directory , root of solr or {solr_root}/bin ? I try both , it doesn¹t work . Can anybody has any other idea ? Thanks, -Judy On 12/11/15, 4:10 PM, "Ahmet Arslan" wrote: >Hi, > >Apparently best way thing to do is create lib directory under the solr >home dir

Re: SolrCloud 4.8.1 - commit wait

Hi All, an update, I have switched logging from WARN to INFO for all except for those two: - org.apache.solr.core - org.apache.solr.handler.component.SpellCheckComponent Well, looking at log file I'm unable to find any autowarm log line, even after few updates and commits. Looking at solrconfig

Re: Solrcloud 4.8.1 - Solr cores reload

Thanks for your suggestion Erick, I'm changing the code and I'll use the Collections API RELOAD. I have done few test changing synonyms dictionary or solrconfig and everything works fine. Well, I think you already know, but looking at solr.log file after the collections api reload call, I have see

Re: In Solr 5.3.0, how to load customized analyzer jar file ?

Hi Judy, It is where solr.xml file resides. Ahmet On Saturday, December 12, 2015 3:30 AM, Mingzhu Gao wrote: Thanks Ahmet , You mean solr home directory , root of solr or {solr_root}/bin ? I try both , it doesn¹t work . Can anybody has any other idea ? Thanks, -Judy On 12/11/15, 4:10

Getting a document version back after updating

Is there a way I can get the version of a document back in response after adding or updating the document via Solrj 5.2.1?

Re: In Solr 5.3.0, how to load customized analyzer jar file ?

I am using Solr 5.3.1 and used the lib directive in solrconfig to load one of my search components and that worked fine. So, I don't think that anything has changed here. What is the stacktrace of the errors you get? On Sat, Dec 12, 2015 at 9:32 AM Ahmet Arslan wrote: > Hi Judy, > > It is where