Re: LTR Model size
Please unsubscribe me. I have tried and tried but still get emails On Mar 9, 2018 10:19 AM, "Roopa Rao" wrote: > what is the way to configure the model size for LTR? We have about 3MB > model and Solr is not holding this model as a ManagedResource. > > How can this be configured? > > Thanks, > Roopa >
Re: The Impact of the Number of Collections on Indexing Performance in Solr 6.0
I have manually unsubscribed many times. But I still get emails from the list. Can some admin please unsubscribe me? On Mar 9, 2018 9:52 PM, "苗海泉" wrote: > hello,We found a problem. In solr 6.0, the indexing speed of solr is > influenced by the number of solr collections. The speed is normal before > the limit is reached. If the limit is reached, the indexing speed will > decrease by 50 times. > > In our environment, there are 49 solr nodes. If each collection has 25 > shards, you can maintain high-speed indexing until the total number of > collections is about 900. To reduce the number of collections to the limit, > the speed will increase. Go up. > If each collection is 49 shards, the total number of collections can only > be about 700, exceeding this value will cause the index to drop > dramatically. > In the explanation, we are single copies, and multiple copies will cause > serious stability problems in the large solr cluster environment. > > At first I suspect that it was due to too many thread submissions, and > there are still problems with this method, so I'm inclined to > searcherExecutor thread pool thread. This is just my guess, I want to know > the real reason. Can someone know if I can help? > > Also, I noticed that the searcherExecutor thread and solr collection's > shards basically correspond to each other. How can I reduce the number of > threads or even close it? Although there are many collections in our > environment, there are few queries and it is not necessary to keep the > threads open to provide queries. This is too wasteful. > > thank you . >
Re: The Impact of the Number of Collections on Indexing Performance in Solr 6.0
Wow thanks. Just trying to unsubscribe. Most email lists let u do that On Mar 10, 2018 2:36 PM, "Erick Erickson" wrote: > Spoonerk: > > You say you've tried "many times", but you haven't provided full > header as described in the "problems" link at the link below. You > haven't e-mailed the list owner as suggested in the "problems" link. > You haven't, in short, provided any of the information that's > necessary to actually unsubscribe you. > > Please follow the instructions here: > http://lucene.apache.org/solr/community.html#mailing-lists-irc. In > particular look at the "problems" link. > > You must use the _exact_ same e-mail as you used to subscribe. > > If the initial try doesn't work and following the suggestions at the > "problems" link doesn't work for you, let us know. But note you need > to show us the _entire_ return header to allow anyone to diagnose the > problem. > > Best, > Erick > > On Sat, Mar 10, 2018 at 1:03 PM, spoonerk wrote: > > I have manually unsubscribed many times. But I still get emails from the > > list. Can some admin please unsubscribe me? > > > > On Mar 9, 2018 9:52 PM, "苗海泉" wrote: > > > >> hello,We found a problem. In solr 6.0, the indexing speed of solr is > >> influenced by the number of solr collections. The speed is normal before > >> the limit is reached. If the limit is reached, the indexing speed will > >> decrease by 50 times. > >> > >> In our environment, there are 49 solr nodes. If each collection has 25 > >> shards, you can maintain high-speed indexing until the total number of > >> collections is about 900. To reduce the number of collections to the > limit, > >> the speed will increase. Go up. > >> If each collection is 49 shards, the total number of collections can > only > >> be about 700, exceeding this value will cause the index to drop > >> dramatically. > >> In the explanation, we are single copies, and multiple copies will cause > >> serious stability problems in the large solr cluster environment. > >> > >> At first I suspect that it was due to too many thread submissions, and > >> there are still problems with this method, so I'm inclined to > >> searcherExecutor thread pool thread. This is just my guess, I want to > know > >> the real reason. Can someone know if I can help? > >> > >> Also, I noticed that the searcherExecutor thread and solr collection's > >> shards basically correspond to each other. How can I reduce the number > of > >> threads or even close it? Although there are many collections in our > >> environment, there are few queries and it is not necessary to keep the > >> threads open to provide queries. This is too wasteful. > >> > >> thank you . > >> >
Re: The Impact of the Number of Collections on Indexing Performance in Solr 6.0
I have tried emailing to.unsubscribe. I have tried disrupting threads hoping to anger the admin into getting me out of the spam list. All I get is arrogant emails about headers On Mar 12, 2018 1:15 AM, "苗海泉" wrote: > Thanks Erick and Shawn , Thank you for your patience. I said that the > above phenomenon was caused by the IO, cpu, memory, and network io. The > swap was turned off and the machine's memory was sufficient. When the speed > of indexing is declining, QTime is found to take 3 seconds to 4 seconds to > reload the index, so it can be guessed that it is more likely to be a Solr > problem than a jetty. It is worth mentioning that when the speed of the > index under construction dropped sharply, the Solr used only about 5% of > the CPU, and when it was normal, the CPU usage was 200 percent, and the > overall system's CPU usage was 100 percent. About twenty. > > Basic information: > 1) The data volume of each collection is between 2 billion and 3 billion. > 2) The configuration of the machine is 24 cpu and 128G memory. > 3) The disk usage per copy is about 10G. > > In addition, I noticed that the work of zookeeper is normal and there is no > error or warning message. > > So all these phenomena make me think that the internal specific mechanism > of solr may lead to a sharp drop in the index construction speed. At > present, it seems that our solr's machine resources are sufficient. > > As for the reduction of the number of collections that you said, we also > have this plan, and we are looking for ways to reform it. Are there any > other suggestions? > > > Best . > miaohq > > 2018-03-11 10:15 GMT+08:00 spoonerk : > > > Wow thanks. Just trying to unsubscribe. Most email lists let u do that > > > > On Mar 10, 2018 2:36 PM, "Erick Erickson" > wrote: > > > > > Spoonerk: > > > > > > You say you've tried "many times", but you haven't provided full > > > header as described in the "problems" link at the link below. You > > > haven't e-mailed the list owner as suggested in the "problems" link. > > > You haven't, in short, provided any of the information that's > > > necessary to actually unsubscribe you. > > > > > > Please follow the instructions here: > > > http://lucene.apache.org/solr/community.html#mailing-lists-irc. In > > > particular look at the "problems" link. > > > > > > You must use the _exact_ same e-mail as you used to subscribe. > > > > > > If the initial try doesn't work and following the suggestions at the > > > "problems" link doesn't work for you, let us know. But note you need > > > to show us the _entire_ return header to allow anyone to diagnose the > > > problem. > > > > > > Best, > > > Erick > > > > > > On Sat, Mar 10, 2018 at 1:03 PM, spoonerk > > wrote: > > > > I have manually unsubscribed many times. But I still get emails from > > the > > > > list. Can some admin please unsubscribe me? > > > > > > > > On Mar 9, 2018 9:52 PM, "苗海泉" wrote: > > > > > > > >> hello,We found a problem. In solr 6.0, the indexing speed of solr is > > > >> influenced by the number of solr collections. The speed is normal > > before > > > >> the limit is reached. If the limit is reached, the indexing speed > will > > > >> decrease by 50 times. > > > >> > > > >> In our environment, there are 49 solr nodes. If each collection has > 25 > > > >> shards, you can maintain high-speed indexing until the total number > of > > > >> collections is about 900. To reduce the number of collections to the > > > limit, > > > >> the speed will increase. Go up. > > > >> If each collection is 49 shards, the total number of collections can > > > only > > > >> be about 700, exceeding this value will cause the index to drop > > > >> dramatically. > > > >> In the explanation, we are single copies, and multiple copies will > > cause > > > >> serious stability problems in the large solr cluster environment. > > > >> > > > >> At first I suspect that it was due to too many thread submissions, > and > > > >> there are still problems with this method, so I'm inclined to > > > >> searcherExecutor thread pool thread. This is just my guess, I want > to > > > know > > > >> the real reason. Can someone know if I can help? > > > >> > > > >> Also, I noticed that the searcherExecutor thread and solr > collection's > > > >> shards basically correspond to each other. How can I reduce the > number > > > of > > > >> threads or even close it? Although there are many collections in our > > > >> environment, there are few queries and it is not necessary to keep > the > > > >> threads open to provide queries. This is too wasteful. > > > >> > > > >> thank you . > > > >> > > > > > > > > > -- > == > 联创科技 > 知行如一 > == >
Re: Continuing Saga of Authorization on 6.6.0
I wish unsubscribe worked
On Mar 13, 2018 9:47 AM, "Terry Steichen" wrote:
> I switched solr from standalone to cloud and created the two collections
> (emails1 and emails2).
>
> I was able to create a basic set of credentials via the curl-based
> API's. I could create users, and toggle the blockUnknown property
> status. However, the system refused to allow me to delete a user, or to
> set a permission.
>
> Here are the curl commands (with *terry:admin* as admin credentials) and
> results:
>
> *succeeded in setting blockUnknown property (verified by
> admin/authentication dump):*
>
> curl --user terry:admin http://localhost:8983/solr/admin/authentication
> -H 'Content-type:application/json' -d '{
> "set-property": {"blockUnknown" : true}}'
>
> *succeeded in adding a user (verified by admin/authentication dump):*
>
> curl --user terry:admin http://localhost:8983/solr/admin/authentication
> -H 'Content-type:application/json' -d '{
> > "set-user": {"lanny" : "hawaii"}}'
>
> *succeeded in changing lanny's password (verified by
> admin/authentication dump):*
>
> curl --user terry:admin http://localhost:8983/solr/admin/authentication
> -H 'Content-type:application/json' -d '{
> "set-user": {"lanny" : "hawaii_five_o"}}'
>
> *failed to delete a user:*
>
> curl --user terry:admin http://localhost:8983/solr/admin/authentication
> -H 'Content-type:application/json' -d '{
> "delete-user": {"lanny"}}'
> {
> "responseHeader":{
> "status":500,
> "QTime":1},
>
> "error":{ "msg":"Expected key,value separator ':': char=},position=26
> BEFORE='{ \"delete-user\": {\"lanny\"}' AFTER='}'",
> [terry here: plus a very long stack trace}
>
> *failed to set a permission: *
>
> curl --user terry:admin http://localhost:8983/solr/admin/authentication
> -H 'Content-type:application/json' -d '{"set-permission" :
> {"name":"collection-admin-edit", "role":"admin"}}'
> {
> "responseHeader":{
> "status":0,
> "QTime":2},
> "errorMessages":[{
> "set-permission":{
> "name":"collection-admin-edit",
> "role":"admin"},
> "errorMessages":["Unknown operation 'set-permission' "]}]}
>
>
> This really makes no sense at all (or, I'm really losing it - always a
> distinct possibility). It's almost as if half of the documented
> parameters must have been changed, though I can't find any references to
> any such changes.
>
> I confess I'm about to just give up and find some other route to go.
>
> Terry
>
>
> On 03/12/2018 11:15 PM, Shawn Heisey wrote:
> > On 3/12/2018 8:39 PM, Terry Steichen wrote:
> >> I'm increasingly of the view that Solr's authentication/authorization
> >> mechanism doesn't work correctly in a _standalone_ mode. It was present
> >> in the cloud mode for quite a few versions back, but as of 6.0.0 (or so)
> >> it was supposed to be available in standalone mode too. It seems to
> >> partly work (when using the built-in permissions), but does not seem to
> >> work with customized, core-specific permissions.
> >
> > I suspected based on your last message that the authorization feature
> > might only work correctly in SolrCloud. The entire authentication
> > feature was designed for SolrCloud. Version 6.5 brought the
> > security.json file to standalone mode. This was LONG after the
> > feature was introduced in 5.2 and had a LOT of bugs fixed in the three
> > 5.3.x releases.
> >
> > I just found the section in the documentation confirming what I
> > suspected.
> >
> > https://lucene.apache.org/solr/guide/7_2/authentication-
> and-authorization-plugins.html#authorization
> >
> >
> > There is a note here that says "The authorization plugin is only
> > supported in SolrCloud mode. Also, reloading the plugin isn’t yet
> > supported and requires a restart of the Solr installation (meaning,
> > the JVM should be restarted, not simply a core reload)." The 6.6
> > documentation contains the same note that you can see here in the
> > latest docs.
> >
> > I have no idea how hard it would be to extend the authorization plugin
> > to support standalone cores as well as collections. I imagine that if
> > it were easy, it would have been done already.
> >
> > Thanks,
> > Shawn
> >
> >
>
>
