Re: code v.s. schema for BEST_COMPRESSION mode
The schema configuration in your first link is the way to tell Solr to use a particular compression mode in the Lucene index (the second link). So yes, the schema change should be enough. On Sun, Jun 17, 2018 at 6:39 AM Zahra Aminolroaya wrote: > I want to reduce the size of indexed and stored documents in Solr. I found > two way in the first solution > http:// > https://lucene.apache.org/solr/guide/6_6/codec-factory.html#solr-schemacodecfactory > https://lucene.apache.org/solr/guide/6_6/codec-factory.html#solr-schemacodecfactory> > > it is only needed to change the compressionMode in the codecFactory section > of schema. In the other way, > http:// > https://github.com/apache/lucene-solr/blob/releases/lucene-solr/6.4.0/lucene/core/src/java/org/apache/lucene/codecs/compressing/CompressingStoredFieldsFormat.java > https://github.com/apache/lucene-solr/blob/releases/lucene-solr/6.4.0/lucene/core/src/java/org/apache/lucene/codecs/compressing/CompressingStoredFieldsFormat.java> > > it is needed to use the java code to compress stored field. > > What is the difference between these solutions? Is it enough to use the > schema editting way? > > > > -- > Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html >
Broken pipe when indexing in solr using ConcurrentUpdateSolrClient
Hello, Sometimes we have "broken pipe" error when indexing 40 000 000 of documents using ConcurrentUpdateSolrClient in java. I have find on google that's probably is because à time out. But in ConcurrentUpdateSolrClient we can't configure timeout. For example the last broken pie was for indexing 40 000 000 of documents and several broken pipe occurred and 16 000 files would'nt indexing. Thank you for helping. Yassin
Re: code v.s. schema for BEST_COMPRESSION mode
Thanks -- Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Streaming expressions and fetch()
Hi, I am trying to use streaming expressions with SOLR 7.3.1. I have successfully used innerJoin, leftOuterJoin and several other functions but failed to achieve expected results with the fetch() function. Example below is silmplfied, in reality the base search() function uses fuzzy matching and scoring. And works perfectly. But I need to enrich the search results with additional column from the same collection. search() call does a query on nested documents, and returns parentId (yes, i know there is _root_, tried it as well) + some calculated custom values, requiring some aggregation calls, like rollup(). This part works perfectly. But then I want to enrich the resultset with attributes from the top level document, where "parentId=id". And all my attempts to fetch additional data have failed, the fetch() call below always gives the same results as the search() call inside. fetch(users, search(users, q="*:*", fq="type:name", fl="parentId", sort="parentId asc"), fl="id,name", on="parentId=id") As I understand fetch() should retrieve only records narrowed by the "parentId" results. If I call leftOuterJoin(), then I loose the benefit of such nice narrowing call. Any clue what i am doing wrong with fetch()? Best regards, Darek
Re: Solr basic auth
Of course, but Dinesh explicitly set blockUnknown=true below, so in this case
ALL requests must have credentials. There is currently no feature that lets
Solr accept any request by other rules, all requests are forwarded to the
chosen authentication plugin.
--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com
> 15. jun. 2018 kl. 19:12 skrev Terry Steichen :
>
> "When authentication is enabled ALL requests must carry valid
> credentials." I believe this behavior depends on the value you set for
> the *blockUnknown* authentication parameter.
>
>
> On 06/15/2018 06:25 AM, Jan Høydahl wrote:
>> When authentication is enabled ALL requests must carry valid credentials.
>>
>> Are you asking for a feature where a request is authenticated based on IP
>> address of the client, not username/password?
>>
>> Jan
>>
>> Sendt fra min iPhone
>>
>>> 14. jun. 2018 kl. 22:24 skrev Dinesh Sundaram :
>>>
>>> Hi,
>>>
>>> I have configured basic auth for solrcloud. it works well when i access the
>>> solr url directly. i have integrated this solr with test.com domain. now if
>>> I access the solr url like test.com/solr it prompts the credentials but I
>>> dont want to ask this time since it is known domain. is there any way to
>>> achieve this. much appreciate your quick response.
>>>
>>> my security json below. i'm using the default security, want to allow my
>>> domain default without prompting any credentials.
>>>
>>> {"authentication":{
>>> "blockUnknown": true,
>>> "class":"solr.BasicAuthPlugin",
>>> "credentials":{"solr":"IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0=
>>> Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c="}
>>> },"authorization":{
>>> "class":"solr.RuleBasedAuthorizationPlugin",
>>> "permissions":[{"name":"security-edit",
>>> "role":"admin"}],
>>> "user-role":{"solr":"admin"}
>>> }}
>
Re: Remove schema.xml in favor of managed-schema
I think my query got misinterpreted. I only wanted to know if schema.xml offer anything that managed-schema does not. Best, SG On Sat, Jun 16, 2018 at 6:45 PM Erick Erickson wrote: > Currently, there are no restrictions on hand-editing config files, > mutable or not. > > The rub is that any of the APIs that modify configs operate on their > in-memory copy and write that out (both Cloud and stand-alone modes). > > So if I start Solr, the nodes have image of the configs at time T. > Now I hand-edit the file(s) and push then to ZooKeeper say at time T1 > Now I use the API to update them at T2 > At this point, my changes pushed at T1 are lost since the T2 changes > used the in-memory copies read at time T as a basis for mods. > > If I change this even slightly by: > Start Solr at T > hand-edit and push at T1 _and reload my collection_ > use the API at T2 > At this point my configs have all the changes T1 and T2 in them since > the reload re-read the configs. > > Ditto if I restart all my Solr instances after T1 but before T2. > > That said, how this will change in the future I have no idea. I > usually hand-edit them but that's a personal problem. > > IIRC, at one point, there was one restriction: A mutable schema could > _not_ be named schema.xml. But whether that's an accurate memory and > if so whether it's still current I'm not sure about. > > And all of _that_ said, hand-editing mutable configs does, indeed, > violate all sorts of contracts and support may change in the future, > it's "at your own risk and you better know what you're doing". The > same could be said for hand-editing the configs in the first place > though I suppose ;) > > Best, > Erick > > On Sat, Jun 16, 2018 at 1:34 PM, Doug Turnbull > wrote: > > I'm not sure changing something from mutable -> unmutable means it > suddenly > > becomes hand-editable. > > > > I don't know the details here, but I can imagine a case that unmutable > > implies some level of consistency, where the file is hashed, and later > > might be confirmed to still be the same 'unmutable' state. Hand editing > > would violate that contract. > > > > One might also imagine a future where 'managed-schema' isn't a config > file, > > and is just an API you use to configure a Solr. In this case 'mutable' > > doesn't imply anything about files, just the state of the Solr config. > > > > -Doug > > > > On Sat, Jun 16, 2018 at 12:24 AM S G wrote: > > > >> Hi, > >> > >> As per > >> > >> > https://lucene.apache.org/solr/guide/7_2/schema-factory-definition-in-solrconfig.html#SchemaFactoryDefinitioninSolrConfig-Classicschema.xml > >> , > >> the only difference between schema.xml and managed-schema is that one > >> accepts schema-changes through an API while the other one does not. > >> > >> However, there is a flag "mutable" which can be used with managed-schema > >> too to turn dynamic-changes ON or OFF > >> > >> If that is true, then it means schema.xml does not offer anything which > >> managed-schema does not. > >> > >> Is that a valid statement to make? > >> > >> Infact, I see that schema.xml is not shipped anymore with Solr ? > >> > >> Thanks > >> SG > >> > > -- > > CTO, OpenSource Connections > > Author, Relevant Search > > http://o19s.com/doug >
Re: Soft commit impact on replication
Hi Erick, Thanks for the response. First thing we not indexing on Slave. And we are not re-indexing/optimizing entire the core in Master node. The only warning which I see in the log is "Unable clean the unused index directory so starting full copy". That one i can understand and I don't have issue with that as its normal behaviour but most of the time. But most of the time it’s just triggers full-copy without any details in the log. And recently in one of the nodes i enabled soft-commit in master nodes and monitored the corresponding slave node, what i observed is it didn't even trigger the full-copy not even once for almost 3 consecutive days. So am wondering do we need to have soft commit enabled in master for replication to happen smooth if so what’s the dependency there Thanks -- Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
