Re: BasicAuth help

2020-09-03 Thread Jason Gerlowski
Hi Ali, 1. Solr doesn't have any support for LDAP authentication ootb (at least, as far as I'm aware). The BasicAuth plugin requires users to be defined in the JSON configuration. 2. What failed when you ran the documented BasicAuth example? What error messages did you get etc.? If there's som

[ANNOUNCE] Apache Solr 8.6.3 released

2020-10-08 Thread Jason Gerlowski
The Lucene PMC is pleased to announce the release of Apache Solr 8.6.3. Solr is the popular, blazing fast, open source NoSQL search platform from the Apache Lucene project. Its major features include powerful full-text search, hit highlighting, faceted search, dynamic clustering, database integrat

Faceting: !terms vs mincount precedence

2020-11-17 Thread Jason Gerlowski
Hey all, I was using the {!terms} local parameter on some traditional field facets to make sure particular values were returned. e.g. facet=true&facet.field={!terms='fantasy,scifi,mystery'}genre_s&f.genre_s.facet.mincount=2 On single-shard collections in 8.6.3 this worked as I expected - "fanta

Multiple Facets on Same Field

2020-11-17 Thread Jason Gerlowski
Hi all, Is it possible to have multiple facets on the same field with different parameters (mincount, limit, prefix, etc.) on each? The ref-guide describes these per-facet parameters as being settable on a "per-field basis" with syntax of "f..facet." [1]. But I wasn't sure whether to take that a

Re: Multiple Facets on Same Field

2020-11-17 Thread Jason Gerlowski
7;s much > cleaner separation between different facets (and output is assigned to > arbitrary keys). > Michael > > On Tue, Nov 17, 2020 at 9:36 AM Jason Gerlowski wrote: > > > > Hi all, > > > > Is it possible to have multiple facets on the same field with >

Re: Faceting: !terms vs mincount precedence

2020-11-17 Thread Jason Gerlowski
confess I've used that faceting local-param in an > app once before too; it's useful. I know my response isn't a direct answer > to your question RE mincount... perhaps it can be made to work? > > ~ David Smiley > Apache Lucene/Solr Search Developer > http://www.lin

Re: Using fromIndex for single collection

2020-11-19 Thread Jason Gerlowski
Hi Irina, Yes, the "fromIndex" parameter can be used to perform a join from the host collection to a separate, single-shard collection in SolrCloud. If specified, this "fromIndex" collection must be present on whichever host is processing the request. (Often this involves over-replicating your "f

Re: SolrJ NestableJsonFacet ordering of query facet

2020-11-19 Thread Jason Gerlowski
Hi Shivram, I think the short answer is "no". At least, not without sub-classing some of the JSON-Facet classes in SolrJ. But it's hard for me to understand your particular concern without seeing a concrete example. If you provide an example (maybe in the form of a JUnit test snippet showing th

Re: disallowing delete through security.json

2020-11-24 Thread Jason Gerlowski
Hey Craig, I think this will be tricky to do with the current Rule-Based Authorization support. As you pointed out in your initial post - there are lots of ways to delete documents. The Rule-Based Auth code doesn't inspect request bodies (AFAIK), so it's going to have trouble differentiating bet

Re: security.json help

2020-11-25 Thread Jason Gerlowski
Hi Mark, It looks like you're using the "path" wildcard as it's intended, but some bug is causing the behavior you're seeing. It should be working as you expected, but evidently it's not. One potential workaround might be to leave out the "path" property entirely in your "custom-example" permiss

Re: nested facets of query and terms type in JSON format

2020-12-10 Thread Jason Gerlowski
Hey Arturas, Can't help you with the secrets of Michael's inspiration (though I'm also curious :-p). And I'm not sure if there's any equivalent of facet.threads for JSON Faceting. You're on your own there unfortunately. But you (or other readers) might find this "Query Facet" example handy - it

Re: JSON Facet query to retrieve count all collections in Solr 8.0.0

2019-04-16 Thread Jason Gerlowski
Hi Edwin, To clarify what you're running into: - on 7.6, this query works all the time - on 7.7 this query works all the time - on 8.0, this query works the first time you run it, but subsequent runs return a 401 error? Is that correct? It might be helpful for others if you could share your sec

Re: JSON Facet query to retrieve count all collections in Solr 8.0.0

2019-04-17 Thread Jason Gerlowski
hyHXXuJSqcZdNgdSTGUvrQZRpqrYFUQ2ffmlWQ4GUTk= > > E0w3/2FD+rlxulbPm2G7i9HZqT+2gMBzcyJCcGcMWwA="} > > }, > > "authorization":{ > > "class":"solr.RuleBasedAuthorizationPlugin", > >"user-role":{"user1":"admi

Re: Solr RuleBasedAuthorizationPlugin question

2019-05-06 Thread Jason Gerlowski
Hey Jeremy, One important thing to remember about the RuleBasedAuthorizationPlugin is that if it doesn't find any rules matching a particular API call, it will allow the request. I think that's what you're running into here. Let's trace through how RBAP will process your rules: 1. Solr receives

Re: Solr RuleBasedAuthorizationPlugin question

2019-05-07 Thread Jason Gerlowski
uot;authorization": { >"class": "solr.RuleBasedAuthorizationPlugin", >"permissions": [ > { >"name": "read", >"role": "readonly" > }, > { > "path&qu

Re: alias read access impossible for anyone other than admin?

2019-05-24 Thread Jason Gerlowski
Hi Sotiris, First, what version of Solr are you running? We've made some fixes recently (esp. SOLR-13355) to RBAP, and they might affect the behavior you're seeing or any fixes we can recommend. Second, the order of permissions in security.json has a huge effect on how . Solr always uses the fi

Re: alias read access impossible for anyone other than admin?

2019-05-28 Thread Jason Gerlowski
to fix or create a documentation for the same that works > :) ? > > aroop > > > > On May 24, 2019, at 9:55 AM, Jason Gerlowski wrote: > > > > Hi Sotiris, > > > > First, what version of Solr are you running? We've made some fixes > > recentl

Re: alias read access impossible for anyone other than admin?

2019-05-31 Thread Jason Gerlowski
Hi Sotiris, Is this your second time asking this question here, or is there a subtle difference I'm missing? You asked a very similar question a week or so ago, and I replied with a few suggestions for changing your security.json and with a few questions. In case you missed it for whatever reaso

Re: Adding Multiple JSON Documents

2019-06-03 Thread Jason Gerlowski
Hi John, I believe the documentation there is correct. That is: those are two different "update" APIs. /update takes a JSON array of potentially multiple docs, /update/json/docs takes either a JSON array of multiple docs, or a single document not wrapped in the JSON array syntax. Best, Jason

Re: Intermittent error 401 with JSON Facet query to retrieve count all collections

2019-06-03 Thread Jason Gerlowski
I'm also able to reproduce this bug on master. A few more notes about the bad behavior: - the behavior occurs regardless of the specific permissions configured in security.json. (i.e. whether the top permission is "all", or "security-edit", or there are no permissions at all.) - I tried looking

Re: Intermittent error 401 with JSON Facet query to retrieve count all collections

2019-06-03 Thread Jason Gerlowski
One last note: as far as I can tell, nothing about this issue is specific to JSON Faceting or the JSON request API. It can be triggered just as easily with "/select?q=*:*". The bug created for this is: SOLR-13510 On Mon, Jun 3, 2019 at 9:17 AM Jason Gerlowski wrote: > >

Re: Intermittent error 401 with JSON Facet query to retrieve count all collections

2019-06-03 Thread Jason Gerlowski
g. { "authentication": { "blockUnknown": true, "class": "solr.BasicAuthPlugin", "credentials": { "solradmin": "" }, "forwardCredentials": true }, ... } Jason

Re: alias read access impossible for anyone other than admin?

2019-06-03 Thread Jason Gerlowski
erC":["readCollCProduction"], > "userD":[ > "readCollCProduction", > "readCollC55b", > "readCollB", > "readCollA"]}, > > > > On Fri, May 31, 2019 at 9:07 PM Sotiris F

Re: alias read access impossible for anyone other than admin?

2019-06-04 Thread Jason Gerlowski
ath:/select, role:readColl} 3. Looking at that permission further, Solr makes sure the "method" and "params" properties match the request. Since the properties aren't present, they're treated as wildcards and implicitly match. 4. So we've found a matching perm

Re: Intermittent error 401 with JSON Facet query to retrieve count all collections

2019-06-04 Thread Jason Gerlowski
n. Oops. > > > > A JIRA issue was raised on that thread > > https://issues.apache.org/jira/browse/SOLR-13421 but it's not had any > > attention. > > > > > > On Mon, 3 Jun 2019 at 14:46, Jason Gerlowski > > wrote: > > > > > Hi Co

Re: Issue with Solr 7.7.2 - ClassCastException: org.apache.solr.common.util.ByteArrayUtf8CharSequence

2019-06-19 Thread Jason Gerlowski
Hi David, Thanks for the heads up. We'd hoped to put an end to these issues as a part of SOLR-13331, but missed some field types as you pointed out. We're aware of the issue and working on a fix for upcoming Solr versions. Anyone interested can watch our progress here: https://issues.apache.org/

Re: Release of Solr 8.1.2 bug fix

2019-07-03 Thread Jason Gerlowski
Hi Edwin, Solr releases can be a messy process. They're subject to a lot of unforeseen issues that can drag the process out: test failures springing up at the last minute, other committers asking to squeeze in last minute fixes, infrastructure problems cropping up unexpectedly, etc. So release-m

Re: Getting list of unique values in a field

2019-07-15 Thread Jason Gerlowski
The Solr ref-guide has examples which show how to do this too. Take a look at some of the faceting examples here: https://lucene.apache.org/solr/guide/8_1/json-facet-api.html#bucketing-facet-example Best, Jason On Fri, Jul 12, 2019 at 9:50 AM David Hastings wrote: > > i found this: > > https:/

Re: Problems using a suggester component in the /select handler in cloud mode

2019-07-22 Thread Jason Gerlowski
Hi Alexandros, The first step would be to package up your changes in a patch file, and upload that to the JIRA you linked to in your initial email. (SOLR-12060). More detailed instructions can be found here: https://cwiki.apache.org/confluence/display/solr/HowToContribute#HowToContribute-Generatin

Re: HowtoConfigureIntelliJ link is broken

2019-07-22 Thread Jason Gerlowski
Hi Richard, Thanks for the heads up. I think this was a known issue. We recently moved our wiki from a Moin wiki to a Confluence one, and this changed the urls. There was an issue with the redirects at first, but they appear to be working now. Glad you were able to find what you needed regardl

Re: Contact for Wiki / Support page maintainer

2019-07-29 Thread Jason Gerlowski
I was under the impression that non-committers could also edit the wiki pages if the requested the appropriate karma on the mailing list. Though maybe that changed with the move to cwiki, or maybe that's never been the case On Thu, Jul 25, 2019 at 4:10 PM Jan Høydahl wrote: > > All committer

Re: Problems with restricting access to users using Basic auth

2019-08-20 Thread Jason Gerlowski
Hi Salmaan, Solr's RuleBasedAuthorizationPlugin allows requests through if none of the specified permissions apply. I think that's what you're running into in your example above. If you want to lockdown a particular API (or set of APIs) then you need to explicitly add a permission that restricts

Re: Problems with restricting access to users using Basic auth

2019-08-21 Thread Jason Gerlowski
link which lists the > users and their *SHA256* coded passwords. How can I hide this and restrict > access to other collections? > > Thanks and regards > Salmaan > > > On Wed, Aug 21, 2019 at 5:07 AM Jason Gerlowski > wrote: > > > Hi Salmaan, > > > >

Re: Problems with restricting access to users using Basic auth

2019-08-21 Thread Jason Gerlowski
", "role": "admin"}, {"name": "all", "role": "admin"} Hope that helps. Let me know if that still has any problems for you. Jason On Wed, Aug 21, 2019 at 6:48 AM Salmaan Rashid Syed wrote: > > Hi Jason, > > Is there a

Re: Problems with restricting access to users using Basic auth

2019-08-28 Thread Jason Gerlowski
t?q=*:*] returned status [200] Request [/collection3/select?q=*:*] returned status [200] Testing permissions for user [solr-user] Request [/admin/collections?action=LIST] returned status [403] Request [/collection1/select?q=*:*] returned status [200] Request [/collection2/select?q=*

Re: Problems with restricting access to users using Basic auth

2019-09-03 Thread Jason Gerlowski
ring wher I am going wrong. > > Thanks, > Salmaan > > > > > On Thu, Aug 29, 2019 at 1:34 PM Salmaan Rashid Syed < > salmaan.ras...@mroads.com> wrote: > > > Thanks a lot Jason, > > > > I will try this out and let you know. > > > > Thanks ag

Re: Solaris Install Package

2019-10-17 Thread Jason Gerlowski
Hi Andrew, I believe that yes, Solr should work on Solaris. I've never done so personally, but very occasionally I hear of someone doing so. Additionally, Uwe runs a Jenkins server that runs tests on Solaris (among other OSs), and the results for Solaris look to be pretty standard for our test su

Re: Solr enabled kerberos and create collection failed

2019-10-22 Thread Jason Gerlowski
Hi, You posted this same question in a different thread and Jorn Franke replied to say that you likely need to run "kinit" before invoking "bin/solr". That seems like a likely possible explanation to me. But, since you've given us very little information on how you've setup Kerberos and what you'

Re: Solr enabled kerberos and create collection failed

2019-10-22 Thread Jason Gerlowski
I _think_ this is the third time you've submitted this exact question as a different email thread. Both of your other threads have responses on them, but maybe you're not seeing that for some reason. Maybe you won't be able to see this response either, but in case you can: I think you'll have bet

Re: Solr enabled kerberos and create collection failed

2019-10-22 Thread Jason Gerlowski
I _think_ this is the fourth time you've submitted this exact question as a different email thread. Most of your other threads have responses on them, but maybe you're not seeing that for some reason. Maybe you won't be able to see this response either, but in case you can: I think you'll have be

Re: Query on changing FieldType

2019-10-22 Thread Jason Gerlowski
Hi Shubbham, Emir gave you accurate advice - you cannot (safely) change field types without reindexing. You may avoid errors for a time, and searches may even return the results you expect. But the type-change is still a ticking time bomb...Solr might try to merge segments down the road or do so

Re: Anyway to encrypt admin user plain text password in Solr

2019-11-13 Thread Jason Gerlowski
Hi, To clarify, Solr credentials are stored and shown in a few different places. In some situations the password might live in your "solr.in.sh" file. It also might live in a separate basicAuth.conf file. If you're using SolrCloud, the password might appear in Solr's Admin UI (depending on your

Re: Anyway to encrypt admin user plain text password in Solr

2019-11-14 Thread Jason Gerlowski
Hi Vinodh, I don't know of any way to encrypt the credentials in "basicAuth.conf", and looking at the code that loads that file I don't see any logic to handle that sort of thing. So I'm near positive there's no way to avoid plaintext here. But, that said, I don't think this should really be tha

Re: CloudSolrClient - basic auth - multi shard collection

2019-11-20 Thread Jason Gerlowski
Hi Nicholas, I'm not really familiar with spring-data-solr, so I can't speak to that detail, but it sounds like you might be running into either https://issues.apache.org/jira/browse/SOLR-13510 or https://issues.apache.org/jira/browse/SOLR-13472. There are partial workarounds on those issues that

Re: Possible bug in cluster status - > solr 8.3

2019-11-21 Thread Jason Gerlowski
It seems like an issue to me. Can you open a JIRA with these details? On Fri, Nov 15, 2019 at 10:51 AM Jacek Kikiewicz wrote: > > I found interesting situation, I've created a collection with only one > replica. > Then I scaled solr-cloud cluster, and run 'addreplica' call to add 2 more. > So

Re: exact matches on a join

2019-11-21 Thread Jason Gerlowski
Are these fields "string" or "text" fields? Text fields receive analysis that splits them into a series of terms. That's why the query "Freeman" matches the document "A-1 Freeman". "A-1 Freeman" gets split up into multiple terms, and the "Freeman" query matches one of those terms. Text fields are

Re: Possible data corruption in JavaBinCodec in Solr 8.3 during distributed update?

2019-11-21 Thread Jason Gerlowski
Very curious what the config change that's related to reproducing this looks like. Maybe it's something that is worth adding test-randomization around? Just thinking aloud.

Re: Facing jwt authentication problem using solr 8.1.1

2019-12-17 Thread Jason Gerlowski
Hey Jan, Is this a case of something that'd be fixed by https://issues.apache.org/jira/browse/SOLR-13071 ? Just wondering Best, Jason On Thu, Dec 12, 2019 at 5:43 PM Jan Høydahl wrote: > > Try something like this > https://gist.github.com/b330e1bea7842bcdc1e5fa3940b4a4f7 >

Re: Move SOLR from cloudera HDFS to SOLR on Docker

2019-12-19 Thread Jason Gerlowski
Hi Wael, Getting configs and data out of Cloudera's HDP is about the same as moving data between any 2 Solr clusters. Moving configs is going to be the easy part. If you're currently using Solr in SolrCloud mode, then your configs all live in ZooKeeper. Recent versions of Solr have a utility fo

Re: Facing jwt authentication problem using solr 8.1.1

2019-12-20 Thread Jason Gerlowski
> initial JWT config, with some default Rule-based config. > > Jan > > > 17. des. 2019 kl. 16:42 skrev Jason Gerlowski : > > > > Hey Jan, > > > > Is this a case of something that'd be fixed by > > https://issues.apache.org/jira/browse/SOLR-13071 ?

Re: understanding solr metrics

2020-01-02 Thread Jason Gerlowski
Hi Akhil, I'm not an expert on these metrics, but the way I've been reading them: "meanRate" is a measure of how many requests come in per some unit of time. It has nothing to do with how long individual requests take. "mean_ms" is the average time taken by requests (in milliseconds). Hope tha

Re: SOLR 7.5 Performance WARN

2020-01-15 Thread Jason Gerlowski
Hi Akreeti, The "onDeckSearcher" count is the number of searchers that are currently being opened/warmed for a given core. New searchers are opened by (some types of) commits. So essentially, what this message means is that you're asking Solr to do commits so close together that commit N is happ

Re: SolrCloud upgrade concern

2020-01-22 Thread Jason Gerlowski
Hi Arnold, The stability and complexity issues Mark highlighted in his post aren't just imagined - there are real, sometimes serious, bugs in SolrCloud features. But at the same time there are many many stable deployments out there where SolrCloud is a real success story for users. Small example

Re: Getting Solr's statistic using SolrJ

2021-02-01 Thread Jason Gerlowski
Hi Steven, AFAIK, SolrJ doesn't have built in request objects for the metrics API. But you can still use the "GenericSolrRequest" class to hit any Solr API: e.g. SolrParams params = new ModifiableSolrParams(); params.set("action", "list"); GenericSolrRequest request = new GenericSolrRequest(Sol

Re: Ghost Documents or Shards out of Sync

2021-02-01 Thread Jason Gerlowski
Hi Ronen, The first thing I'd figure out in your situation is whether the results are actually different each time, or whether the ordering is what differs (which might push a particular result off the page you're looking at, giving the appearance that it didn't match). In the case of the former,

Re: Ghost Documents or Shards out of Sync

2021-02-01 Thread Jason Gerlowski
rent of course, but something to consider. Best, Jason On Mon, Feb 1, 2021 at 3:49 PM Jason Gerlowski wrote: > > Hi Ronen, > > The first thing I'd figure out in your situation is whether the > results are actually different each time, or whether the ordering is > what

Re: Change uniqueKey using SolrJ

2021-02-01 Thread Jason Gerlowski
Hi, SolrJ doesn't have any purpose-made request class to change the uniqueKey, afaict. However doing so is still possible (though less convenient) using the "GenericSolrRequest" class, which can be used to hit arbitrary Solr APIs. If you'd like to see better support for this in SolrJ, open a JIR

Re: Solr Cloud freezes during scheduled backup

2021-02-02 Thread Jason Gerlowski
Hi Pawel, This definitely sounds like garbage collection biting you. Backups themselves aren't usually memory intensive, but if indexing is going on at the same time you should expect elevated memory usage. Essentially this is because for each core being backed up, Solr needs to hold pieces of tw

Re: Congratulations to the new Apache Solr PMC Chair, Jan Høydahl!

2021-02-19 Thread Jason Gerlowski
Congrats! On Fri, Feb 19, 2021 at 10:06 AM Divye wrote: > > Congratulations Jan! > > Regards, > Divye > > On Fri, 19 Feb, 2021, 00:26 Anshum Gupta, wrote: > > > Hi everyone, > > > > I’d like to inform everyone that the newly formed Apache Solr PMC nominated > > and elected Jan Høydahl for the po

<    1   2