Re: Solr Cloud configuration
David, One benefit of the way recommended in the reference guide is that it lets you use zookeeper upconfig/downconfig as deployment tools on a set of text files, which in turn allows you to manage your Solr configuration like any other bit of source code, e.g. with version control and, if your situation permits, things like branching and pull requests or other review mechanisms. In particular I have found the capacity to view diffs, have peers review, and the ease of deploying changes to test and staging environments before moving them into production is worth the effort all by itself. HTH, AC On Tue, Nov 20, 2018 at 2:22 PM David Hastings wrote: > Well considering that any access to the user interface by anyone can > completely destroy entire collections/cores, I would think the security of > the stop word file wouldnt be that important > Thanks Erick, it seems the only reason I have any desire to use SolrCloud > is the use of streaming expressions. I think thats the only benefit that > more hardware cant solve. > > On Tue, Nov 20, 2018 at 2:17 PM Erick Erickson > wrote: > > > David: > > > > Sure would. See https://issues.apache.org/jira/browse/SOLR-5287. > > Especially the bits about how allowing this leads to security > > vulnerabilities. You're not the first one who had this idea ;). > > > > Whether those security issues are still valid is another question I > > suppose. > > > > Best, > > Erick > > On Tue, Nov 20, 2018 at 11:01 AM David Hastings > > wrote: > > > > > > Thanks, researching that now, but this seems extremely annoying. > wouldnt > > > it just be easier if you could edit the config files raw from the admin > > > UI? > > > > > > On Tue, Nov 20, 2018 at 1:41 PM Pure Host - Wolfgang Freudenberger < > > > w.freudenber...@pure-host.de> wrote: > > > > > > > Hi David, > > > > > > > > > > > > You can upload configuration to the zookeeper - it is nearly the same > > as > > > > the standaloneconfig. > > > > > > > > You can also edit the schema.xml in this file. At least I do it like > > this. > > > > > > > > Mit freundlichem Gruß / kind regards > > > > > > > > Wolfgang Freudenberger > > > > Pure Host IT-Services > > > > Münsterstr. 14 > > > > 48341 Altenberge > > > > GERMANY > > > > Tel.: (+49) 25 71 - 99 20 170 > > > > Fax: (+49) 25 71 - 99 20 171 > > > > > > > > Umsatzsteuer ID DE259181123 > > > > > > > > Informieren Sie sich über unser gesamtes Leistungsspektrum unter > > > > www.pure-host.de > > > > Get our whole services at www.pure-host.de > > > > > > > > Am 20.11.2018 um 19:38 schrieb David Hastings: > > > > > I cant seem to find the documentation on how to actually edit the > > schema > > > > > file myself, everything seems to lead me to using an API to add > > fields > > > > and > > > > > stop words etc. this is more or less obnoxious, and the admin api > > for > > > > > adding fields/field types is not exactly functional. is there a > > guide or > > > > > something to let me know how to do it normally like in standalone > > solr? > > > > > > > > > > > > > > > >
Local Params handling (6.5->7.x)
Hi folks,
I am trying to track down what might have changed and where i could tweak a
configuration that worked under Solr 6.5.1 but which is not working as
expected under 7.3.1 or 7.4.0. We have default "qf" and "pf" defeined for
our select handler, along with "special" versions of those for use in
particular kinds of queries; our application (based on Project Blacklight)
make extensive use of local parameter expansion.
Default query parser is edismax, and we are sending in queries like:
"q={! qf=$special_qf pf=$special_pf}query terms'
what used to happen, and what we expect given the documentation at
https://lucene.apache.org/solr/guide/6_6/local-parameters-in-queries.html
(6.6) and
https://lucene.apache.org/solr/guide/7_3/local-parameters-in-queries.html
(7.3) is that this should set qf and pf to their 'special' variants, where
the values are pulled from solrconfig.xml/confogoverlay.json (as
appropriate for the version). We can achieve the indended effect via:
q=query terms&pf=${special_pf}&qf=${special_qf}
(and we have tested this and it seems to yield the expected behavior) but
this seems like it would be more involved than figuring out *why* the old
syntax isn't working and what we could change in our configset to make it
work again. Our select handler configuration doesn't look particularly
weird:
The actual values of qf/pf and special_qf/pf are really big, but I created
a 'smaller' configuration for the select handler:
"requestHandler": { "/select": {"name": "/select",
"class": "solr.SearchHandler","defaults": {
"defType": "edismax", "echoParams": "explicit",
"rows": 10, "q.alt": "*:*", "mm": "6<90%",
"facet.mincount": "1", "qf": "original_qf_notexpanded^1.5",
"pf": "original_pf_notexpanded^1.5", "sample_qf":
"sample_qf_expanded^2", "sample_pf": "sample_pf_expanded^2"
} },
What we are seeing when we turn debugging on and look at the parsed query
is something like:
"parsedquery":"+(DisjunctionMaxQuery(((original_qf_notexpanded:[[7b
21] TO [7b 21]])^1.5))
DisjunctionMaxQuery(((original_qf_notexpanded:[[71 66 3d 24 73 61 6d
70 6c 65 5f 71 66] TO [71 66 3d 24 73 61 6d 70 6c 65 5f 71 66]])^1.5))
DisjunctionMaxQuery(((original_qf_notexpanded:[[70 66 3d 24 73 61 6d
70 6c 65 5f 70 66 7d 71 75 65 72 79] TO [70 66 3d 24 73 61 6d 70 6c 65
5f 70 66 7d 71 75 65 72 79]])^1.5))
DisjunctionMaxQuery(((original_qf_notexpanded:[[74 65 72 6d 73] TO [74
65 72 6d 73]])^1.5)))~4 ()",
"parsedquery_toString":"+original_qf_notexpanded:[[7b 21] TO
[7b 21]])^1.5) ((original_qf_notexpanded:[[71 66 3d 24 73 61 6d 70 6c
65 5f 71 66] TO [71 66 3d 24 73 61 6d 70 6c 65 5f 71 66]])^1.5)
((original_qf_notexpanded:[[70 66 3d 24 73 61 6d 70 6c 65 5f 70 66 7d
71 75 65 72 79] TO [70 66 3d 24 73 61 6d 70 6c 65 5f 70 66 7d 71 75 65
72 79]])^1.5) ((original_qf_notexpanded:[[74 65 72 6d 73] TO [74 65 72
6d 73]])^1.5))~4) ()",
the expanded fields are the ones from the default `qf` and `pf`
settings, and so i it looks like the local param syntax is not even
recognized, that it's somehow being "short-circuited". Debug output
indicates edismax parser is still used in this case, and this is true
even if we change our query string to something like:
q={!lucene ...}
or
q={!type=lucene ...}
We've tried a number of variations, including
(a) setting the 'sow' parameter to true and false, both inside the
expression and as a standalone parameter
(b) changing the luceneMatchVersion in the solrconfig (originally it
was 6.0.0, have changed it to match Solr versions, both via
re-uploading teh configset or creating a copy and modifying it before
upload).
What else should I look at? Changing the searchcomponent stack?
thanks!
AC
Re: Local Params handling (6.5->7.x)
Thanks Joel, that's it!
I had tried all sorts of variants on the search terms but somehow missed
that issue (probably because my eyes glazed over before getting to the 7.2
upgrade notes). While we were negatively impacted by the change, given
the existence of a simple workaround (anybody coming across the issue in
the future: defType=lucene in your query parameters), I don't think I'd
advocate for a behavioral change. It might be worth adding a note to the
Local Params page in the reference guide, though, as I imagine edismax is a
fairly popular defType.
cheers,
AC
On Thu, Jul 12, 2018 at 12:37 PM Joel Bernstein wrote:
> This may be the issue:
>
> https://issues.apache.org/jira/browse/SOLR-11501
>
> If it turns out that this causing the problem, please create a jira. It's
> important to discuss how SOLR-11501 is affecting real deployments.
>
> Joel Bernstein
> http://joelsolr.blogspot.com/
>
> On Thu, Jul 12, 2018 at 12:30 PM, Adam Constabaris
> wrote:
>
> > Hi folks,
> >
> > I am trying to track down what might have changed and where i could
> tweak a
> > configuration that worked under Solr 6.5.1 but which is not working as
> > expected under 7.3.1 or 7.4.0. We have default "qf" and "pf" defeined for
> > our select handler, along with "special" versions of those for use in
> > particular kinds of queries; our application (based on Project
> Blacklight)
> > make extensive use of local parameter expansion.
> >
> > Default query parser is edismax, and we are sending in queries like:
> >
> > "q={! qf=$special_qf pf=$special_pf}query terms'
> >
> > what used to happen, and what we expect given the documentation at
> >
> https://lucene.apache.org/solr/guide/6_6/local-parameters-in-queries.html
> > (6.6) and
> >
> https://lucene.apache.org/solr/guide/7_3/local-parameters-in-queries.html
> > (7.3) is that this should set qf and pf to their 'special' variants,
> where
> > the values are pulled from solrconfig.xml/confogoverlay.json (as
> > appropriate for the version). We can achieve the indended effect via:
> >
> > q=query terms&pf=${special_pf}&qf=${special_qf}
> >
> > (and we have tested this and it seems to yield the expected behavior) but
> > this seems like it would be more involved than figuring out *why* the old
> > syntax isn't working and what we could change in our configset to make it
> > work again. Our select handler configuration doesn't look particularly
> > weird:
> >
> > The actual values of qf/pf and special_qf/pf are really big, but I
> created
> > a 'smaller' configuration for the select handler:
> >
> > "requestHandler": { "/select": {"name": "/select",
> > "class": "solr.SearchHandler","defaults": {
> > "defType": "edismax", "echoParams": "explicit",
> > "rows": 10, "q.alt": "*:*", "mm": "6<90%",
> > "facet.mincount": "1", "qf": "original_qf_notexpanded^1.5",
> >"pf": "original_pf_notexpanded^1.5", "sample_qf":
> > "sample_qf_expanded^2", "sample_pf": "sample_pf_expanded^2"
> > } },
> >
> >
> >
> > What we are seeing when we turn debugging on and look at the parsed query
> > is something like:
> >
> > "parsedquery":"+(DisjunctionMaxQuery(((original_qf_notexpanded:[[7b
> > 21] TO [7b 21]])^1.5))
> > DisjunctionMaxQuery(((original_qf_notexpanded:[[71 66 3d 24 73 61 6d
> > 70 6c 65 5f 71 66] TO [71 66 3d 24 73 61 6d 70 6c 65 5f 71 66]])^1.5))
> > DisjunctionMaxQuery(((original_qf_notexpanded:[[70 66 3d 24 73 61 6d
> > 70 6c 65 5f 70 66 7d 71 75 65 72 79] TO [70 66 3d 24 73 61 6d 70 6c 65
> > 5f 70 66 7d 71 75 65 72 79]])^1.5))
> > DisjunctionMaxQuery(((original_qf_notexpanded:[[74 65 72 6d 73] TO [74
> > 65 72 6d 73]])^1.5)))~4 ()",
> > "parsedquery_toString":"+original_qf_notexpanded:[[7b 21] TO
> > [7b 21]])^1.5) ((original_qf_notexpanded:[[71 66 3d 24 73 61 6d 70 6c
> > 65 5f 71 66] TO [71 66 3d 24 73 61 6d 70 6c 65 5f 71 66]])^1.5)
> > ((original_qf_notexpanded:[[70 66 3d 24 73 61 6d 70 6c 65 5f 70 66 7d
> > 71 75 65 72 79] TO [70 66 3d 24 73 61 6d 70 6c 65 5f 70 66 7d 71 75 65
> > 72 79]])^1.5) ((original_qf_notexpanded:[[74 65 72 6d 73] TO [74 65 72
> > 6d 73]])^1
