Spellcheck response format differs between a single core and SolrCloud

[Ryan Yacyshyn]

Hello,

I am using the spellcheck component for spelling suggestions and I've used
the same configurations in two separate projects, the only difference is
one project uses a single core and the other is a collection on SolrCloud
with three shards. The single core has about 56K docs and the one on
SolrCloud has 1M docs. Strangely, the format of the response is slightly
different between the two and I'm not sure why (particularly the collations
part). Was wondering if any can shed some light on this? Below is my
configuration and the results I'm getting.

This is in my "/select" searchHandler:


on
false
5
2
5
true
true
5
3

And my spellcheck component:



  
  
default
spelling
solr.DirectSolrSpellChecker
internal
0.5
2
1
5
4
0.01
  


Examples of each output can be found here:
https://gist.github.com/ryac/ceff8da00ec9f5b84106

Thanks,
Ryan

Re: Spellcheck response format differs between a single core and SolrCloud

[Ryan Yacyshyn]

That's solves the mystery. The single-core is running 4.10.1 and SolrCloud
on 5.3.1.

Thanks James.



On Mon, 11 Jan 2016 at 22:24 Dyer, James 
wrote:

> Ryan,
>
> The json response format changed for Solr 5.0.  See
> https://issues.apache.org/jira/browse/SOLR-3029 .  Is the single-core
> solr running a 4.x version with the cloud solr running 5.x ?  If they are
> both on the same major version, then we have a bug.
>
> James Dyer
> Ingram Content Group
>
>
> -Original Message-
> From: Ryan Yacyshyn [mailto:ryan.yacys...@gmail.com]
> Sent: Monday, January 11, 2016 12:32 AM
> To: solr-user@lucene.apache.org
> Subject: Spellcheck response format differs between a single core and
> SolrCloud
>
> Hello,
>
> I am using the spellcheck component for spelling suggestions and I've used
> the same configurations in two separate projects, the only difference is
> one project uses a single core and the other is a collection on SolrCloud
> with three shards. The single core has about 56K docs and the one on
> SolrCloud has 1M docs. Strangely, the format of the response is slightly
> different between the two and I'm not sure why (particularly the collations
> part). Was wondering if any can shed some light on this? Below is my
> configuration and the results I'm getting.
>
> This is in my "/select" searchHandler:
>
> 
> on
> false
> 5
> 2
> 5
> true
> true
> 5
> 3
>
> And my spellcheck component:
>
> 
> 
>   
>   
> default
> spelling
> solr.DirectSolrSpellChecker
> internal
> 0.5
> 2
> 1
> 5
> 4
> 0.01
>   
> 
>
> Examples of each output can be found here:
> https://gist.github.com/ryac/ceff8da00ec9f5b84106
>
> Thanks,
> Ryan
>

Parallel SQL Interface returns "java.lang.NullPointerException" after reloading collection

[Ryan Yacyshyn]

Hi all,

I'm exploring with parallel SQL queries and found something strange after
reloading the collection: the same query will return a
java.lang.NullPointerException error. Here are my steps on a fresh install
of Solr 6.0.0.

*Start Solr in cloud mode with example*
bin/solr -e cloud -noprompt

*Index some data*
bin/post -c gettingstarted example/exampledocs/*.xml

*Send query, which works*
curl --data-urlencode 'stmt=select id,name from gettingstarted where
inStock = true limit 2' http://localhost:8983/solr/gettingstarted/sql

*Reload the collection*
curl '
http://localhost:8983/solr/admin/collections?action=RELOAD&name=gettingstarted
'

After reloading, running the exact query above will return the null pointer
exception error. Any idea why?

If I stop all Solr severs and restart, then it's fine.

*java -version*
java version "1.8.0_25"
Java(TM) SE Runtime Environment (build 1.8.0_25-b17)
Java HotSpot(TM) 64-Bit Server VM (build 25.25-b02, mixed mode)

Thanks,
Ryan

Re: Parallel SQL Interface returns "java.lang.NullPointerException" after reloading collection

[Ryan Yacyshyn]

Yes stack trace can be found here:

http://pastie.org/10821638



On Mon, 2 May 2016 at 01:05 Joel Bernstein  wrote:

> Can you post your stack trace? I suspect this has to do with how the
> Streaming API is interacting with SolrCloud. We can probably also create a
> jira ticket for this.
>
> Joel Bernstein
> http://joelsolr.blogspot.com/
>
> On Sun, May 1, 2016 at 4:02 AM, Ryan Yacyshyn 
> wrote:
>
> > Hi all,
> >
> > I'm exploring with parallel SQL queries and found something strange after
> > reloading the collection: the same query will return a
> > java.lang.NullPointerException error. Here are my steps on a fresh
> install
> > of Solr 6.0.0.
> >
> > *Start Solr in cloud mode with example*
> > bin/solr -e cloud -noprompt
> >
> > *Index some data*
> > bin/post -c gettingstarted example/exampledocs/*.xml
> >
> > *Send query, which works*
> > curl --data-urlencode 'stmt=select id,name from gettingstarted where
> > inStock = true limit 2' http://localhost:8983/solr/gettingstarted/sql
> >
> > *Reload the collection*
> > curl '
> >
> >
> http://localhost:8983/solr/admin/collections?action=RELOAD&name=gettingstarted
> > '
> >
> > After reloading, running the exact query above will return the null
> pointer
> > exception error. Any idea why?
> >
> > If I stop all Solr severs and restart, then it's fine.
> >
> > *java -version*
> > java version "1.8.0_25"
> > Java(TM) SE Runtime Environment (build 1.8.0_25-b17)
> > Java HotSpot(TM) 64-Bit Server VM (build 25.25-b02, mixed mode)
> >
> > Thanks,
> > Ryan
> >
>

Re: Parallel SQL Interface returns "java.lang.NullPointerException" after reloading collection

[Ryan Yacyshyn]

Thanks Joel! :)

On Tue, 3 May 2016, 23:37 Joel Bernstein,  wrote:

> Ryan, there is a patch (for the master branch) up on SOLR-9059 that
> resolves the issue. This will be in 6.1 and 6.0.1 if there is one. Thanks
> for the bug report!
>
> Joel Bernstein
> http://joelsolr.blogspot.com/
>
> On Tue, May 3, 2016 at 10:41 AM, Joel Bernstein 
> wrote:
>
> > I opened SOLR-9059.
> >
> > Joel Bernstein
> > http://joelsolr.blogspot.com/
> >
> > On Tue, May 3, 2016 at 10:31 AM, Joel Bernstein 
> > wrote:
> >
> >> What I believe is happening is that the core is closing on the reload,
> >> which is triggering the closeHook and shutting down all the connections
> in
> >> SolrClientCache.
> >>
> >> When the core reopens the connections are all still closed because the
> >> SolrClientCache is instantiated statically with the creation of the
> >> StreamHandler.
> >>
> >> So I think the correct fix is to create the SolrClientCache in inform(),
> >> that way it will get recreated with each reload. As long as the
> closeHook
> >> has closed the existing SolrClientCache this shouldn't cause any
> connection
> >> leaks with reloads.
> >>
> >>
> >>
> >>
> >> Joel Bernstein
> >> http://joelsolr.blogspot.com/
> >>
> >> On Tue, May 3, 2016 at 10:01 AM, Joel Bernstein 
> >> wrote:
> >>
> >>> I'll look into this today.
> >>>
> >>> Joel Bernstein
> >>> http://joelsolr.blogspot.com/
> >>>
> >>> On Tue, May 3, 2016 at 9:22 AM, Kevin Risden <
> risd...@avalonconsult.com>
> >>> wrote:
> >>>
> >>>> What I think is happening is that since the CloudSolrClient is from
> the
> >>>> SolrCache and the collection was reloaded. zkStateReader is actually
> >>>> null
> >>>> since there was no cloudSolrClient.connect() call after the reload. I
> >>>> think
> >>>> that would cause the NPE on anything that uses the zkStateReader like
> >>>> getClusterState().
> >>>>
> >>>> ZkStateReader zkStateReader = cloudSolrClient.getZkStateReader();
> >>>> ClusterState clusterState = zkStateReader.getClusterState();
> >>>>
> >>>>
> >>>> Kevin Risden
> >>>> Apache Lucene/Solr Committer
> >>>> Hadoop and Search Tech Lead | Avalon Consulting, LLC
> >>>> <http://www.avalonconsult.com/>
> >>>> M: 732 213 8417
> >>>> LinkedIn <http://www.linkedin.com/company/avalon-consulting-llc> |
> >>>> Google+
> >>>> <http://www.google.com/+AvalonConsultingLLC> | Twitter
> >>>> <https://twitter.com/avalonconsult>
> >>>>
> >>>>
> >>>>
> -
> >>>> This message (including any attachments) contains confidential
> >>>> information
> >>>> intended for a specific individual and purpose, and is protected by
> >>>> law. If
> >>>> you are not the intended recipient, you should delete this message.
> Any
> >>>> disclosure, copying, or distribution of this message, or the taking of
> >>>> any
> >>>> action based on it, is strictly prohibited.
> >>>>
> >>>> On Mon, May 2, 2016 at 9:58 PM, Joel Bernstein 
> >>>> wrote:
> >>>>
> >>>> > Looks like the loop below is throwing a Null pointer. I suspect the
> >>>> > collection has not yet come back online. In theory this should be
> self
> >>>> > healing and when the collection comes back online it should start
> >>>> working
> >>>> > again. If not then that would be a bug.
> >>>> >
> >>>> > for(String col : clusterState.getCollections()) {
> >>>> >
> >>>> >
> >>>> > Joel Bernstein
> >>>> > http://joelsolr.blogspot.com/
> >>>> >
> >>>> > On Mon, May 2, 2016 at 10:06 PM, Ryan Yacyshyn <
> >>>> ryan.yacys...@gmail.com>
> >>>> > wrote:
> >>>> >
> >>>> > > Yes stack trace can be found here:
> >>>> > >
> >>>> > > http://pastie.org/10821638
> &

Spellcheck on first character

[Ryan Yacyshyn]

Hi all,

Is it possible to provide spelling suggestions if it's just the first
character that's wrong (or has an additional character added)?

We have users querying for "eappointment" when they should just be
searching for "appointment". I'd like to show "appointment" as a spelling
suggestion for "eappointment".

Is this possible?

I'm using 4.10.1 and below are my configs:

**
on
false
1
1
1
true
false
5
1

**

  
  
default
spell
solr.DirectSolrSpellChecker

internal

0.5

2

1

5

4

0.01
  


Thanks,
Ryan

Re: Spellcheck on first character

[Ryan Yacyshyn]

hmm.. I must be missing something then. I do get spelling suggestions if I
make a mistake anywhere *after* the first character (eg: apointment or
apppointment or apppointmenz). But if there's only an additional character
in front, I get nothing back. Have you tried this?

I did try an online Levenshtein calculator and results were as expected - 1
edit distance away when an additional char is in front.




On Fri, 27 Nov 2015 at 19:50 Alessandro Benedetti 
wrote:

> Hi Ryan,
> "eappointment" has a Leveinstein distance of 1 edit from "appointment" ( 1
> insertion) .
> There's should be any problem to provide that suggestion, as the solr
> spellcheck supports up to 2 edits as a maximum distance.
>
> Cheers
>
> On 27 November 2015 at 03:41, Ryan Yacyshyn 
> wrote:
>
> > Hi all,
> >
> > Is it possible to provide spelling suggestions if it's just the first
> > character that's wrong (or has an additional character added)?
> >
> > We have users querying for "eappointment" when they should just be
> > searching for "appointment". I'd like to show "appointment" as a spelling
> > suggestion for "eappointment".
> >
> > Is this possible?
> >
> > I'm using 4.10.1 and below are my configs:
> >
> > **
> > on
> > false
> > 1
> > 1
> > 1
> > true
> > false
> > 5
> > 1
> >
> > **
> > 
> >   
> >   
> > default
> > spell
> > solr.DirectSolrSpellChecker
> > 
> > internal
> > 
> > 0.5
> > 
> > 2
> > 
> > 1
> > 
> > 5
> > 
> > 4
> > 
> > 0.01
> >   
> > 
> >
> > Thanks,
> > Ryan
> >
>
>
>
> --
> --
>
> Benedetti Alessandro
> Visiting card : http://about.me/alessandro_benedetti
>
> "Tyger, tyger burning bright
> In the forests of the night,
> What immortal hand or eye
> Could frame thy fearful symmetry?"
>
> William Blake - Songs of Experience -1794 England
>

Re: Spellcheck on first character

[Ryan Yacyshyn]

Thanks Markus, missed that. I'll try it out.


On Sat, 28 Nov 2015 at 00:47 Markus Jelsma 
wrote:

> Hi - this is default behaviour, see
> https://lucene.apache.org/core/4_1_0/suggest/org/apache/lucene/search/spell/DirectSpellChecker.html#setMinPrefix%28int%29
> lucky for you it is configurable via Solr:
> http://lucene.apache.org/solr/5_3_0/solr-core/org/apache/solr/spelling/DirectSolrSpellChecker.html
>
>
>
> -Original message-
> > From:Ryan Yacyshyn 
> > Sent: Friday 27th November 2015 15:44
> > To: solr-user@lucene.apache.org
> > Subject: Re: Spellcheck on first character
> >
> > hmm.. I must be missing something then. I do get spelling suggestions if
> I
> > make a mistake anywhere *after* the first character (eg: apointment or
> > apppointment or apppointmenz). But if there's only an additional
> character
> > in front, I get nothing back. Have you tried this?
> >
> > I did try an online Levenshtein calculator and results were as expected
> - 1
> > edit distance away when an additional char is in front.
> >
> >
> >
> >
> > On Fri, 27 Nov 2015 at 19:50 Alessandro Benedetti  >
> > wrote:
> >
> > > Hi Ryan,
> > > "eappointment" has a Leveinstein distance of 1 edit from "appointment"
> ( 1
> > > insertion) .
> > > There's should be any problem to provide that suggestion, as the solr
> > > spellcheck supports up to 2 edits as a maximum distance.
> > >
> > > Cheers
> > >
> > > On 27 November 2015 at 03:41, Ryan Yacyshyn 
> > > wrote:
> > >
> > > > Hi all,
> > > >
> > > > Is it possible to provide spelling suggestions if it's just the first
> > > > character that's wrong (or has an additional character added)?
> > > >
> > > > We have users querying for "eappointment" when they should just be
> > > > searching for "appointment". I'd like to show "appointment" as a
> spelling
> > > > suggestion for "eappointment".
> > > >
> > > > Is this possible?
> > > >
> > > > I'm using 4.10.1 and below are my configs:
> > > >
> > > > **
> > > > on
> > > > false
> > > > 1
> > > > 1
> > > > 1
> > > > true
> > > > false
> > > > 5
> > > > 1
> > > >
> > > > **
> > > > 
> > > >   
> > > >   
> > > > default
> > > > spell
> > > > solr.DirectSolrSpellChecker
> > > > 
> > > > internal
> > > > 
> > > > 0.5
> > > > 
> > > > 2
> > > > 
> > > > 1
> > > > 
> > > > 5
> > > > 
> > > > 4
> > > > 
> > > > 0.01
> > > >   
> > > > 
> > > >
> > > > Thanks,
> > > > Ryan
> > > >
> > >
> > >
> > >
> > > --
> > > --
> > >
> > > Benedetti Alessandro
> > > Visiting card : http://about.me/alessandro_benedetti
> > >
> > > "Tyger, tyger burning bright
> > > In the forests of the night,
> > > What immortal hand or eye
> > > Could frame thy fearful symmetry?"
> > >
> > > William Blake - Songs of Experience -1794 England
> > >
> >
>

A Synonym Searching for Phrase?

[Ryan Yacyshyn]

Hi All,

I'm running into an issue where I have some tokens that really mean the
same thing as two. For example, there are a couple ways users might want to
search for certain type of visa called the "s pass", but they might query
for spass or s-pass.

I thought I could add a line in my synonym file to solve this, such as:

s-pass, spass => s pass

This doesn't seem to work. I found an Auto Phrase TokenFilter (
https://github.com/LucidWorks/auto-phrase-tokenfilter) that looks like it
might help, but it sounds like it needs to use a specific query parser as
well (we're using edismax).

Has anyone came across this specific problem before? Would really
appreciate your suggestions / help.

We're using Solr 4.8.x (and lucidWorks 2.9).

Thanks!
Ryan

Re: A Synonym Searching for Phrase?

[Ryan Yacyshyn]

Thanks all for your suggestions.

What we've done in the end - and I'm not so sure why it works - is adding
"s-pass, spass, s pass" to the synonyms.txt file rather than s-pass, spass
=> s pass.







On Fri, 15 May 2015 at 16:02 Rajani Maski  wrote:

> Hi Ryan,
>
> I am not really sure whether this[1] solution mentioned in the link below
> can work for your case considering its cons. However, I recommend having a
> quick look at it.
>
> @Chris, Would eagerly wait for your contribution.
>
>
> [1] https://support.lucidworks.com/hc/en-us/articles/205359448
>
>
>
> On Thu, May 14, 2015 at 11:30 PM, Chris Morley 
> wrote:
>
> > I have implemented that but it's not open sourced yet.  It will be soon.
> >
> >  -Chris.
> >
> >
> >
> >
> > 
> >  From: "Ryan Yacyshyn" 
> > Sent: Thursday, May 14, 2015 12:07 PM
> > To: solr-user@lucene.apache.org
> > Subject: A Synonym Searching for Phrase?
> > Hi All,
> >
> > I'm running into an issue where I have some tokens that really mean the
> > same thing as two. For example, there are a couple ways users might want
> > to
> > search for certain type of visa called the "s pass", but they might query
> > for spass or s-pass.
> >
> > I thought I could add a line in my synonym file to solve this, such as:
> >
> > s-pass, spass => s pass
> >
> > This doesn't seem to work. I found an Auto Phrase TokenFilter (
> > https://github.com/LucidWorks/auto-phrase-tokenfilter) that looks like
> it
> > might help, but it sounds like it needs to use a specific query parser as
> > well (we're using edismax).
> >
> > Has anyone came across this specific problem before? Would really
> > appreciate your suggestions / help.
> >
> > We're using Solr 4.8.x (and lucidWorks 2.9).
> >
> > Thanks!
> > Ryan
> >
> >
> >
>

Spell checking the synonym list?

[Ryan Yacyshyn]

Hi all,

I'm wondering if it's possible to have spell checking performed on terms in
the synonym list?

For example, let's say I have documents with the word "lawyer" in them and
I add "lawyer, attorney" in the synonyms.txt file. Then a query is made for
the word "atorney". Is there any way to provide spell checking on this?

Thanks,
Ryan

Re: Spell checking the synonym list?

[Ryan Yacyshyn]

Thanks both!

James, I like that approach. I'll give it a try. I forgot to mention I was
only using query-time synonyms but shouldn't be a problem in my case to add
synonyms during index-time.

Ryan



On Thu, 9 Jul 2015 at 22:07 Dyer, James 
wrote:

> Ryan,
>
> If you use index-time synonyms on the spellcheck field, this will give you
> what you want.
>
> For instance, if the document has "lawyer" and you index both terms
> "lawyer","attorney", then the spellchecker will see that "atorney" is 1
> edit away from an indexed term and will suggest "attorney".
>
> You'll need to have the same synonyms set up against the query field, but
> you have the option of making these query-time synonyms if you prefer.
>
> James Dyer
> Ingram Content Group
>
> -Original Message-
> From: Ryan Yacyshyn [mailto:ryan.yacys...@gmail.com]
> Sent: Thursday, July 09, 2015 2:28 AM
> To: solr-user@lucene.apache.org
> Subject: Spell checking the synonym list?
>
> Hi all,
>
> I'm wondering if it's possible to have spell checking performed on terms in
> the synonym list?
>
> For example, let's say I have documents with the word "lawyer" in them and
> I add "lawyer, attorney" in the synonyms.txt file. Then a query is made for
> the word "atorney". Is there any way to provide spell checking on this?
>
> Thanks,
> Ryan
>

The Query Elevation Component

[Ryan Yacyshyn]

Hi everyone,

I'm reading the docs on the query elevation component and some questions
came up:

Can I specify a field that the elevate component will look at, such as only
looking at the title field? My search handler (using eDisMax) is searching
across multiple fields, but if I only want the elevate component to look at
one field, is this possible? I'd like the search request to search multiple
fields, but only elevate if the query is found in one of the fields.

Also, is there a recommended way to analyze the query? For example, when
using the queryFieldType parameter, I'd think I'd only want to use the
KeywordTokenizer and maybe lowercasing.

Thanks,
Ryan

Re: The Query Elevation Component

[Ryan Yacyshyn]

Hi Alessandro,

My mistake, I thought for a second there that the elevation component
needed to actually searches through documents, which isn't the case.

Thanks,
Ryan





On Wed, 27 Jul 2016 at 19:15 Alessandro Benedetti 
wrote:

> Hi Ryan,
> can you explain this ?
> " I'd like the search request to search multiple
> fields, but only elevate if the query is found in one of the fields."
>
> You mean, that you want to apply the elevation component only if the user
> selected a particular field in the query ?
> If i remember well, you have the possibility of associate a list of
> documents to each query you prefer in the elevation file.
>
> But maybe I misunderstood your question, are you actually thinking to boost
> the results only if they have a certain match in a particular field ?
> Because maybe you are looking for the classic edismax with different field
> boosting instead than the query elevation component.
> Let us know and we can help you better!
>
> Cheers
>
> On Wed, Jul 27, 2016 at 4:49 AM, Ryan Yacyshyn 
> wrote:
>
> > Hi everyone,
> >
> > I'm reading the docs on the query elevation component and some questions
> > came up:
> >
> > Can I specify a field that the elevate component will look at, such as
> only
> > looking at the title field? My search handler (using eDisMax) is
> searching
> > across multiple fields, but if I only want the elevate component to look
> at
> > one field, is this possible? I'd like the search request to search
> multiple
> > fields, but only elevate if the query is found in one of the fields.
> >
> > Also, is there a recommended way to analyze the query? For example, when
> > using the queryFieldType parameter, I'd think I'd only want to use the
> > KeywordTokenizer and maybe lowercasing.
> >
> > Thanks,
> > Ryan
> >
>
>
>
> --
> --
>
> Benedetti Alessandro
> Visiting card : http://about.me/alessandro_benedetti
>
> "Tyger, tyger burning bright
> In the forests of the night,
> What immortal hand or eye
> Could frame thy fearful symmetry?"
>
> William Blake - Songs of Experience -1794 England
>

Spellcheck: using multiple dictionaries (DirectSolrSpellChecker and FileBasedSpellChecker)

[Ryan Yacyshyn]

Hi everyone,

I'm looking at using two different implementations of spell checking
together: DirectSolrSpellChecker and FileBasedSpellChecker but I get the
following error:

msg: "All checkers need to use the same Analyzer.",
trace: "java.lang.IllegalArgumentException: All checkers need to use the
same Analyzer. at
org.apache.solr.spelling.ConjunctionSolrSpellChecker.addChecker(ConjunctionSolrSpellChecker.java:79)
at
org.apache.solr.handler.component.SpellCheckComponent.getSpellChecker(SpellCheckComponent.java:603)
at
org.apache.solr.handler.component.SpellCheckComponent.prepare(SpellCheckComponent.java:126)
at ...

The source mentions that the "initial use-case was to use
WordBreakSolrSpellChecker in conjunction with the DirectSolrSpellChecker".

If I make a query with only of the dictionaries (file or direct), they both
work fine, combining them into one query throws the error. I'm not sure if
I'm doing something wrong or if I just can't use these two together (yet).

I'm using 6.2.0. Thanks for any help!

Ryan

Re: Spellcheck: using multiple dictionaries (DirectSolrSpellChecker and FileBasedSpellChecker)

[Ryan Yacyshyn]

Ok, thanks Andrey.



On Tue, 27 Sep 2016 at 00:13 Kydryavtsev Andrey  wrote:

> Hello, Ryan
>
>
> As it obvious from exception message - you are forced to use same instance
> of Analyzer to all of spell checkers which should be conjuncted.
>
> How this instance is initialized inside SpellChecker instance could be
> found here -
> https://github.com/apache/lucene-solr/blob/master/solr/core/src/java/org/apache/solr/spelling/SolrSpellChecker.java#L65
>
> So one of possibilities to make it work - use same field for both spell
> checkers. solrconfig.xml could looks like this:
>
>   
> default
> solr.DirectSolrSpellChecker
> field_for_spell_check
> …
> 
>
>
> 
> wordbreak
> solr.WordBreakSolrSpellChecker
>  field_for_spell_check 
>   ….
> 
>
> 23.09.2016, 12:13, "Ryan Yacyshyn" :
> > Hi everyone,
> >
> > I'm looking at using two different implementations of spell checking
> > together: DirectSolrSpellChecker and FileBasedSpellChecker but I get the
> > following error:
> >
> > msg: "All checkers need to use the same Analyzer.",
> > trace: "java.lang.IllegalArgumentException: All checkers need to use the
> > same Analyzer. at
> >
> org.apache.solr.spelling.ConjunctionSolrSpellChecker.addChecker(ConjunctionSolrSpellChecker.java:79)
> > at
> >
> org.apache.solr.handler.component.SpellCheckComponent.getSpellChecker(SpellCheckComponent.java:603)
> > at
> >
> org.apache.solr.handler.component.SpellCheckComponent.prepare(SpellCheckComponent.java:126)
> > at ...
> >
> > The source mentions that the "initial use-case was to use
> > WordBreakSolrSpellChecker in conjunction with the
> DirectSolrSpellChecker".
> >
> > If I make a query with only of the dictionaries (file or direct), they
> both
> > work fine, combining them into one query throws the error. I'm not sure
> if
> > I'm doing something wrong or if I just can't use these two together
> (yet).
> >
> > I'm using 6.2.0. Thanks for any help!
> >
> > Ryan
>

When to use LTR

[Ryan Yacyshyn]

Hey all,

There's a lot of interest and articles explaining how learning to rank can
be implemented in search engines and that's awesome. I'm looking forward to
giving this a try.

However, I was wondering.. when would you know that implementing LTR
would *really
help* in any given case? Is it better for large indices, frequently updated
content, or more structured content, etc.? Or is it all trial and error?

Would be great to hear your thoughts on this.

Regards,
Ryan

Re: When to use LTR

[Ryan Yacyshyn]

Thanks for your help Alessandro!

Ryan


On Wed, 21 Jun 2017 at 19:25 alessandro.benedetti 
wrote:

> Hi Ryan,
> first thing to know is that Learning To Rank is about relevancy and
> specifically it is about to improve your relevancy function.
> Deciding if to use or not LTR has nothing to do with your index size or
> update frequency ( although LTR brings some performance consideration you
> will need to evaluate) .
>
> Functionally, the moment you realize you want LTR is when you start tuning
> your relevancy.
> Normally the first approach is the manual one, you identify a set of
> features, interesting for your use case and you tune a boosting function to
> improve your search experience.
>
> e.g.
> you decide to weight more the title field than the content and then
> boosting
> recent documents.
>
> What happens next is :
> "How much should I weight more the title ?"
> "How much should I boost recent documents ?"
>
> Normally you just check some golden queries and you try to manually
> optimise
> these boosting factors by hand.
>
> LTR answers to this requirements.
> To make it simple LTR will bring you a model that will tell you the best
> weighting factors given your domain ( and past experience) to get the most
> relevant results for all the queries ( this is the ideal, of course it is
> quite complicated and it depends of a lot of factors)
>
> Of course it doesn't work like magic and you will need to extensively
> design
> your features ( features engineering), build a valid training set (
> explicit
> or implicit), decide the model that best suites your needs ( linear model
> or
> Tree based ?) and a lot of corollary configurations.
>
> hope this helps!
>
>
>
>
>
> -
> ---
> Alessandro Benedetti
> Search Consultant, R&D Software Engineer, Director
> Sease Ltd. - www.sease.io
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/When-to-use-LTR-tp4342130p4342140.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>

Re: Graph Visualizing tool

[Ryan Yacyshyn]

Hello,

Have you tried Gephi? https://gephi.org/



On Sat, 22 Jul 2017 at 14:46 mganeshs  wrote:

> Tried this, but it's not working as expected.
>
> http://solr.pl/en/2016/04/25/graph-visualization-using-solr-6/
>
>
> Any of you used this or any other tool ?
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Graph-Visualizing-tool-tp4347240p4347241.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>

LambdaMART XML model to JSON

[Ryan Yacyshyn]

Hi everyone,

I'm trying out the LTR plugin and have a couple questions when it comes to
converting the LambdaMart XML to JSON. Below is a snippet of the model
generated from rankLib:


  

   10 
   0.28156844 
  
 11 
 7.11 

   7 
   2.2759523 
  
 0.8436763 
  
  
 1.4320849 
  

--

And a sample of training data:

1 qid:1 1:0.0 2:1.0 3:0.0 4:0.0 5:0.0 6:0.0 7:19.496738 8:0.0 9:0.0
10:0.08307255 11:7.11 #docId: oeqzg5-165248

It's probably obvious, but I just want to check if the  node in
the XML is referring to the ID in my training set and that it's possible
that the model doesn't use all features in the training data?

I'll be mapping these feature IDs with the names I gave them in the feature
store in Solr..

Is there a script or utility already made out there to convert the XML to
this Solr JSON format? The closest I found to something was this:
https://sourceforge.net/p/lemur/feature-requests/144/

Thanks for your help!

Ryan

Re: LambdaMART XML model to JSON

[Ryan Yacyshyn]

Thanks Doug, this is helpful.

I also started something last night to output to JSON for Solr, I'll post
it up as well.

Ryan




On Sun, 23 Jul 2017 at 23:48 Doug Turnbull <
dturnb...@opensourceconnections.com> wrote:

> Yes you're correct that the feature is the 1-based identifier from your
> training data.
>
> For a script. Not one to Solr exactly, but when developing the
> Elasticsearch plugin, I started to work on a JSON serialization format, and
> as part of that built a Python script for reading the Ranklib XML and
> outputting to my own JSON format. It could be helpful to you or anyone
> constructing a script:
>
>
> https://github.com/o19s/elasticsearch-learning-to-rank/blob/7426858c2afb168ac426cab6d857fddccb9c26fc/demo/ranklibToJson.py
>
> On Sun, Jul 23, 2017 at 7:18 AM Ryan Yacyshyn 
> wrote:
>
> > Hi everyone,
> >
> > I'm trying out the LTR plugin and have a couple questions when it comes
> to
> > converting the LambdaMart XML to JSON. Below is a snippet of the model
> > generated from rankLib:
> >
> > 
> >   
> > 
> >10 
> >0.28156844 
> >   
> >  11 
> >  7.11 
> > 
> >7 
> >2.2759523 
> >   
> >  0.8436763 
> >   
> >   
> >  1.4320849 
> >   
> > 
> > --
> >
> > And a sample of training data:
> >
> > 1 qid:1 1:0.0 2:1.0 3:0.0 4:0.0 5:0.0 6:0.0 7:19.496738 8:0.0 9:0.0
> > 10:0.08307255 11:7.11 #docId: oeqzg5-165248
> >
> > It's probably obvious, but I just want to check if the  node in
> > the XML is referring to the ID in my training set and that it's possible
> > that the model doesn't use all features in the training data?
> >
> > I'll be mapping these feature IDs with the names I gave them in the
> feature
> > store in Solr..
> >
> > Is there a script or utility already made out there to convert the XML to
> > this Solr JSON format? The closest I found to something was this:
> > https://sourceforge.net/p/lemur/feature-requests/144/
> >
> > Thanks for your help!
> >
> > Ryan
> >
>

Re: LambdaMART XML model to JSON

[Ryan Yacyshyn]

Here's something that'll create a JSON model that can be directly uploaded
into Solr:

https://github.com/ryac/lambdamart-xml-to-json

It'll map the feature IDs to the names found in the feature-store as well.

I had this error when uploading model:

Model type does not exist
org.apache.solr.ltr.model.MultipleAdditiveTreesModel

Once I changed the values from floats to strings it uploaded fine.

Ryan




On Mon, 24 Jul 2017 at 12:14 Ryan Yacyshyn  wrote:

> Thanks Doug, this is helpful.
>
> I also started something last night to output to JSON for Solr, I'll post
> it up as well.
>
> Ryan
>
>
>
>
> On Sun, 23 Jul 2017 at 23:48 Doug Turnbull <
> dturnb...@opensourceconnections.com> wrote:
>
>> Yes you're correct that the feature is the 1-based identifier from your
>> training data.
>>
>> For a script. Not one to Solr exactly, but when developing the
>> Elasticsearch plugin, I started to work on a JSON serialization format,
>> and
>> as part of that built a Python script for reading the Ranklib XML and
>> outputting to my own JSON format. It could be helpful to you or anyone
>> constructing a script:
>>
>>
>> https://github.com/o19s/elasticsearch-learning-to-rank/blob/7426858c2afb168ac426cab6d857fddccb9c26fc/demo/ranklibToJson.py
>>
>> On Sun, Jul 23, 2017 at 7:18 AM Ryan Yacyshyn 
>> wrote:
>>
>> > Hi everyone,
>> >
>> > I'm trying out the LTR plugin and have a couple questions when it comes
>> to
>> > converting the LambdaMart XML to JSON. Below is a snippet of the model
>> > generated from rankLib:
>> >
>> > 
>> >   
>> > 
>> >10 
>> >0.28156844 
>> >   
>> >  11 
>> >  7.11 
>> > 
>> >7 
>> >2.2759523 
>> >   
>> >  0.8436763 
>> >   
>> >   
>> >  1.4320849 
>> >   
>> > 
>> > --
>> >
>> > And a sample of training data:
>> >
>> > 1 qid:1 1:0.0 2:1.0 3:0.0 4:0.0 5:0.0 6:0.0 7:19.496738 8:0.0 9:0.0
>> > 10:0.08307255 11:7.11 #docId: oeqzg5-165248
>> >
>> > It's probably obvious, but I just want to check if the  node in
>> > the XML is referring to the ID in my training set and that it's possible
>> > that the model doesn't use all features in the training data?
>> >
>> > I'll be mapping these feature IDs with the names I gave them in the
>> feature
>> > store in Solr..
>> >
>> > Is there a script or utility already made out there to convert the XML
>> to
>> > this Solr JSON format? The closest I found to something was this:
>> > https://sourceforge.net/p/lemur/feature-requests/144/
>> >
>> > Thanks for your help!
>> >
>> > Ryan
>> >
>>
>

Re: LambdaMART XML model to JSON

[Ryan Yacyshyn]

Hi Alessandro,

Ok no prob. The script-based approach seems to work just fine for me right
now!

Ryan


On Mon, 24 Jul 2017 at 18:56 alessandro.benedetti 
wrote:

> hi Ryan,
> the issue you mentioned was mine :
> https://sourceforge.net/p/lemur/feature-requests/144/
>
> My bad It got lost in sea of "To Dos" .
> I still think it could be a good contribution to the library, but at the
> moment I think going with a custom script/app to do the transformation is
> the way to go.
>
>
>
>
>
> -
> ---
> Alessandro Benedetti
> Search Consultant, R&D Software Engineer, Director
> Sease Ltd. - www.sease.io
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/LambdaMART-XML-model-to-JSON-tp4347277p4347343.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>

SolrTextTagger on SolrCloud

[Ryan Yacyshyn]

Hi Everyone,

A quick question.. does the SolrTextTagger plugin (
https://github.com/OpenSextant/SolrTextTagger) work in SolrCould mode? It
works in standalone, but in SolrCloud, while I do get a response, it's
empty:

{
  "responseHeader": {
"status": 0,
"QTime": 36,
"params": {
  "tagsLimit": "10",
  "field": "taggers",
  "echoParams": "all",
  "fl": "id,name,label",
  "overlaps": "NO_SUB",
  "collection": "tagger",
  "wt": "json"
}
  },
  "tagsCount": 0,
  "tags": [],
  "response": {
"numFound": 0,
"start": 0,
"docs": []
  }
}

Configs and content in both indices are the same.

Ryan

Re: Solr learning to rank features question

[Ryan Yacyshyn]

haha.. all good! I'm trying to find my way there now too.

On Wed, Sep 13, 2017, 17:01 Diego Ceccarelli 
wrote:

> Hey man, how's life?
> Are you at the conference? I'm going a the reception now, see you there in
> case :)
>
> On 29 Aug 2017 12:10, "Brian Yee"  wrote:
>
> > Thank you Diego! This works with some small changes. My feature ended up
> > looking like this:
> >
> > {
> >   "name" : "FeatureA",
> >   "store" : "commonFeatureStore",
> >   "class" : "org.apache.solr.ltr.feature.SolrFeature",
> >   "params" : {
> > "q" : "{!func}if(gt(ms(CutoffDate,NOW),0),exists(query({!v=
> > PreZones:${zone}})),exists(query({!v=PostZones:${zone}})))"
> >   }
> > },
> >
> > On Tue, Aug 29, 2017 at 12:32 PM, Diego Ceccarelli <
> > diego.ceccare...@gmail.com> wrote:
> >
> > > Hi Brian,
> > >
> > > The plugin doesn't allow you to express multiple function queries in
> the
> > > same feature. Maybe in this case you can express both the tw queries in
> > one
> > > unique function query, using the if function.
> > >
> > > Something like:
> > >
> > > "fq":"if(gt(ms(NOW,mydatefield),0,query(PreCutOffZones:${zone}), query(
> > > PostCutOffZones:${zone}))"
> > >
> > >
> > > (Function query documentation here:
> > > https://cwiki.apache.org/confluence/display/solr/Function+Queries)
> > >
> > > Let me know if it works.
> > >
> > > Cheers,
> > > Diego
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > On Tue, Aug 29, 2017 at 5:47 PM, Brian Yee 
> wrote:
> > >
> > > > Hello,
> > > >
> > > > I have a somewhat complex solr filter query that I am hoping to turn
> > > into a
> > > > feature for learning to rank, but I am not sure if it's possible. I
> > would
> > > > like to OR two fq together for one of my features. I have a cutoff
> date
> > > and
> > > > I need to check one field if NOW is before the cutoff, and a
> different
> > > > field if NOW is after the cutoff. Is this possible?
> > > >
> > > > Something to the effect of this:
> > > > {
> > > >   "name" : "FeatureA",
> > > >   "store" : "commonFeatureStore",
> > > >   "class" : "org.apache.solr.ltr.feature.SolrFeature",
> > > >   "params" : {
> > > > "fq" : [
> > > >   "{!frange l=NOW}CutOffDate",
> > > >   "{!term f=PreCutOffZones}${zone}"
> > > > ]
> > > > OR
> > > > "fq" : [
> > > >   "{!frange u=NOW}CutOffDate",
> > > >   "{!term f=PostCutOffZones}${zone}"
> > > > ]
> > > >   }
> > > > },
> > > >
> > > > Thank you!
> > > > Brian Yee
> > > >
> > >
> >
>

Using the suggester component on distributed indexes

[Ryan Yacyshyn]

Hi all,

I know the suggester component will work on distributed indexes. It works
fine when I'm only using the suggester component in the components chain,
but I'd like to apply the suggester to the end of the default components
chain (query, facet, mlt, etc..). When I do, I get an error. For example,
if I add:


  suggest


at the end of the default "/select" handler, the error I see is:

http://localhost:8983/solr/suggest-test/select?q=obie

{
  "responseHeader": {
"zkConnected": true,
"status": 500,
"QTime": 63,
"params": {
  "q": "obie"
}
  },
  "error": {
"trace": "java.lang.NullPointerException\n\tat
org.apache.solr.handler.component.QueryComponent.unmarshalSortValues(QueryComponent.java:1034)\n\tat
org.apache.solr.handler.component.QueryComponent.mergeIds(QueryComponent.java:885)\n\tat
org.apache.solr.handler.component.QueryComponent.handleRegularResponses(QueryComponent.java:585)\n\tat
org.apache.solr.handler.component.QueryComponent.handleResponses(QueryComponent.java:564)\n\tat
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:423)\n\tat
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177)\n\tat
org.apache.solr.core.SolrCore.execute(SolrCore.java:2484)\n\tat
org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:720)\n\tat
org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:526)\n\tat
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:382)\n\tat
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)\n\tat
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1751)\n\tat
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)\n\tat
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)\n\tat
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)\n\tat
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)\n\tat
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)\n\tat
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)\n\tat
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)\n\tat
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)\n\tat
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)\n\tat
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)\n\tat
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)\n\tat
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)\n\tat
org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)\n\tat
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)\n\tat
org.eclipse.jetty.server.Server.handle(Server.java:534)\n\tat
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)\n\tat
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)\n\tat
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)\n\tat
org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)\n\tat
org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)\n\tat
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)\n\tat
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)\n\tat
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)\n\tat
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)\n\tat
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)\n\tat
java.lang.Thread.run(Thread.java:745)\n",
"code": 500
  }
}



If I target one specific shard I get a response:

http://localhost:8983/solr/suggest-test/select?q=obie&shards=shard2

{
  "responseHeader": {
"zkConnected": true,
"status": 0,
"QTime": 1,
"params": {
  "q": "obie",
  "shards": "shard2"
}
  },
  "response": {
"numFound": 1,
"start": 0,
"docs": [
  {
"id": "911",
"suggest_t": [
  "Obie Nasey"
],
"_version_": 1583864689849270273,
"suggest_t_str": [
  "Obie Nasey"
]
  }
]
  },
  "suggest": {
"fuzzySuggester": {
  "obie": {
"numFound": 1,
"suggestions": [
  {
"term": "Obie Nasey",
"weight": 0,
"payload": ""
  }
]
  }
}
  }
}



But if I target the other one (I have two nodes) I get the same error:

http://localhost:8983/solr/suggest-test/select?q=obie&shards=shard1

{
  "responseHeader": {
"zkConnected": true,
"status": 500,
"QTime": 12,
"params": {
 

Use case for the Shingle Filter

[Ryan Yacyshyn]

Hi everyone,

I was thinking of using the Shingle Filter to help solve an issue I'm
facing. I can see this working in the analysis panel in the Solr admin, but
not when I make my queries.

I find out it's because of the query parser splitting up the tokens on
white space before passing them along.

This made me wonder what a practical use case can be, for using the shingle
filter?

Any enlightenment on this would be much appreciated!

Thanks,
Ryan

Re: Use case for the Shingle Filter

[Ryan Yacyshyn]

The query parser will split on whitespace. I'm not sure how I can use the
shingle filter in my query, and use-cases for it. For example, if my
fieldType looks like this:


  


  
  


**
  


and I have a document that has "my babysitter is terrific" in the content_t
field, a query such as:

http://localhost:8983/solr/collection_name/select?q={!lucene}content_t:(the
baby
sitter was here)

won't return the document. I was hoping I'd get tokens like "the
thebaby baby babysitter sitter sitterwas ..." when querying.





On Sun, 5 Mar 2017 at 23:59 Ryan Josal  wrote:

> I thought new versions of solr didn't split on whitespace at the query
> parser anymore, so this should work?
>
> That being said, I think I remember it having a problem coming after a
> synonym filter.  IIRC, if your input is "Foo Bar" and you have a synonym
> "foo <=> baz" you would get foobaz bazbar instead of foobar and bazbar.  I
> wrote a custom shingler to account for that.
>
> Ryan
>
> On Sun, Mar 5, 2017 at 02:48 Markus Jelsma 
> wrote:
>
> > Hello - we use it for text classification and online near-duplicate
> > document detection/filtering. Using shingles means you want to consider
> > order in the text. It is analogous to using bigrams and trigrams when
> doing
> > language detection, you cannot distinguish between Danish and Norwegian
> > solely on single characters.
> >
> > Markus
> >
> >
> >
> > -Original message-
> > > From:Ryan Yacyshyn 
> > > Sent: Sunday 5th March 2017 5:57
> > > To: solr-user@lucene.apache.org
> > > Subject: Use case for the Shingle Filter
> > >
> > > Hi everyone,
> > >
> > > I was thinking of using the Shingle Filter to help solve an issue I'm
> > > facing. I can see this working in the analysis panel in the Solr admin,
> > but
> > > not when I make my queries.
> > >
> > > I find out it's because of the query parser splitting up the tokens on
> > > white space before passing them along.
> > >
> > > This made me wonder what a practical use case can be, for using the
> > shingle
> > > filter?
> > >
> > > Any enlightenment on this would be much appreciated!
> > >
> > > Thanks,
> > > Ryan
> > >
> >
>

Building Solr greater than 6.2.1

[Ryan Yacyshyn]

Hi all,

I'm trying to build Solr 6.5.1 but it's is failing. I'm able to
successfully build 6.2.1. I've tried 6.4.0, 6.4.2, and 6.5.1 but the build
fails. I'm not sure what the issue could be. I'm running `ant server` in
the solr dir and this is where it fails:

ivy-configure:
[ivy:configure] :: loading settings :: file =
/Users/rye/lucene-solr2/lucene/top-level-ivy-settings.xml

resolve:

common.init:

compile-lucene-core:

init:

-clover.disable:

-clover.load:

-clover.classpath:

-clover.setup:

clover:

compile-core:

-clover.disable:

-clover.load:

-clover.classpath:

-clover.setup:

clover:

common.compile-core:
[mkdir] Created dir:
/Users/rye/lucene-solr2/lucene/build/test-framework/classes/java
[javac] Compiling 186 source files to
/Users/rye/lucene-solr2/lucene/build/test-framework/classes/java
[javac]
/Users/rye/lucene-solr2/lucene/test-framework/src/java/org/apache/lucene/util/RamUsageTester.java:164:
error: no suitable method found for
collect(Collector>)
[javac]   .collect(Collectors.toList());
[javac]   ^
[javac] method Stream.collect(Supplier,BiConsumer,BiConsumer) is not applicable
[javac]   (cannot infer type-variable(s) R#1
[javac] (actual and formal argument lists differ in length))
[javac] method Stream.collect(Collector) is not applicable
[javac]   (cannot infer type-variable(s) R#2,A,CAP#3,T#2
[javac] (argument mismatch; Collector>
cannot be converted to Collector>))
[javac]   where R#1,T#1,R#2,A,T#2 are type-variables:
[javac] R#1 extends Object declared in method
collect(Supplier,BiConsumer,BiConsumer)
[javac] T#1 extends Object declared in interface Stream
[javac] R#2 extends Object declared in method
collect(Collector)
[javac] A extends Object declared in method
collect(Collector)
[javac] T#2 extends Object declared in method toList()
[javac]   where CAP#1,CAP#2,CAP#3,CAP#4 are fresh type-variables:
[javac] CAP#1 extends Object from capture of ?
[javac] CAP#2 extends Object from capture of ?
[javac] CAP#3 extends Object from capture of ?
[javac] CAP#4 extends Object from capture of ?
[javac] Note: Some input files use or override a deprecated API.
[javac] Note: Recompile with -Xlint:deprecation for details.
[javac] 1 error

BUILD FAILED
/Users/rye/lucene-solr2/solr/build.xml:463: The following error occurred
while executing this line:
/Users/rye/lucene-solr2/solr/common-build.xml:476: The following error
occurred while executing this line:
/Users/rye/lucene-solr2/solr/contrib/map-reduce/build.xml:53: The following
error occurred while executing this line:
/Users/rye/lucene-solr2/solr/contrib/morphlines-cell/build.xml:45: The
following error occurred while executing this line:
/Users/rye/lucene-solr2/solr/common-build.xml:443: The following error
occurred while executing this line:
/Users/rye/lucene-solr2/solr/test-framework/build.xml:35: The following
error occurred while executing this line:
/Users/rye/lucene-solr2/lucene/common-build.xml:767: The following error
occurred while executing this line:
/Users/rye/lucene-solr2/lucene/common-build.xml:501: The following error
occurred while executing this line:
/Users/rye/lucene-solr2/lucene/common-build.xml:1967: Compile failed; see
the compiler error output for details.

Total time: 2 minutes 28 seconds

Java version:

java version "1.8.0_25"
Java(TM) SE Runtime Environment (build 1.8.0_25-b17)
Java HotSpot(TM) 64-Bit Server VM (build 25.25-b02, mixed mode)

ant: Apache Ant(TM) version 1.10.0 compiled on December 27 2016
ivy: ivy-2.3.0.jar

Any suggestions I can try?

Regards,
Ryan

Re: Building Solr greater than 6.2.1

[Ryan Yacyshyn]

Thanks Alex, it's working now. I had to update Java.

Regards,
Ryan



On Mon, 1 May 2017 at 14:48 Alexandre Rafalovitch 
wrote:

> Make sure your Java is latest update. Seriously
>
> Also, if still failing, try blowing away your Ivy cache.
>
> Regards,
> Alex
>
> On 1 May 2017 6:34 AM, "Ryan Yacyshyn"  wrote:
>
> > Hi all,
> >
> > I'm trying to build Solr 6.5.1 but it's is failing. I'm able to
> > successfully build 6.2.1. I've tried 6.4.0, 6.4.2, and 6.5.1 but the
> build
> > fails. I'm not sure what the issue could be. I'm running `ant server` in
> > the solr dir and this is where it fails:
> >
> > ivy-configure:
> > [ivy:configure] :: loading settings :: file =
> > /Users/rye/lucene-solr2/lucene/top-level-ivy-settings.xml
> >
> > resolve:
> >
> > common.init:
> >
> > compile-lucene-core:
> >
> > init:
> >
> > -clover.disable:
> >
> > -clover.load:
> >
> > -clover.classpath:
> >
> > -clover.setup:
> >
> > clover:
> >
> > compile-core:
> >
> > -clover.disable:
> >
> > -clover.load:
> >
> > -clover.classpath:
> >
> > -clover.setup:
> >
> > clover:
> >
> > common.compile-core:
> > [mkdir] Created dir:
> > /Users/rye/lucene-solr2/lucene/build/test-framework/classes/java
> > [javac] Compiling 186 source files to
> > /Users/rye/lucene-solr2/lucene/build/test-framework/classes/java
> > [javac]
> > /Users/rye/lucene-solr2/lucene/test-framework/src/
> > java/org/apache/lucene/util/RamUsageTester.java:164:
> > error: no suitable method found for
> > collect(Collector>)
> > [javac]   .collect(Collectors.toList());
> > [javac]   ^
> > [javac] method Stream.collect(Supplier,BiConsumer > super CAP#2>,BiConsumer) is not applicable
> > [javac]   (cannot infer type-variable(s) R#1
> > [javac] (actual and formal argument lists differ in length))
> > [javac] method Stream.collect(Collector > CAP#2,A,R#2>) is not applicable
> > [javac]   (cannot infer type-variable(s) R#2,A,CAP#3,T#2
> > [javac] (argument mismatch;
> Collector>
> > cannot be converted to Collector>))
> > [javac]   where R#1,T#1,R#2,A,T#2 are type-variables:
> > [javac] R#1 extends Object declared in method
> > collect(Supplier,BiConsumer > T#1>,BiConsumer)
> > [javac] T#1 extends Object declared in interface Stream
> > [javac] R#2 extends Object declared in method
> > collect(Collector)
> > [javac] A extends Object declared in method
> > collect(Collector)
> > [javac] T#2 extends Object declared in method toList()
> > [javac]   where CAP#1,CAP#2,CAP#3,CAP#4 are fresh type-variables:
> > [javac] CAP#1 extends Object from capture of ?
> > [javac] CAP#2 extends Object from capture of ?
> > [javac] CAP#3 extends Object from capture of ?
> > [javac] CAP#4 extends Object from capture of ?
> > [javac] Note: Some input files use or override a deprecated API.
> > [javac] Note: Recompile with -Xlint:deprecation for details.
> > [javac] 1 error
> >
> > BUILD FAILED
> > /Users/rye/lucene-solr2/solr/build.xml:463: The following error occurred
> > while executing this line:
> > /Users/rye/lucene-solr2/solr/common-build.xml:476: The following error
> > occurred while executing this line:
> > /Users/rye/lucene-solr2/solr/contrib/map-reduce/build.xml:53: The
> > following
> > error occurred while executing this line:
> > /Users/rye/lucene-solr2/solr/contrib/morphlines-cell/build.xml:45: The
> > following error occurred while executing this line:
> > /Users/rye/lucene-solr2/solr/common-build.xml:443: The following error
> > occurred while executing this line:
> > /Users/rye/lucene-solr2/solr/test-framework/build.xml:35: The following
> > error occurred while executing this line:
> > /Users/rye/lucene-solr2/lucene/common-build.xml:767: The following error
> > occurred while executing this line:
> > /Users/rye/lucene-solr2/lucene/common-build.xml:501: The following error
> > occurred while executing this line:
> > /Users/rye/lucene-solr2/lucene/common-build.xml:1967: Compile failed; see
> > the compiler error output for details.
> >
> > Total time: 2 minutes 28 seconds
> >
> > Java version:
> >
> > java version "1.8.0_25"
> > Java(TM) SE Runtime Environment (build 1.8.0_25-b17)
> > Java HotSpot(TM) 64-Bit Server VM (build 25.25-b02, mixed mode)
> >
> > ant: Apache Ant(TM) version 1.10.0 compiled on December 27 2016
> > ivy: ivy-2.3.0.jar
> >
> > Any suggestions I can try?
> >
> > Regards,
> > Ryan
> >
>

Re: Building Solr greater than 6.2.1

[Ryan Yacyshyn]

I was using Java 8 all along but more specifically, it was 1.8.0_25 (full
details below).

java version "1.8.0_25"
Java(TM) SE Runtime Environment (build 1.8.0_25-b17)
Java HotSpot(TM) 64-Bit Server VM (build 25.25-b02, mixed mode)

I initially didn't think it was my Java version so I just cleared my ivy
cache and tried building again but it failed. Only after updating to:

java version "1.8.0_131"
Java(TM) SE Runtime Environment (build 1.8.0_131-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)

did it work.

Regards,
Ryan



On Mon, 1 May 2017 at 20:44 Shawn Heisey  wrote:

> On 5/1/2017 6:34 AM, Ryan Yacyshyn wrote:
> > Thanks Alex, it's working now. I had to update Java.
>
> What version were you using?  Lucene/Solr 6 requires Java 8.  I don't
> think that building 6.2.1 would have been successful if it weren't Java 8.
>
> I'm not familiar with any specific Java release requirements (more
> specific than version 8) for any 6.x version.
>
> Thanks,
> Shawn
>
>

Creating a Custom Query Response Writer

[Ryan Yacyshyn]

Hey Everyone,

I'm a little stuck on building a custom query response writer. I want to
create a response writer similar to the one explained in the book, Taming
Text, on the TypeAheadResponseWriter. I know I need to implement the
QueryResponseWriter, but I'm not sure where to find the Solr JAR files I
need to include. Where can I find these?

Thanks,
Ryan

Re: Creating a Custom Query Response Writer

[Ryan Yacyshyn]

Hi Erik,

Wow that's great. Thanks for the explanation, I tried using the
VelocityResponseWriter with your template provided and it worked as
expected, makes sense!

What if I want to return a custom JSON response back, rather than HTML for
auto-suggesting? I'm thinking about using Twitter's typeahead jQuery plugin
<https://twitter.github.io/typeahead.js/examples/#custom-templates> and
passing it JSON, and have a template on the front-end to the autosuggest.

Ryan



On Sat Dec 06 2014 at 3:41:38 AM Erik Hatcher 
wrote:

> Ryan - I just pulled Taming Text off my shelf and refreshed my memory of
> this custom response writer.
>
> While having a custom writer is a neat example, it’s unnecessary for that
> particular functionality.  Solr has a built-in templatable response writer,
> the VelocityResponseWriter.  You can see it in action for a similar suggest
> feature in Solr’s example /browse interface (type “ip” and wait a second in
> the /browse UI with the sample data indexed).  In there is a little bit of
> jQuery autocomplete plugin usage that calls back to the /terms handler,
> using a suggest.vm template (in conf/velocity).  The difference with the
> Taming Text example is that it is returns stored fields of a standard
> search rather than just raw terms; with a little adjustment you can get
> basically the same thing as TT.  Leveraging the Solr example (v4.10.2 for
> me here), I created a conf/velocity/typeahead.vm:
>
>   
> #foreach($doc in $response.results)
>   $doc.name
> #end
>   
>
> (the docs in the example data have a ‘name’ field)
>
> This request  http://localhost:8983/solr/collection1/select?q=name%
> 3Aip*&wt=velocity&v.template=typeahead <http://localhost:8983/solr/
> collection1/select?q=name:ip*&wt=velocity&v.template=typeahead> results
> in this response:
>
>   
>   Belkin Mobile Power Cord for iPod w/ Dock
>   iPod & iPod Mini USB 2.0 Cable
>   Apple 60 GB iPod with Video Playback Black
>   
>
> Erik
>
>
> > On Dec 6, 2014, at 2:24 AM, Ryan Yacyshyn 
> wrote:
> >
> > Hey Everyone,
> >
> > I'm a little stuck on building a custom query response writer. I want to
> > create a response writer similar to the one explained in the book, Taming
> > Text, on the TypeAheadResponseWriter. I know I need to implement the
> > QueryResponseWriter, but I'm not sure where to find the Solr JAR files I
> > need to include. Where can I find these?
> >
> > Thanks,
> > Ryan
>
>

Re: Creating a Custom Query Response Writer

[Ryan Yacyshyn]

Thanks Erik. That's what I did in the end and it works great. I thought I'd
need to create a custom response to remove unnecessary fields but was able
to make the request return pretty much only what I need, even adding
omitHeader=true. I'm using the EdgeNGramFilterFactory during indexing on
the title field, and then returning the just the title on the matching set.

For example, a call to http://localhost:8983/solr/movies/suggest_movie?q=sav,
will return:

{
"response": {
"numFound": 2,
"start": 0,
"docs": [
{
"title": "Saving Private Ryan"
},
{
"title": "Into the Breach: 'Saving Private Ryan'"
}
]
}
}

Thanks for your help!

Ryan




On Sun Dec 07 2014 at 7:14:23 AM Erik Hatcher 
wrote:

> I wouldn’t personally do anything custom for JSON - but rather just pull
> out what you need client-side (and make the request such that it doesn’t
> return more than you need).  Doing a custom JSON format for this would
> limit your later flexibility in case you wanted to get different pieces of
> the response, possibly.   Also, if you had your own custom JSON format
> writer it could become problematic later on if Solr’s internal APIs changed
> and you needed to upgrade (or worse, maintain multiple versions).
>
> Erik
>
> > On Dec 6, 2014, at 5:13 PM, Ryan Yacyshyn 
> wrote:
> >
> > Hi Erik,
> >
> > Wow that's great. Thanks for the explanation, I tried using the
> > VelocityResponseWriter with your template provided and it worked as
> > expected, makes sense!
> >
> > What if I want to return a custom JSON response back, rather than HTML
> for
> > auto-suggesting? I'm thinking about using Twitter's typeahead jQuery
> plugin
> > <https://twitter.github.io/typeahead.js/examples/#custom-templates <
> https://twitter.github.io/typeahead.js/examples/#custom-templates>> and
> > passing it JSON, and have a template on the front-end to the autosuggest.
> >
> > Ryan
> >
> >
> >
> > On Sat Dec 06 2014 at 3:41:38 AM Erik Hatcher  <mailto:erik.hatc...@gmail.com>>
> > wrote:
> >
> >> Ryan - I just pulled Taming Text off my shelf and refreshed my memory of
> >> this custom response writer.
> >>
> >> While having a custom writer is a neat example, it’s unnecessary for
> that
> >> particular functionality.  Solr has a built-in templatable response
> writer,
> >> the VelocityResponseWriter.  You can see it in action for a similar
> suggest
> >> feature in Solr’s example /browse interface (type “ip” and wait a
> second in
> >> the /browse UI with the sample data indexed).  In there is a little bit
> of
> >> jQuery autocomplete plugin usage that calls back to the /terms handler,
> >> using a suggest.vm template (in conf/velocity).  The difference with the
> >> Taming Text example is that it is returns stored fields of a standard
> >> search rather than just raw terms; with a little adjustment you can get
> >> basically the same thing as TT.  Leveraging the Solr example (v4.10.2
> for
> >> me here), I created a conf/velocity/typeahead.vm:
> >>
> >>  
> >>#foreach($doc in $response.results)
> >>  $doc.name
> >>#end
> >>  
> >>
> >> (the docs in the example data have a ‘name’ field)
> >>
> >> This request  http://localhost:8983/solr/collection1/select?q=name%
> >> 3Aip*&wt=velocity&v.template=typeahead <http://localhost:8983/solr/ <
> http://localhost:8983/solr/>
> >> collection1/select?q=name:ip*&wt=velocity&v.template=typeahead> results
> >> in this response:
> >>
> >>  
> >>  Belkin Mobile Power Cord for iPod w/ Dock
> >>  iPod & iPod Mini USB 2.0 Cable
> >>  Apple 60 GB iPod with Video Playback Black
> >>  
> >>
> >>Erik
> >>
> >>
> >>> On Dec 6, 2014, at 2:24 AM, Ryan Yacyshyn 
> >> wrote:
> >>>
> >>> Hey Everyone,
> >>>
> >>> I'm a little stuck on building a custom query response writer. I want
> to
> >>> create a response writer similar to the one explained in the book,
> Taming
> >>> Text, on the TypeAheadResponseWriter. I know I need to implement the
> >>> QueryResponseWriter, but I'm not sure where to find the Solr JAR files
> I
> >>> need to include. Where can I find these?
> >>>
> >>> Thanks,
> >>> Ryan
>
>

Re: AW: AW: Keeping capitalization in suggestions?

[Ryan Yacyshyn]

Hi Clemens,

I recently added typeahead functionality to something I'm playing with and
I used the EdgeNGramFilterFactory to help. I just tried this out after
adding a doc with "Chamäleon" in my title.

I was able to get "Chamäleon", with a capital C, returned I searched for
chama, Chama, chamã, and Chamã.

Here's what I have in my files:

-
solrconfig.xml:


  
json
edismax
10
true 
value:title 
title^10 suggest_ngram 
  


-
schema.xml:


 
   
   
   
   
   

 
 
   
   
   
   
   
 


...



...



-
request:

http://localhost:8983/solr/movies/suggest_movie?q=chama

-
response:

{
"response": {
"numFound": 1,
"start": 0,
"docs": [
{
"value": "Chamäleon"
}
]
}
}

Hope this helps?

Ryan




On Tue Dec 09 2014 at 7:21:02 AM Michael Sokolov <
msoko...@safaribooksonline.com> wrote:

> Clemens --
>
>what I do (see suggestions of titles of books on $EMPLOYER's web
> site) is to define a field with no analysis (type=keyword, use
> KeywordAnalyzer) and build the suggestions from that.  Then tell AIS to
> use an analyzer internally to pick out word from that (StandardAnalyzer,
> or WhitespaceAnalyzer, with LowerCaseFilter - however you want the
> matching to work in the suggester).  It will return the terms from the
> source field.
>
> You didn't show the definition of your "suggest" field - I expect it
> must be analyzed, right?  Just don't do that.
>
> -Mike
>
> On 12/09/2014 08:58 AM, Clemens Wyss DEV wrote:
> > Thanks for all the insightful links.
> > I tried http://www.cominvent.com/2012/01/25/super-flexible-autocompl
> ete-with-solr but that approach returns searchresults instead of
> term-suggestions.
> >
> > I have (at the moment) a solution based on http://wiki.apache.org/solr/
> TermsComponent . But I might want multi-term-suggestions (and fuzzyness).
> > Therefore I'd be very much interested how AnalyzingInfixLookupFactory
> (or any other suggest-component) would allow to
> > a) return case-sensitive suggestions (i.e. as-indexed/stored)
> > b) allow case-insensitive suggestion-lookup
> > ?
> > Anybody else doing what I'd like to do?
> >
> > -Ursprüngliche Nachricht-
> > Von: Ahmet Arslan [mailto:iori...@yahoo.com.INVALID]
> > Gesendet: Montag, 8. Dezember 2014 19:25
> > An: solr-user@lucene.apache.org
> > Betreff: Re: AW: Keeping capitalization in suggestions?
> >
> > Hi Clemens,
> >
> > There a a number of ways to implement auto complete/suggest. Some of
> them pull data from indexed terms, therefore they will be lowercased. Some
> pull data from stored values, therefore capitalisation is preserved.
> >
> > Here are great resources on this topic.
> >
> > https://lucidworks.com/blog/auto-suggest-from-popular-querie
> s-using-edgengrams/
> > http://blog.trifork.com/2012/02/15/different-ways-to-make-au
> to-suggestions-with-solr/
> > http://www.cominvent.com/2012/01/25/super-flexible-autocompl
> ete-with-solr/
> >
> > Ahmet
> >
> >
> > On Monday, December 8, 2014 5:43 PM, Clemens Wyss DEV <
> clemens...@mysign.ch> wrote:
> >
> > Allthough making use of AnalyzingInfixSuggester I still getting "either
> or".
> >
> > When lowercase-filter is active I always get suggestions, BUT they are
> lowercased (i.e. "chamäleon").
> > When lowercase-filter is not active I only get suggestions when querying
> "Chamä"
> >
> > my solrconfig.xml
> > ...
> >   name="/suggest">
> >  
> >  none
> >  json
> >  false
> >  true
> >  suggestDictionary
> >  true
> >  5
> >  false
> >  
> >  
> >  suggest
> >  
> >  
> > ...
> >  
> >
> >  suggestDictionary
> >  org.apache.solr.spelling.suggest.
> Suggester
> >  org.apache.solr.spelling.suggest.fst.
> AnalyzingInfixLookupFactory
> >  org.apache.solr.spelling.suggest.
> DocumentDictionaryFactory
> >  suggest
> >  true
> >  suggester
> >  text_suggest
> >  4
> >
> >  
> > ...
> >
> > my schema.xml
> > ...
> >  type="text_suggest"/> ...
> >   positionIncrementGap="100">
> >
> >  
> >   words="stopwords.txt"/>
> > 
> >
> >
> >  
> >   words="stopwords.txt"/>
> > 
> >
> >  
> > ...
> >
> >
> > -Ursprüngliche Nachricht-
> > Von: Michael Sokolov [mailto:msoko...@safaribooksonline.com]
> > Gesendet: Donnerstag, 4. Dezember 2014 14:05
> > An: solr-user@lucene.apache.org
> > Betreff: Re: Keeping capitalization in suggestions?
> >
> > Have a look at AnalyzingInfixSuggester - it does what you want.
> >
> > -Mike
> >
> > On 12/4/14 3:05 AM, Clemens Wyss DEV wrote:
> >> When I index a text such as "Chamäleon" and look for suggestions for
> "chamä" and/or "Chamä", I'd expect to get "Chamäleon" (uppe

Re: Contextual Synonym Filter

[Ryan Yacyshyn]

Hi Luca,

I wonder if payloads might be able to solve your case for contextual
synonyms?

http://jorgelbg.github.io/posts/solr-contextual-synonyms

Ryan





On Mon, 3 Sep 2018 at 21:59 Andrea Gazzarini  wrote:

> Hi Luca,
> I believe this is not an easy task to do passing through Solr/Lucene
> internals; did you try to use what Solr offers out of the box?
> For example, you could define several fields associated where each
> corresponding field type uses a different synonym set. So you would have
>
>   * F1 -> FT1 -> SYNSET1
>   * F2 -> FT2 -> SYNSET2
>   * ...
>
> if you query using *fq=F1:something* then the *FT1* (and the *SYNSET1*)
> will be used, if you query using *fq=F2:something* then the *FT2* (and
> the *SYNSET2*) will be used, and so on
>
> I don't know your context so my suggestion could be absolutely in the
> wrong path
>
> Best,
> Andrea
>
> On 03/09/2018 15:41, Vergantini Luca wrote:
> >
> > I need to create a contextual Synonym Filter:
> >
> > I need that the Synonym Filter load different synonym configuration
> > based on the fq query parameter.
> >
> > I’ve already modified the SynonymGraphFilterFactory to load from DB
> > (this is another requirement) but I can’t understand how to make the
> > fq parameter arrive to the Factory.
> >
> > Maybe I need a Query Parser plugin?
> >
> > Please help
> >
> >
> >
> > Luca Vergantini
> >
> > Whitehall Reply
> > Via del Giorgione, 59
> > 00147 - Roma - ITALY
> > phone: +39 06 844341 <+39%2006%20844341>
> > l.vergant...@reply.it 
> > www.reply.it
> >
> > Whitehall Reply
>
>

DateRangeField, but with Integers instead of dates?

[Ryan Yacyshyn]

Hi all,

Is there a way I can perform a filter query on a field that contains a
numeric range? Very much like how we can query over a date range:

https://lucene.apache.org/solr/guide/7_3/working-with-dates.html#more-daterangefield-details

I'd like to index documents that have a field containing a range of
integers, such as:

doc 1 --> { my_field: [200 TO 250] }
doc 2 --> { my_field: [240 TO 270] }

And then have a query such as fq={!field f=my_field op=Contains}[210 TO
230] to return the first doc. This is possible with Dates, can I do
something like this with integers?

Couldn't find much online on this, if anyone can point me in the right
direction that would be great!

Ryan

Re: DateRangeField, but with Integers instead of dates?

[Ryan Yacyshyn]

Hi Edwin,

That's correct. I would like to have one field that stores a *range* of
Integers, like field=[100 TO 250]. And my query might contain a range. If
there's any overlap, I need the document returned.

I did come across this in Lucene:
https://lucene.apache.org/core/7_2_1/core/org/apache/lucene/document/IntRange.html

And it's exactly what I'm looking for but unfortunately adding a fieldType
with this class in Solr didn't work for me.





On Thu, Mar 7, 2019 at 1:53 PM Zheng Lin Edwin Yeo 
wrote:

> Hi,
>
> Do you mean you plan to index a range of number into a single field?
>
> Regards,
> Edwin
>
> On Wed, 6 Mar 2019 at 17:53, Ryan Yacyshyn 
> wrote:
>
> > Hi all,
> >
> > Is there a way I can perform a filter query on a field that contains a
> > numeric range? Very much like how we can query over a date range:
> >
> >
> >
> https://lucene.apache.org/solr/guide/7_3/working-with-dates.html#more-daterangefield-details
> >
> > I'd like to index documents that have a field containing a range of
> > integers, such as:
> >
> > doc 1 --> { my_field: [200 TO 250] }
> > doc 2 --> { my_field: [240 TO 270] }
> >
> > And then have a query such as fq={!field f=my_field op=Contains}[210 TO
> > 230] to return the first doc. This is possible with Dates, can I do
> > something like this with integers?
> >
> > Couldn't find much online on this, if anyone can point me in the right
> > direction that would be great!
> >
> > Ryan
> >
>

Re: DateRangeField, but with Integers instead of dates?

[Ryan Yacyshyn]

Thanks Mikhail. I missed this.

I'll try it out. My range of numbers are in the hundreds, I'll encode it as
dates in seconds.







On Thu, Mar 7, 2019 at 3:50 PM Mikhail Khludnev  wrote:

> I think https://wiki.apache.org/solr/SpatialForTimeDurations provides a
> good guidance.
>
> On Thu, Mar 7, 2019 at 8:53 AM Zheng Lin Edwin Yeo 
> wrote:
>
> > Hi,
> >
> > Do you mean you plan to index a range of number into a single field?
> >
> > Regards,
> > Edwin
> >
> > On Wed, 6 Mar 2019 at 17:53, Ryan Yacyshyn 
> > wrote:
> >
> > > Hi all,
> > >
> > > Is there a way I can perform a filter query on a field that contains a
> > > numeric range? Very much like how we can query over a date range:
> > >
> > >
> > >
> >
> https://lucene.apache.org/solr/guide/7_3/working-with-dates.html#more-daterangefield-details
> > >
> > > I'd like to index documents that have a field containing a range of
> > > integers, such as:
> > >
> > > doc 1 --> { my_field: [200 TO 250] }
> > > doc 2 --> { my_field: [240 TO 270] }
> > >
> > > And then have a query such as fq={!field f=my_field op=Contains}[210 TO
> > > 230] to return the first doc. This is possible with Dates, can I do
> > > something like this with integers?
> > >
> > > Couldn't find much online on this, if anyone can point me in the right
> > > direction that would be great!
> > >
> > > Ryan
> > >
> >
>
>
> --
> Sincerely yours
> Mikhail Khludnev
>

Streaming Expressions using Solrj.io

[Ryan Yacyshyn]

Hello all,

I'd like to get a stream of search results using the solrj.io package but
running into a small issue. It seems to have something to do with the
HttpClientUtil. I'm testing on SolrCloud 7.1.0, using the
sample_techproducts_configs configs, and indexed the manufacturers.xml file.

I'm following the test code in the method `testCloudSolrStreamWithZkHost`
found in StreamExpressionTest.java:

```
package ca.ryac.testing;

import java.io.IOException;
import java.util.ArrayList;
import java.util.List;

import org.apache.solr.client.solrj.io.SolrClientCache;
import org.apache.solr.client.solrj.io.Tuple;
import org.apache.solr.client.solrj.io.stream.CloudSolrStream;
import org.apache.solr.client.solrj.io.stream.StreamContext;
import org.apache.solr.client.solrj.io.stream.TupleStream;
import org.apache.solr.client.solrj.io.stream.expr.StreamExpression;
import org.apache.solr.client.solrj.io.stream.expr.StreamExpressionParser;
import org.apache.solr.client.solrj.io.stream.expr.StreamFactory;

public class SolrStreamingClient {

  String zkHost = "localhost:9983";
  String COLLECTIONORALIAS = "gettingstarted";

  public SolrStreamingClient() throws Exception {
init();
  }

  public static void main(String[] args) throws Exception {
new SolrStreamingClient();
  }

  private void init() throws Exception {

System.out.println(zkHost);

StreamFactory factory = new StreamFactory();

StreamExpression expression;
CloudSolrStream stream;
List tuples;
StreamContext streamContext = new StreamContext();
SolrClientCache solrClientCache = new SolrClientCache();
streamContext.setSolrClientCache(solrClientCache);

// basic test..
String expr = "search(" + COLLECTIONORALIAS + ", zkHost=\"" + zkHost
+ "\", q=*:*, fl=\"id,compName_s\", sort=\"compName_s asc\")";

System.out.println(expr);
expression = StreamExpressionParser.parse(expr);

stream = new CloudSolrStream(expression, factory);
stream.setStreamContext(streamContext);
tuples = getTuples(stream);

System.out.println(tuples.size());
  }

  protected List getTuples(TupleStream tupleStream) throws
IOException {
List tuples = new ArrayList();

try {
  System.out.println("open stream..");
  tupleStream.open();
  for (Tuple t = tupleStream.read(); !t.EOF; t = tupleStream.read()) {
tuples.add(t);
  }
} finally {
  tupleStream.close();
}
return tuples;
  }
}
```

And this is the output I get:

---
localhost:9983
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further
details.
search(gettingstarted, zkHost="localhost:9983", q=*:*, fl="id,compName_s",
sort="compName_s asc")
open stream..
Exception in thread "main" java.lang.NoSuchMethodError:
org.apache.http.impl.client.HttpClientBuilder.evictIdleConnections(JLjava/util/concurrent/TimeUnit;)Lorg/apache/http/impl/client/HttpClientBuilder;
  at
org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:279)
  at
org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:298)
  at
org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:236)
  at
org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:223)
  at
org.apache.solr.client.solrj.impl.CloudSolrClient.(CloudSolrClient.java:276)
  at
org.apache.solr.client.solrj.impl.CloudSolrClient$Builder.build(CloudSolrClient.java:1525)
  at
org.apache.solr.client.solrj.io.SolrClientCache.getCloudSolrClient(SolrClientCache.java:62)
  at
org.apache.solr.client.solrj.io.stream.TupleStream.getShards(TupleStream.java:138)
  at
org.apache.solr.client.solrj.io.stream.CloudSolrStream.constructStreams(CloudSolrStream.java:368)
  at
org.apache.solr.client.solrj.io.stream.CloudSolrStream.open(CloudSolrStream.java:274)
  at
ca.ryac.testing.SolrStreamingClient.getTuples(SolrStreamingClient.java:61)
  at ca.ryac.testing.SolrStreamingClient.init(SolrStreamingClient.java:51)
  at ca.ryac.testing.SolrStreamingClient.(SolrStreamingClient.java:22)
  at ca.ryac.testing.SolrStreamingClient.main(SolrStreamingClient.java:26)
---

It's not finding or connecting to my SolrCloud instance, I can put
*anything* in zkHost and get the same results. Not really sure why it can't
find or connect to it. Any thoughts or ideas?

Thank you,
Ryan

Re: Streaming Expressions using Solrj.io

[Ryan Yacyshyn]

Hi Shawn,

Thanks for this.

I'm using maven and I had another library unrelated to solrj loading an
older version of the httpclient (4.4.1), which was causing the problem.

Regards,
Ryan




On Wed, 21 Feb 2018 at 12:50 Shawn Heisey  wrote:

> On 2/20/2018 7:54 PM, Ryan Yacyshyn wrote:
> > I'd like to get a stream of search results using the solrj.io package
> but
> > running into a small issue.
>
> 
>
> > Exception in thread "main" java.lang.NoSuchMethodError:
> >
> org.apache.http.impl.client.HttpClientBuilder.evictIdleConnections(JLjava/util/concurrent/TimeUnit;)Lorg/apache/http/impl/client/HttpClientBuilder;
>
> There is a problem accessing the HttpClient library. Either the
> httpclient jar is missing from your project, or it's the wrong version.
> You can use pretty much any 4.5.x version for recent SolrJ versions.
> 3.x versions won't work at all, and older 4.x versions not work.  The
> 5.0 beta releases also won't work. You can find information about 4.0
> and later versions of HttpClient here:
>
> http://hc.apache.org/
>
> If you use a dependency manager like gradle, maven, or ivy for your
> project, just be sure it's set to pull in all transitive dependencies
> for solrj, and you should be fine.  If you manage dependencies manually,
> you will find all of the extra jars required by the solrj client in the
> download, in the dist/solrj-lib directory.  Note that you can very
> likely upgrade individual dependencies to newer versions than Solr
> includes with no issues.
>
> Thanks,
> Shawn
>
>

Altering the query if query contains all stopwods

[Ryan Yacyshyn]

Hi all,

I'm wondering if its possible to change/alter the query after it has been
analyzed by the stopword filter?

For example, if my query contains all stopwords then I'll get zero results
because the query is empty. I'd like to have a condition to change the
query so that if the query happens to be completely blank after stopword
removal, then do a wildcard search.

Thanks,
Ryan

Re: Altering the query if query contains all stopwods

[Ryan Yacyshyn]

Alright, thanks.

Yeah, the SuggestStopFilterFactory gets closer but isn't what I'm looking
for in this case!

Ryan



On Sat, 10 Mar 2018 at 06:12 Rick Leir  wrote:

> Tav, Ryan
> Now you have me wondering, should it be returning *:* or some general
> landing page.
>
> Suppose you had typeahead or autocomplete, it should ignore any stopwords
> list.
>
> By the way, people on this list have had good reasons why we should stop
> using stopwords.
> Cheers -- Rick
>
> On March 9, 2018 1:13:22 PM EST, tapan1707  wrote:
> >Hello Ryan,
> >Solr has a Filter class called solr.SuggestStopFilterFactory, which
> >basically works similar to solr.StopFilterFactory but with a slight
> >modification that if all of the words are present in stopwords.txt then
> >it
> >won't remove the last one.
> >I am not sure about wildcard search but if all of the query tokens are
> >stopwords.txt then at the very least it won't be returning the zero
> >results.(assuming that search results for the last word exists)
> >
> >
> >
> >--
> >Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
>
> --
> Sorry for being brief. Alternate email is rickleir at yahoo dot com

Solr 7.3.0 loading OpenNLPExtractNamedEntitiesUpdateProcessorFactory

[Ryan Yacyshyn]

Hi all,

I'm running into a small problem loading
the OpenNLPExtractNamedEntitiesUpdateProcessorFactory class, getting an
error saying it's not found. I'm loading all the required jar files,
according to the readme:

*solrconfig.xml*

  
  

  ...

  

  en-ner-person.bin
  opennlp-en-tokenization
  text
  people_ss

  

*managed-schema*

  

  

  

I have the three *.bin files in my conf directory, but when I try to reload
with this config I get this error:

```
{
  "responseHeader": {
"status": 500,
"QTime": 390
  },
  "error": {
"metadata": [
  "error-class",
  "org.apache.solr.common.SolrException",
  "root-error-class",
  "java.lang.ClassNotFoundException"
],
"msg": "Error handling 'reload' action",
"trace": "org.apache.solr.common.SolrException: Error handling 'reload'
action\n\tat
org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$2(CoreAdminOperation.java:112)\n\tat
org.apache.solr.handler.admin.CoreAdminOperation$$Lambda$103/335708295.execute(Unknown
Source)\n\tat
org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:358)\n\tat
org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:389)\n\tat
org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:174)\n\tat
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:195)\n\tat
org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:736)\n\tat
org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:717)\n\tat
org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:498)\n\tat
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:384)\n\tat
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:330)\n\tat
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1629)\n\tat
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)\n\tat
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)\n\tat
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)\n\tat
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)\n\tat
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:190)\n\tat
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)\n\tat
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)\n\tat
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)\n\tat
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)\n\tat
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)\n\tat
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)\n\tat
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)\n\tat
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)\n\tat
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)\n\tat
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:219)\n\tat
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)\n\tat
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)\n\tat
org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)\n\tat
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)\n\tat
org.eclipse.jetty.server.Server.handle(Server.java:530)\n\tat
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:347)\n\tat
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:256)\n\tat
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)\n\tat
org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)\n\tat
org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)\n\tat
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:247)\n\tat
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:140)\n\tat
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)\n\tat
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:382)\n\tat
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)\n\tat
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)\n\tat
java.lang.Thread.run(Thread.java:745)\nCaused by:
org.apache.solr.common.SolrException: Unable to reload core [nlp]\n\tat
org.apache.solr.core.CoreContainer.reload(CoreContainer.java:1311)\n\tat
org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$2(CoreAdminOperation.java:110)\n\t...
43 more\nCaused by: org.apache.solr.common.SolrException: Error load

Re: Solr 7.3.0 loading OpenNLPExtractNamedEntitiesUpdateProcessorFactory

[Ryan Yacyshyn]

Hi Shawn,

I'm pretty sure the paths to load the jars in analysis-extras is correct,
the jars in /contrib/analysis-extras/lib load fine. I verified this by
changing the name of solr.OpenNLPTokenizerFactory to
solr.OpenNLPTokenizerFactory2
and saw the new error. Changing it back to solr.OpenNLPTokenizerFactory
(without the "2") doesn't throw any errors, so I'm assuming these two
jar files (opennlp-maxent-3.0.3.jar and opennlp-tools-1.8.3.jar) must be
loading.

I tried swapping the order in which these jars are loaded as well, but no
luck there.

I have attached my solr.log file after a restart. Also included is my
solrconfig.xml and managed-schema. The path to my config
is /Users/ryan/solr-7.3.0/server/solr/nlp/conf and this is where I have the
OpenNLP bin files (en-ner-person.bin, en-sent.bin, and en-token.bin).
Configs are derived from the _default configset.

On a mac, and my Java version is:

java version "1.8.0_45"
Java(TM) SE Runtime Environment (build 1.8.0_45-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.45-b02, mixed mode)

Thanks,
Ryan



On Sun, 8 Apr 2018 at 21:34 Shawn Heisey  wrote:

> On 4/8/2018 2:36 AM, Ryan Yacyshyn wrote:
> > I'm running into a small problem loading
> > the OpenNLPExtractNamedEntitiesUpdateProcessorFactory class, getting an
> > error saying it's not found. I'm loading all the required jar files,
> > according to the readme:
>
> You've got a  element to load analysis-extras jars, but are you
> certain it's actually loading anything?
>
> Can you share a solr.log file created just after a Solr restart?  Not
> just a reload -- I'm asking for a restart so the log is more complete.
> With that, I can see what's happening and then ask more questions that
> may pinpoint something.
>
> Thanks,
> Shawn
>
>





  

  
  7.3.0

  

  
  
  

  
  

  
  

  
  

  
  
  

  
  

  
  ${solr.data.dir:}


  
  

  
  

  
  


















${solr.lock.type:native}













  


  
  
  
  
  
  

  
  



  ${solr.ulog.dir:}
  ${solr.ulog.numVersionBuckets:65536}




  ${solr.autoCommit.maxTime:15000}
  false





  ${solr.autoSoftCommit.maxTime:-1}




  

  
  

  
  


1024























true





20


200




  

  


  

  



false

  


  
  








  

  
  
  


  explicit
  10
  
  








  

  
  

  explicit
  json
  true

  


  
  

  explicit

  

  

  _text_

  

  
  

  true
  ignored_
  _text_

  

  

  
  

text_general





  default
  _text_
  solr.DirectSolrSpellChecker
  
  internal
  
  0.5
  
  2
  
  1
  
  5
  
  4
  
  0.01
  




  

  
  

  
  default
  on
  true
  10
  5
  5
  true
  true
  10
  5


  spellcheck

  

  
  

  
  

  true


  tvComponent

  

  

  
  

  
  

  true
  false


  terms

  


  
  

string
  

  
  

  explicit


  elevator

  

  
  

  
  
  

  100

  

  
  

  
  70
  
  0.5
  
  [-\w ,/\n\"']{20,200}

  

  
  

  
  

  

  
  

  
  

  
  

  
  

  
  

  

  
  

  
  

  

  

  10
  .,!? 	


  

  

  
  WORD
  
  
  en
  US

  

  

  

  
  
  
  
[^\w-\.]
_
  
  
  
  
  

  -MM-dd'T'HH:mm:ss.SSSZ
  -MM-dd'T'HH:mm:ss,SSSZ
  -MM-dd'T'HH:mm:ss.SSS
  -MM-dd'T'HH:mm:ss,SSS
  -MM-dd'T'HH:mm:ssZ
  -MM-dd'T'HH:mm:ss
  -MM-dd'T'HH:mmZ
  -MM-dd'T'HH:mm
  -MM-dd HH:mm:ss.SSSZ
  -MM-dd HH:mm:ss,SSSZ
  -MM-dd HH:mm:ss.SSS
  -MM-dd HH:mm:ss,SSS
  -MM-dd HH:mm:ssZ
  -MM-dd HH:mm:ss
  -MM-dd HH:mmZ
  -MM-dd HH:mm
  -MM-dd

  
  

  java.lang.String
  text_general
  
*_str
256
  

Re: Solr 7.3.0 loading OpenNLPExtractNamedEntitiesUpdateProcessorFactory

[Ryan Yacyshyn]

Hi,

I found the problem: there was an additional jar file in the /dist folder
that needed to be loaded as well (dist/solr-analysis-extras-7.3.0.jar). I
didn't see this one.

Thanks,
Ryan



On Mon, 9 Apr 2018 at 14:58 Ryan Yacyshyn  wrote:

> Hi Shawn,
>
> I'm pretty sure the paths to load the jars in analysis-extras is correct,
> the jars in /contrib/analysis-extras/lib load fine. I verified this by
> changing the name of solr.OpenNLPTokenizerFactory to 
> solr.OpenNLPTokenizerFactory2
> and saw the new error. Changing it back to solr.OpenNLPTokenizerFactory
> (without the "2") doesn't throw any errors, so I'm assuming these two
> jar files (opennlp-maxent-3.0.3.jar and opennlp-tools-1.8.3.jar) must be
> loading.
>
> I tried swapping the order in which these jars are loaded as well, but no
> luck there.
>
> I have attached my solr.log file after a restart. Also included is my
> solrconfig.xml and managed-schema. The path to my config
> is /Users/ryan/solr-7.3.0/server/solr/nlp/conf and this is where I have the
> OpenNLP bin files (en-ner-person.bin, en-sent.bin, and en-token.bin).
> Configs are derived from the _default configset.
>
> On a mac, and my Java version is:
>
> java version "1.8.0_45"
> Java(TM) SE Runtime Environment (build 1.8.0_45-b14)
> Java HotSpot(TM) 64-Bit Server VM (build 25.45-b02, mixed mode)
>
> Thanks,
> Ryan
>
>
>
> On Sun, 8 Apr 2018 at 21:34 Shawn Heisey  wrote:
>
>> On 4/8/2018 2:36 AM, Ryan Yacyshyn wrote:
>> > I'm running into a small problem loading
>> > the OpenNLPExtractNamedEntitiesUpdateProcessorFactory class, getting an
>> > error saying it's not found. I'm loading all the required jar files,
>> > according to the readme:
>>
>> You've got a  element to load analysis-extras jars, but are you
>> certain it's actually loading anything?
>>
>> Can you share a solr.log file created just after a Solr restart?  Not
>> just a reload -- I'm asking for a restart so the log is more complete.
>> With that, I can see what's happening and then ask more questions that
>> may pinpoint something.
>>
>> Thanks,
>> Shawn
>>
>>

Re: Solr 7.3.0 loading OpenNLPExtractNamedEntitiesUpdateProcessorFactory

[Ryan Yacyshyn]

Hi,

The readme found in '/contrib/analysis-extras' mentions: OpenNLP relies on
lucene-libs/lucene-analyzers-opennlp-X.Y.jar and lib/opennlp-*.jar

Make sure you are loading these as well as the
'solr-analysis-extras-7.3.0.jar' file in the /dist directory.

Regards,
Ryan



On Mon, 16 Apr 2018 at 23:53 Erick Erickson  wrote:

> Have you tried starting Solr with the "-v" option? That will show you
> exactly (in the solr.log file) where every jar is loaded from. Make
> sure you are
> 1> loading the jar at all
> 2> loading the jar you think you are (i.e. not loading an old version
> from somewhere on your classpath).
>
> Copying jar files around is often a source of user error, you can wind
> up loading them from someplace unexpected..
>
> Best,
> Erick
>
> On Mon, Apr 16, 2018 at 2:58 AM, Fatalityap 
> wrote:
> > Hi, can you post final solution how you solve this problem?
> >
> > I have copied solr-analysis-extras-7.3.0.jar to
> /contrib/analysis-extras/lib
> > folder and loaded it like this:
> >
> > / > regex=".*\.jar" /> /
> >
> > but error have not gone...
> >
> > I still see:
> >
> org.apache.solr.common.SolrException:org.apache.solr.common.SolrException:
> > Error loading class
> 'solr.OpenNLPExtractNamedEntitiesUpdateProcessorFactory'
> >
> >
> >
> > --
> > Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
>

Solr 7.3 debug/explain with boost applied

[Ryan Yacyshyn]

Hi all,

When viewing the explain under debug=true in Solr 7.3.0 using
the edismax query parser with a boost, I only see the "boost" part of the
explain. Without applying a boost I see the full explain. Is this the
expected behaviour?

Here's how to check using the techproducts example..

bin/solr -e techproducts

```
http://localhost:8983/solr/techproducts/select?q={!edismax}samsung&qf=name&debug=true
```

returns:

```
"debug": {
"rawquerystring": "{!edismax}samsung",
"querystring": "{!edismax}samsung",
"parsedquery": "+DisjunctionMaxQuery((name:samsung))",
"parsedquery_toString": "+(name:samsung)",
"explain": {
  "SP2514N": "\n2.3669035 = weight(name:samsung in 1)
[SchemaSimilarity], result of:\n  2.3669035 = score(doc=1,freq=1.0 =
termFreq=1.0\n), product of:\n2.6855774 = idf, computed as log(1 +
(docCount - docFreq + 0.5) / (docFreq + 0.5)) from:\n  1.0 = docFreq\n
21.0 = docCount\n0.8813388 = tfNorm, computed as (freq * (k1 + 1))
/ (freq + k1 * (1 - b + b * fieldLength / avgFieldLength)) from:\n  1.0
= termFreq=1.0\n  1.2 = parameter k1\n  0.75 = parameter b\n
7.5238094 = avgFieldLength\n  10.0 = fieldLength\n"
},
"QParser": "ExtendedDismaxQParser",
...
```

If I just add &boost=2 to this, I get this explain back:

```
"debug": {
"rawquerystring": "{!edismax}samsung",
"querystring": "{!edismax}samsung",
"parsedquery": "FunctionScoreQuery(FunctionScoreQuery(+(name:samsung),
scored by boost(const(2",
"parsedquery_toString": "FunctionScoreQuery(+(name:samsung), scored by
boost(const(2)))",
"explain": {
  "SP2514N": "\n4.733807 = product of:\n  1.0 = boost\n  4.733807 =
boost(const(2))\n"
},
"QParser": "ExtendedDismaxQParser",
...
```

Is this normal? I was expecting to see more like the first example, with
the addition of the boost applied.

Thanks,
Ryan

Re: Solr 7.3 debug/explain with boost applied

[Ryan Yacyshyn]

Typically you would use a function query there to manipulate the score
rather than a constant of 2. This doesn't do much but simply multiply all
scores by that value. You can do something like boost=sqrt(popularity) if
you wanted to boost on the popularity field for example. In both cases,
however, the explain is still the same in that it's not showing the full
explain when applying a boost function (or constant number).

Wondering if I'm missing something here?

Ryan







On Wed, 25 Apr 2018 at 04:21 Nawab Zada Asad Iqbal  wrote:

> I didn't know you can add boosts like that (&boost=2 ). Are you boosting on
> a field or document by using that syntax?
>
> On Sun, Apr 22, 2018 at 10:51 PM, Ryan Yacyshyn 
> wrote:
>
> > Hi all,
> >
> > When viewing the explain under debug=true in Solr 7.3.0 using
> > the edismax query parser with a boost, I only see the "boost" part of the
> > explain. Without applying a boost I see the full explain. Is this the
> > expected behaviour?
> >
> > Here's how to check using the techproducts example..
> >
> > bin/solr -e techproducts
> >
> > ```
> > http://localhost:8983/solr/techproducts/select?q={!
> > edismax}samsung&qf=name&debug=true
> > ```
> >
> > returns:
> >
> > ```
> > "debug": {
> > "rawquerystring": "{!edismax}samsung",
> > "querystring": "{!edismax}samsung",
> > "parsedquery": "+DisjunctionMaxQuery((name:samsung))",
> > "parsedquery_toString": "+(name:samsung)",
> > "explain": {
> >   "SP2514N": "\n2.3669035 = weight(name:samsung in 1)
> > [SchemaSimilarity], result of:\n  2.3669035 = score(doc=1,freq=1.0 =
> > termFreq=1.0\n), product of:\n2.6855774 = idf, computed as log(1 +
> > (docCount - docFreq + 0.5) / (docFreq + 0.5)) from:\n  1.0 =
> docFreq\n
> > 21.0 = docCount\n0.8813388 = tfNorm, computed as (freq * (k1 +
> 1))
> > / (freq + k1 * (1 - b + b * fieldLength / avgFieldLength)) from:\n
> 1.0
> > = termFreq=1.0\n  1.2 = parameter k1\n  0.75 = parameter b\n
> > 7.5238094 = avgFieldLength\n  10.0 = fieldLength\n"
> > },
> > "QParser": "ExtendedDismaxQParser",
> > ...
> > ```
> >
> > If I just add &boost=2 to this, I get this explain back:
> >
> > ```
> > "debug": {
> > "rawquerystring": "{!edismax}samsung",
> > "querystring": "{!edismax}samsung",
> > "parsedquery":
> "FunctionScoreQuery(FunctionScoreQuery(+(name:samsung),
> > scored by boost(const(2",
> > "parsedquery_toString": "FunctionScoreQuery(+(name:samsung), scored
> by
> > boost(const(2)))",
> > "explain": {
> >   "SP2514N": "\n4.733807 = product of:\n  1.0 = boost\n  4.733807 =
> > boost(const(2))\n"
> > },
> > "QParser": "ExtendedDismaxQParser",
> > ...
> > ```
> >
> > Is this normal? I was expecting to see more like the first example, with
> > the addition of the boost applied.
> >
> > Thanks,
> > Ryan
> >
>

Block join query parser

[Ryan Yacyshyn]

Hi all,

I'm looking for a way to query nested documents that would return the
parent documents along with its child documents nested under it, but only
the child documents that match the query. The [child] doc transformer comes
close, but it returns all child docs.

I'm looking for something similar to ES' inner hits (
https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-inner-hits.html
).

Is this possible?

Thanks,
Ryan