Re: Solr Slack Workspace

[Ishan Chattopadhyaya]

There is a Slack backed by official IRC support. Please see
https://lucene.472066.n3.nabble.com/Solr-Users-Slack-td4466856.html for
details on how to join it.

On Tue, 19 Jan, 2021, 2:54 pm Charlie Hull, 
wrote:

> Relevance Slack is open to anyone working on search & relevance - #solr is
> only one of the channels, there's lots more! Hope to see you there.
>
> Cheers
>
> Charlie
> https://opensourceconnections.com/slack
>
>
> On 16/01/2021 02:18, matthew sporleder wrote:
> > IRC has kind of died off,
> > https://lucene.apache.org/solr/community.html has a slack mentioned,
> > I'm on https://opensourceconnections.com/slack after taking their solr
> > training class and assume it's mostly open to solr community.
> >
> > On Fri, Jan 15, 2021 at 8:10 PM Justin Sweeney
> >  wrote:
> >> Hi all,
> >>
> >> I did some googling and didn't find anything, but is there a Slack
> >> workspace for Solr? I think this could be useful to expand interaction
> >> within the community of Solr users and connect people solving similar
> >> problems.
> >>
> >> I'd be happy to get this setup if it does not exist already.
> >>
> >> Justin
>
>
> --
> Charlie Hull - Managing Consultant at OpenSource Connections Limited
> 
> Founding member of The Search Network 
> and co-author of Searching the Enterprise
> 
> tel/fax: +44 (0)8700 118334
> mobile: +44 (0)7767 825828
>

RE: Solr Slack Workspace

[ufuk yılmaz]

It’s asking for a searchscale.com email address?

Sent from Mail for Windows 10

From: Ishan Chattopadhyaya
Sent: 26 January 2021 13:33
To: solr-user
Subject: Re: Solr Slack Workspace

There is a Slack backed by official IRC support. Please see
https://lucene.472066.n3.nabble.com/Solr-Users-Slack-td4466856.html for
details on how to join it.

On Tue, 19 Jan, 2021, 2:54 pm Charlie Hull, 
wrote:

> Relevance Slack is open to anyone working on search & relevance - #solr is
> only one of the channels, there's lots more! Hope to see you there.
>
> Cheers
>
> Charlie
> https://opensourceconnections.com/slack
>
>
> On 16/01/2021 02:18, matthew sporleder wrote:
> > IRC has kind of died off,
> > https://lucene.apache.org/solr/community.html has a slack mentioned,
> > I'm on https://opensourceconnections.com/slack after taking their solr
> > training class and assume it's mostly open to solr community.
> >
> > On Fri, Jan 15, 2021 at 8:10 PM Justin Sweeney
> >  wrote:
> >> Hi all,
> >>
> >> I did some googling and didn't find anything, but is there a Slack
> >> workspace for Solr? I think this could be useful to expand interaction
> >> within the community of Solr users and connect people solving similar
> >> problems.
> >>
> >> I'd be happy to get this setup if it does not exist already.
> >>
> >> Justin
>
>
> --
> Charlie Hull - Managing Consultant at OpenSource Connections Limited
> 
> Founding member of The Search Network 
> and co-author of Searching the Enterprise
> 
> tel/fax: +44 (0)8700 118334
> mobile: +44 (0)7767 825828
>

Re: Solr Slack Workspace

[Ishan Chattopadhyaya]

We should update the official documentation for community page to include
this Slack org. If someone would like to volunteer to submit a patch
against lucene-site repo, would be great.

On Tue, 26 Jan, 2021, 4:02 pm Ishan Chattopadhyaya, <
ichattopadhy...@gmail.com> wrote:

> There is a Slack backed by official IRC support. Please see
> https://lucene.472066.n3.nabble.com/Solr-Users-Slack-td4466856.html for
> details on how to join it.
>
> On Tue, 19 Jan, 2021, 2:54 pm Charlie Hull, <
> ch...@opensourceconnections.com> wrote:
>
>> Relevance Slack is open to anyone working on search & relevance - #solr
>> is only one of the channels, there's lots more! Hope to see you there.
>>
>> Cheers
>>
>> Charlie
>> https://opensourceconnections.com/slack
>>
>>
>> On 16/01/2021 02:18, matthew sporleder wrote:
>> > IRC has kind of died off,
>> > https://lucene.apache.org/solr/community.html has a slack mentioned,
>> > I'm on https://opensourceconnections.com/slack after taking their solr
>> > training class and assume it's mostly open to solr community.
>> >
>> > On Fri, Jan 15, 2021 at 8:10 PM Justin Sweeney
>> >  wrote:
>> >> Hi all,
>> >>
>> >> I did some googling and didn't find anything, but is there a Slack
>> >> workspace for Solr? I think this could be useful to expand interaction
>> >> within the community of Solr users and connect people solving similar
>> >> problems.
>> >>
>> >> I'd be happy to get this setup if it does not exist already.
>> >>
>> >> Justin
>>
>>
>> --
>> Charlie Hull - Managing Consultant at OpenSource Connections Limited
>> 
>> Founding member of The Search Network 
>> and co-author of Searching the Enterprise
>> 
>> tel/fax: +44 (0)8700 118334
>> mobile: +44 (0)7767 825828
>>
>

Re: Solr Slack Workspace

[Ishan Chattopadhyaya]

Thanks ufuk. I'll take a look.

On Tue, 26 Jan, 2021, 4:05 pm ufuk yılmaz, 
wrote:

> It’s asking for a searchscale.com email address?
>
> Sent from Mail for Windows 10
>
> From: Ishan Chattopadhyaya
> Sent: 26 January 2021 13:33
> To: solr-user
> Subject: Re: Solr Slack Workspace
>
> There is a Slack backed by official IRC support. Please see
> https://lucene.472066.n3.nabble.com/Solr-Users-Slack-td4466856.html for
> details on how to join it.
>
> On Tue, 19 Jan, 2021, 2:54 pm Charlie Hull, <
> ch...@opensourceconnections.com>
> wrote:
>
> > Relevance Slack is open to anyone working on search & relevance - #solr
> is
> > only one of the channels, there's lots more! Hope to see you there.
> >
> > Cheers
> >
> > Charlie
> > https://opensourceconnections.com/slack
> >
> >
> > On 16/01/2021 02:18, matthew sporleder wrote:
> > > IRC has kind of died off,
> > > https://lucene.apache.org/solr/community.html has a slack mentioned,
> > > I'm on https://opensourceconnections.com/slack after taking their solr
> > > training class and assume it's mostly open to solr community.
> > >
> > > On Fri, Jan 15, 2021 at 8:10 PM Justin Sweeney
> > >  wrote:
> > >> Hi all,
> > >>
> > >> I did some googling and didn't find anything, but is there a Slack
> > >> workspace for Solr? I think this could be useful to expand interaction
> > >> within the community of Solr users and connect people solving similar
> > >> problems.
> > >>
> > >> I'd be happy to get this setup if it does not exist already.
> > >>
> > >> Justin
> >
> >
> > --
> > Charlie Hull - Managing Consultant at OpenSource Connections Limited
> > 
> > Founding member of The Search Network 
> > and co-author of Searching the Enterprise
> > 
> > tel/fax: +44 (0)8700 118334
> > mobile: +44 (0)7767 825828
> >
>
>

Re: Error Adding a Replica to SOLR Cloud 8.2.0

[Joe Lerner]

We finally got this fixed by temporarily disabling any updates to the SOLR
index. 



--
Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Steps to write a custom StreamingExpression

[ufuk yılmaz]

Should I create a java project with a dependency on solrj, or solr core ?,
then implement the Expressible interface
then build my project as a jar and put it into each node of SolrColud’s 
classpath?

Or should I take a completely different route?

Many thanks
~ufuk

Sent from Mail for Windows 10

Re: Multi-select faceting for nested documents

[Lance Snell]

Thanks for the response.  I'll try and give a little more detail.
Hopefully my terminology is correct, I'm still new to solr.

My current schema has a parent document, two child documents(siblings), and
a grandchild document.  I am using the JSON API .

Product -> Item-> Price
   |
   \/
ProductCategory

I created my schema to represent a multi-store, and multi-customer setup.

I am looking for multi-select facets from each document.
Product document - General product information.  *Same for all customers*
Item document - Size/Color/etc.  Product variations.  *Same for all
customers*
Price document - Customer pricing.  *Unique per customer*
ProductCategory- Ids for linking Product to Store and Catalogs.  *Unique
per customer*

I was attempting to traverse from the child documents upwards, filtering on
( ProductCategory ), (Price/Item), and then finally combine the products of
both to get (Product).
Originally I was using '_nest_path_' for my blockMasks.  However I ended up
adding a 'path' field for each document, thinking this would give me more
control?? (ex. '1.product.item.price', '1.product.item', '1.product',
'2.product.item.price' , '2..product.item', etc. )



Below is my current query.

Currently when I remove the nested {!filters tag=ITEM...}, I get the
correct number of ProductCategory documents.  When I remove the nested
{!filters tag=PRODUCT_CATEGORY...},I get the correct number of Item
documents.
When I combine them both, I get *0* results.

Is this because the two nested filters ITEM and PRODUCT_CATEGORY are not
mutually exclusive?  If so could you point or provide an example.

OR

Is this because the ITEM and PRODUCT_CATEGORY filters are not filtering on
the Product documents yet, and they are both children of the Product
document?

OR

Am I not even close??


I thought I had seen some well written examples or videos of this in the
past, but I can't seem to find those examples anymore.   Any help/direction
is greatly appreciated.


{
"params": {
"omitHeader": "true",
"defType": "edismax",
"df": "pdocname_txt_en_1",
"qf":
"pdocname_txt_en_1^2.0 description_txt_en_1^1.5
longDescription_txt_en_1^1.5 custItemNum_txt_en_split_tight_1^1.5"
,
"pf":
"pdocname_txt_en_1^3.0 description_txt_en_1^2.0
longDescription_txt_en_1^2.0 custItemNum_txt_en_split_tight_1^3"
,
"ps": "2",
"qs": "2",
"LIMIT": 0,
"prdoc_Q2":
"prefixes:ASB00434166111481 prefixes:ASB0043416611148All",
//Query entry point for price documents
"pcdoc_Q2":
"categoryPrefixes:ASB00434166111481
categoryPrefixes:ASB0043416611148All"
,   //Query entry point for product category
"BRAND_FILTER":"sku:SMB505",
"COLOR_FILTER":"",
"SIZE_FILTER":"",
"PRICE_FILTER":"fprice:[29 TO 30]",
"CATEGORY_FILTER":"categoryIds:2921",
"STORECATALOG_FILTER":"",
"productFQ":"{!parent filters=$pidocFQ which='path:2.product.item'}"
,
"productFilters":
"+{!parent which='path:2.product.item' v=$prdoc_Q2} +{!child
of='path:1.product.productCategory'}{!parent
which='path:1.product.productCategory' v=$pcdoc_Q2}"
,



"priceFilters":
"prefixes:(ASB00434166111481 ASB0043416611148All)",
//product filters
"productCategoryFilters":
"categoryPrefixes:(ASB00434166111481 ASB0043416611148All)",
//productCategory filters
"itemFilters":"docType:pidoc"
,   //item(skudocument) filters
"itemFQ":
"+(${priceFilters}) +{!child of='path:2.product.item'}{!q.op=AND
v=$itemFilters}"
, //Item filter query
"productCategoryFQ":"+(${productCategoryFilters})",
//ProductCategory filter query
"baseQ":
"categoryPrefixes:(ASB00434166111481 ASB0043416611148All)
prefixes:(ASB00434166111481 ASB0043416611148All) docType:pidoc
docType:pdoc -docType:catdoc -docType:stdoc -docType:cupsd"
,

"groupingFilter1":
"{!parent filters=$itemFQ which='path:2.product.item'}",
//Filter for priceDocument -> itemDocument
"groupingFilter2":"{!q.op=AND v=$productCategoryFQ}",
//Filter for productCategoryDocument -> productDocument
"test":
"{!filters tag=ITEM param=$groupingFilter1} {!filters
tag=PRODUCT_CATEGORY param=$groupingFilter2}"
,
"topQuery_product":""

},
"fields":
"*, [child fl=$returnFields limit=-1
childFilter='/productItems/{!filters v=$child_FQ2}']"
,
"filter": [

// "${prdoc_Q2} {!parent which='*:* -_nest_path_:/*' v=$prdoc_Q2}
{!parent which='*:* -_nest_path_:/productItems/*' v=$prdoc_Q2}
categoryPrefixes:(ASB00434166111481 ASB0043416611148All)"

"_query_:(${prdoc_Q2} ${pcdoc_Q2} docType:pcdoc docType:prdoc
docType:pidoc docType:pdoc)"
,
"{!filters tag=MID param=$test v=$baseQ}"
],
"sort":
"{!parent which='*:* -_nest_path_:*' score=max v='+docType:prdoc
+{!func}fprice'} asc"
,
"offset": 0,
"limit": "${LIMIT}",
"facet": {
"testin

Nested documents (parent,child,grandchild), multi-select facets

[Lance Snell]

 Hey all,

I am having trouble finding current examples of multi-select faceting for
nested documents.  Specifically ones with *multiple *levels of nested
documents.

Any help/direction would be appreciated.Solr. 8.6

Hopefully my terminology is correct, I'm still new to solr.

My current schema has a parent document, two child documents(siblings), and
a grandchild document.  I am using the JSON API .

Product -> Item-> Price
   |
   \/
ProductCategory

I created my schema to represent a multi-store, and multi-customer setup.

I am looking for multi-select facets from *each* document.
Product document - General product information.  *Same for all customers*
Item document - Size/Color/etc.  Product variations.  *Same for all
customers*
Price document - Customer pricing.  *Unique per customer*
ProductCategory- Ids for linking Product to Store and Catalogs.  *Unique
per customer*

I was attempting to traverse from the child documents upwards, filtering on
( ProductCategory ), (Price/Item), and then finally combine the result of
both to get (Product).
Originally I was using '_nest_path_' for my blockMasks.  However I ended up
adding a 'path' field for each document, thinking this would give me more
control?? (ex. '1.product.item.price', '1.product.item', '1.product',
'2.product.item.price' , '2..product.item', etc. )


Below is my current query.

Currently when I remove the nested {!filters tag=ITEM...}, I get the
correct number of ProductCategory documents.  When I remove the nested
{!filters tag=PRODUCT_CATEGORY...},I get the correct number of Item
documents.
When I combine them both, I get *0* results.

Is this because the two nested filters ITEM and PRODUCT_CATEGORY are not
mutually exclusive?  If so could you point or provide an example.

OR

Is this because the ITEM and PRODUCT_CATEGORY filters are not filtering on
the Product documents yet, and they are both children of the Product
document?

OR

Am I not even close??


I thought I had seen some well written examples or videos of this in the
past, but I can't seem to find those examples anymore.   Any
help/direction/links would be greatly appreciated.


{
"params": {
"omitHeader": "true",
"defType": "edismax",
"df": "pdocname_txt_en_1",
"qf":
"pdocname_txt_en_1^2.0 description_txt_en_1^1.5
longDescription_txt_en_1^1.5 custItemNum_txt_en_split_tight_1^1.5"
,
"pf":
"pdocname_txt_en_1^3.0 description_txt_en_1^2.0
longDescription_txt_en_1^2.0 custItemNum_txt_en_split_tight_1^3"
,
"ps": "2",
"qs": "2",
"LIMIT": 0,
"prdoc_Q2":
"prefixes:ASB00434166111481 prefixes:ASB0043416611148All",
//Query entry point for price documents
"pcdoc_Q2":
"categoryPrefixes:ASB00434166111481
categoryPrefixes:ASB0043416611148All"
,   //Query entry point for product category
"BRAND_FILTER":"sku:SMB505",
"COLOR_FILTER":"",
"SIZE_FILTER":"",
"PRICE_FILTER":"fprice:[29 TO 30]",
"CATEGORY_FILTER":"categoryIds:2921",
"STORECATALOG_FILTER":"",
"productFQ":"{!parent filters=$pidocFQ which='path:2.product.item'}"
,
"productFilters":
"+{!parent which='path:2.product.item' v=$prdoc_Q2} +{!child
of='path:1.product.productCategory'}{!parent
which='path:1.product.productCategory' v=$pcdoc_Q2}"
,



"priceFilters":
"prefixes:(ASB00434166111481 ASB0043416611148All)",
//product filters
"productCategoryFilters":
"categoryPrefixes:(ASB00434166111481 ASB0043416611148All)",
//productCategory filters
"itemFilters":"docType:pidoc"
,   //item(skudocument) filters
"itemFQ":
"+(${priceFilters}) +{!child of='path:2.product.item'}{!q.op=AND
v=$itemFilters}"
, //Item filter query
"productCategoryFQ":"+(${productCategoryFilters})",
//ProductCategory filter query
"baseQ":
"categoryPrefixes:(ASB00434166111481 ASB0043416611148All)
prefixes:(ASB00434166111481 ASB0043416611148All) docType:pidoc
docType:pdoc -docType:catdoc -docType:stdoc -docType:cupsd"
,

"groupingFilter1":
"{!parent filters=$itemFQ which='path:2.product.item'}",
//Filter for priceDocument -> itemDocument
"groupingFilter2":"{!q.op=AND v=$productCategoryFQ}",
//Filter for productCategoryDocument -> productDocument
"test":
"{!filters tag=ITEM param=$groupingFilter1} {!filters
tag=PRODUCT_CATEGORY param=$groupingFilter2}"
,
"topQuery_product":""

},
"fields":
"*, [child fl=$returnFields limit=-1
childFilter='/productItems/{!filters v=$child_FQ2}']"
,
"filter": [

// "${prdoc_Q2} {!parent which='*:* -_nest_path_:/*' v=$prdoc_Q2}
{!parent which='*:* -_nest_path_:/productItems/*' v=$prdoc_Q2}
categoryPrefixes:(ASB00434166111481 ASB0043416611148All)"

"_query_:(${prdoc_Q2} ${pcdoc_Q2} docType:pcdoc docType:prdoc
docType:pidoc docType:pdoc)"
,
"{!filters tag=MID param=$test v=$baseQ}"
],
"sort"

Re: Solr Slack Workspace

[Houston Putman]

There is https://solr-dev.slack.com

It's not really used, but it's there and we can open it up for people to
join and start using.

On Tue, Jan 26, 2021 at 5:38 AM Ishan Chattopadhyaya <
ichattopadhy...@gmail.com> wrote:

> Thanks ufuk. I'll take a look.
>
> On Tue, 26 Jan, 2021, 4:05 pm ufuk yılmaz, 
> wrote:
>
> > It’s asking for a searchscale.com email address?
> >
> > Sent from Mail for Windows 10
> >
> > From: Ishan Chattopadhyaya
> > Sent: 26 January 2021 13:33
> > To: solr-user
> > Subject: Re: Solr Slack Workspace
> >
> > There is a Slack backed by official IRC support. Please see
> > https://lucene.472066.n3.nabble.com/Solr-Users-Slack-td4466856.html for
> > details on how to join it.
> >
> > On Tue, 19 Jan, 2021, 2:54 pm Charlie Hull, <
> > ch...@opensourceconnections.com>
> > wrote:
> >
> > > Relevance Slack is open to anyone working on search & relevance - #solr
> > is
> > > only one of the channels, there's lots more! Hope to see you there.
> > >
> > > Cheers
> > >
> > > Charlie
> > > https://opensourceconnections.com/slack
> > >
> > >
> > > On 16/01/2021 02:18, matthew sporleder wrote:
> > > > IRC has kind of died off,
> > > > https://lucene.apache.org/solr/community.html has a slack mentioned,
> > > > I'm on https://opensourceconnections.com/slack after taking their
> solr
> > > > training class and assume it's mostly open to solr community.
> > > >
> > > > On Fri, Jan 15, 2021 at 8:10 PM Justin Sweeney
> > > >  wrote:
> > > >> Hi all,
> > > >>
> > > >> I did some googling and didn't find anything, but is there a Slack
> > > >> workspace for Solr? I think this could be useful to expand
> interaction
> > > >> within the community of Solr users and connect people solving
> similar
> > > >> problems.
> > > >>
> > > >> I'd be happy to get this setup if it does not exist already.
> > > >>
> > > >> Justin
> > >
> > >
> > > --
> > > Charlie Hull - Managing Consultant at OpenSource Connections Limited
> > > 
> > > Founding member of The Search Network 
> > > and co-author of Searching the Enterprise
> > > 
> > > tel/fax: +44 (0)8700 118334
> > > mobile: +44 (0)7767 825828
> > >
> >
> >
>

Re: Steps to write a custom StreamingExpression

[Joel Bernstein]

I believe that would be the best path.

Joel Bernstein
http://joelsolr.blogspot.com/


On Tue, Jan 26, 2021 at 7:50 AM ufuk yılmaz 
wrote:

> Should I create a java project with a dependency on solrj, or solr core ?,
> then implement the Expressible interface
> then build my project as a jar and put it into each node of SolrColud’s
> classpath?
>
> Or should I take a completely different route?
>
> Many thanks
> ~ufuk
>
> Sent from Mail for Windows 10
>
>

Re: Streaming expressions, what is the effect of collection name in the request url

[Joel Bernstein]

The URL path should be for one collection. This will be where the
collection is compiled and run. It has no effect on what is actually being
searched. That is specified in the expression themselves.


Joel Bernstein
http://joelsolr.blogspot.com/


On Wed, Jan 20, 2021 at 1:34 PM ufuk yılmaz 
wrote:

> Do collection names in request url affect how the query works in any way?
>
> A streaming expression is sent to http://mySolrHost/solr/col1,col2/stream
> (notice multiple collections in url)
>
> Col1 has 2 shards, each have 3 replicas.
> * Shard1 has replicas on nodes A, B, C
> * Shard2 has replicas on D,E,F
>
> Col2 has 2 shards, each have 3 replicas. Its shards have the same
> configuration as Col1.
>
>
> Lets say we have a simple search expression:
> search(
> "colA,colB",
> q="*:*",
> qt="/export",
> fl="fl1,fl2",
> sort="id asc"
> )
>
> Collection names in search expression denotes which collections should be
> searched, so we can’t change them. But what would change if we sent the
> query to
> http://mySolrHost/solr/someOtherCollection/stream
>
> and someOtherCollection has 1 shard and 6 replicas in nodes A,B,C,D,E,F ?
>
> I read about worker collections a bit, but as long as I don’t explicitly
> use parallel streams, what is the difference?
>
>
>
> Sent from Mail for Windows 10
>
>

Re: Parallel streaming expression java.lang.IndexOutOfBoundsException

[Joel Bernstein]

Yes, this is not ideal. It means that the worker collection needs to have N
shards rather than N replicas. Changing this should not be difficult if
you'd like to provide a patch.


Joel Bernstein
http://joelsolr.blogspot.com/


On Thu, Jan 21, 2021 at 8:00 AM ufuk yılmaz 
wrote:

> Looked at the source code of the parallel stream and it seems I need equal
> number of SHARDS and workers count parameter. I thought I needed as many
> replicas, it was shards.
> Maybe helps someone.
>
> Sent from Mail for Windows 10
>
> From: ufuk yılmaz
> Sent: 21 January 2021 11:16
> To: solr-user@lucene.apache.org
> Subject: Parallel streaming expression java.lang.IndexOutOfBoundsException
>
> Hello all,
>
>
> https://lucene.apache.org/solr/guide/8_4/stream-decorator-reference.html#parallel
>
> I’m sending the same query in the docs, (just collection names changed) to
> my Solr but always getting the exception:
>
> {
>   "result-set":{
> "docs":[{
> "EXCEPTION":"java.lang.IndexOutOfBoundsException: Index 1 out of
> bounds for length 1",
> "EOF":true,
> "RESPONSE_TIME":93}]}}
>
> My query:
>
> null(
> parallel(
> WorkerCollection,
> rollup(
> search(
> colA,
> q="username: c*",
> qt="/export",
> fl="timestamp,user.id",
> sort="user.id asc",
> partitionKeys="user.id"
> ),
> over="user.id",count(*)
> ),
> workers="4",
> sort="timestamp asc"
> )
> )
>
> WorkerCollection has 1 shard and 4 replicas on 4 different machines. I
> double-triple checked for a silly syntax mistake but there’s none that I
> can see. I tried doing this a few months ago with no success, getting the
> same exception again. colA is on different machines from WorkerCollection.
> user.id is a string field, timestamp is long.
>
> What am I missing?
>
>
> Sent from Mail for Windows 10
>
>
>

RE: Streaming expressions, what is the effect of collection name inthe request url

[ufuk yılmaz]

Does it have any ill side effects when url has multiple collections? Like can 
it cause expression to compile and run on many nodes at once?

Our scripts generate the url, when we are doing a regular searching on multiple 
collections, that url is necessary but if it’s ill for streaming ones, I should 
change them.

Many thanks  Joel

PS: how are tomato thiefs doing? 😊

Sent from Mail for Windows 10

From: Joel Bernstein
Sent: 26 January 2021 21:14
To: solr-user@lucene.apache.org
Subject: Re: Streaming expressions, what is the effect of collection name inthe 
request url

The URL path should be for one collection. This will be where the
collection is compiled and run. It has no effect on what is actually being
searched. That is specified in the expression themselves.


Joel Bernstein
http://joelsolr.blogspot.com/


On Wed, Jan 20, 2021 at 1:34 PM ufuk yılmaz 
wrote:

> Do collection names in request url affect how the query works in any way?
>
> A streaming expression is sent to http://mySolrHost/solr/col1,col2/stream
> (notice multiple collections in url)
>
> Col1 has 2 shards, each have 3 replicas.
> * Shard1 has replicas on nodes A, B, C
> * Shard2 has replicas on D,E,F
>
> Col2 has 2 shards, each have 3 replicas. Its shards have the same
> configuration as Col1.
>
>
> Lets say we have a simple search expression:
> search(
> "colA,colB",
> q="*:*",
> qt="/export",
> fl="fl1,fl2",
> sort="id asc"
> )
>
> Collection names in search expression denotes which collections should be
> searched, so we can’t change them. But what would change if we sent the
> query to
> http://mySolrHost/solr/someOtherCollection/stream
>
> and someOtherCollection has 1 shard and 6 replicas in nodes A,B,C,D,E,F ?
>
> I read about worker collections a bit, but as long as I don’t explicitly
> use parallel streams, what is the difference?
>
>
>
> Sent from Mail for Windows 10
>
>

Re: NullPointerException in Graph Traversal nodes streaming expression

[Joel Bernstein]

How are you constructing the Stream with classes or using a Streaming
Expression?

In either case can you post either the code or expression?

Are there more errors in the logs? The place where this NPE is occurring is
that an underlying stream is null, which leads me to believe there would be
some exceptions before this, possibly on a different server if this has
multiple servers involved.

Joel Bernstein
http://joelsolr.blogspot.com/


On Thu, Jan 21, 2021 at 5:46 PM Mike Drob  wrote:

> Can you provide a sample expression that would be able to reproduce this?
> Are you able to try a newer version by chance - I know we've fixed a few
> NPEs recently, maybe https://issues.apache.org/jira/browse/SOLR-14700
>
> On Thu, Jan 21, 2021 at 4:13 PM ufuk yılmaz 
> wrote:
>
> > Solr version 8.4. I’m getting an unexplanetory NullPointerException when
> > executing a simple 2 level nodes stream, do you have any idea what may
> > cause this?
> >
> > I tried setting /stream?partialResults=true&shards.tolerant=true and
> > shards.tolerant=true in nodes expressions, with no luck. I also tried
> > reading source of GatherNodesStream in branch 8_4, but couldn’t
> understand
> > it. Here is a beautiful stack trace:
> >
> > solr| 2021-01-21 22:00:12.726 ERROR (qtp832292933-25149)
> > [c:WorkerCollection s:shard1 r:core_node10
> > x:WorkerCollection_shard1_replica_n9] o.a.s.c.s.i.s.ExceptionStream
> > java.lang.RuntimeException: java.util.concurrent.ExecutionException:
> > java.lang.RuntimeException: java.lang.NullPointerException
> > solr|   at
> >
> org.apache.solr.client.solrj.io.graph.GatherNodesStream.read(GatherNodesStream.java:607)
> > solr|   at
> >
> org.apache.solr.client.solrj.io.stream.ExceptionStream.read(ExceptionStream.java:71)
> > solr|   at
> >
> org.apache.solr.handler.StreamHandler$TimerStream.read(StreamHandler.java:454)
> > solr|   at
> >
> org.apache.solr.client.solrj.io.stream.TupleStream.lambda$writeMap$0(TupleStream.java:84)
> > solr|   at
> >
> org.apache.solr.common.util.JsonTextWriter.writeIterator(JsonTextWriter.java:141)
> > solr|   at
> > org.apache.solr.common.util.TextWriter.writeVal(TextWriter.java:67)
> > solr|   at
> >
> org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:152)
> > solr|   at
> > org.apache.solr.common.util.JsonTextWriter$2.put(JsonTextWriter.java:176)
> > solr|   at
> >
> org.apache.solr.client.solrj.io.stream.TupleStream.writeMap(TupleStream.java:81)
> > solr|   at
> >
> org.apache.solr.common.util.JsonTextWriter.writeMap(JsonTextWriter.java:164)
> > solr|   at
> > org.apache.solr.common.util.TextWriter.writeVal(TextWriter.java:69)
> > solr|   at
> >
> org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:152)
> > solr|   at
> >
> org.apache.solr.common.util.JsonTextWriter.writeNamedListAsMapWithDups(JsonTextWriter.java:386)
> > solr|   at
> >
> org.apache.solr.common.util.JsonTextWriter.writeNamedList(JsonTextWriter.java:292)
> > solr|   at
> > org.apache.solr.response.JSONWriter.writeResponse(JSONWriter.java:73)
> > solr|   at
> >
> org.apache.solr.response.JSONResponseWriter.write(JSONResponseWriter.java:66)
> > solr|   at
> >
> org.apache.solr.response.QueryResponseWriterUtil.writeQueryResponse(QueryResponseWriterUtil.java:65)
> > solr|   at
> > org.apache.solr.servlet.HttpSolrCall.writeResponse(HttpSolrCall.java:892)
> > solr|   at
> > org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:594)
> > solr|   at
> >
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:419)
> > solr|   at
> >
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:351)
> > solr|   at
> >
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
> > solr|   at
> >
> org.eclipse.jetty.servlets.CrossOriginFilter.handle(CrossOriginFilter.java:311)
> > solr|   at
> >
> org.eclipse.jetty.servlets.CrossOriginFilter.doFilter(CrossOriginFilter.java:265)
> > solr|   at
> >
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1602)
> > solr|   at
> >
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)
> > solr|   at
> >
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
> > solr|   at
> >
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
> > solr|   at
> >
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
> > solr|   at
> >
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
> > solr|   at
> >
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1711)
> > solr|   at
> >
> org.eclipse.jetty.server.handler.ScopedHandl

Re: Streaming expressions, what is the effect of collection name inthe request url

[Joel Bernstein]

I have never tried this and didn't even know that you could have multiple
collections in the URL. So, I'm really not sure what the behavior will be.


Joel Bernstein
http://joelsolr.blogspot.com/


On Tue, Jan 26, 2021 at 1:19 PM ufuk yılmaz 
wrote:

> Does it have any ill side effects when url has multiple collections? Like
> can it cause expression to compile and run on many nodes at once?
>
> Our scripts generate the url, when we are doing a regular searching on
> multiple collections, that url is necessary but if it’s ill for streaming
> ones, I should change them.
>
> Many thanks  Joel
>
> PS: how are tomato thiefs doing? 😊
>
> Sent from Mail for Windows 10
>
> From: Joel Bernstein
> Sent: 26 January 2021 21:14
> To: solr-user@lucene.apache.org
> Subject: Re: Streaming expressions, what is the effect of collection name
> inthe request url
>
> The URL path should be for one collection. This will be where the
> collection is compiled and run. It has no effect on what is actually being
> searched. That is specified in the expression themselves.
>
>
> Joel Bernstein
> http://joelsolr.blogspot.com/
>
>
> On Wed, Jan 20, 2021 at 1:34 PM ufuk yılmaz 
> wrote:
>
> > Do collection names in request url affect how the query works in any way?
> >
> > A streaming expression is sent to
> http://mySolrHost/solr/col1,col2/stream
> > (notice multiple collections in url)
> >
> > Col1 has 2 shards, each have 3 replicas.
> > * Shard1 has replicas on nodes A, B, C
> > * Shard2 has replicas on D,E,F
> >
> > Col2 has 2 shards, each have 3 replicas. Its shards have the same
> > configuration as Col1.
> >
> >
> > Lets say we have a simple search expression:
> > search(
> > "colA,colB",
> > q="*:*",
> > qt="/export",
> > fl="fl1,fl2",
> > sort="id asc"
> > )
> >
> > Collection names in search expression denotes which collections should be
> > searched, so we can’t change them. But what would change if we sent the
> > query to
> > http://mySolrHost/solr/someOtherCollection/stream
> >
> > and someOtherCollection has 1 shard and 6 replicas in nodes A,B,C,D,E,F ?
> >
> > I read about worker collections a bit, but as long as I don’t explicitly
> > use parallel streams, what is the difference?
> >
> >
> >
> > Sent from Mail for Windows 10
> >
> >
>
>

how to use a compass lucene generated index with solr

[Guglielmo Fanini]

With (the latest) lucene 8.7 is it possible to open very old .cfs compound 
index file of lucene 2.2 with "Luke" ? or alternatively could it be possibile 
to generate the .idx file for Luke from the .cfs ?
the .cfs was generated by compass on top of lucene 2.2, not by lucene directly
Is it possible to use a compass generated index containing 
_b.cfs
segments.gen
segments_d
with solr ?

Performance issue with Solr 8.6.1 Unified Highlighter does not occur on Solr 6.

[Kerwin]

Hi,

While upgrading to Solr 8 from 6 the Unified highlighter begins to have
performance issues going from approximately 100ms to more than 4 seconds
with 76 fields in the hl.q  and hl.fl parameters. So I played with
different options and found that the hl.q parameter needs to have any one
field for the performance issue to vanish. I do not know why this would be
so. Could you check if this is a bug or something else? This is not the
case if I use the original highlighter which has same performance on Solr 6
and Solr 8 of ~ 1.5 seconds. The highlighting payload is also mostly same
in all the cases.

Prior Solr 8 configuration with bad performance of > 4sec
{!edismax qf="field1 field2 ..field76" v=$qq}
field1 field2 ..field76

Solr 8 configuration with original Solr 6 performance of ~ 100 ms
{!edismax qf="field1" v=$qq}
field1 field2 ..field76

Other highlighting parameters
true
unified
200
WORD
en
10

If I remove the hl.q parameter altogether, the performance time shoots up
to 6-7 seconds, since our user query is quite large with more fields and is
more complicated, I suspect.