Welcome Michael Gibney to the Solr PMC

[Houston Putman]

I am pleased to announce that Michael Gibney has accepted the PMC's
invitation to join.

Congratulations and welcome, Michael!

Re: Welcome Michael Gibney to the Solr PMC

[David Smiley]

Welcome Michael!

~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley


On Wed, Dec 21, 2022 at 12:11 PM Houston Putman  wrote:

> I am pleased to announce that Michael Gibney has accepted the PMC's
> invitation to join.
>
> Congratulations and welcome, Michael!
>

Trying to understand cross-collection-join routing/hashing choices and behavior

[Zack Kendall]

I'm trying to understand the cross-collection JOIN

documentation,
behavior, choices, and viability.

*# Terminology language choice*

"""routerField - If the documents are routed to shards using the
CompositeID router by the join field, then that field name should be
specified in the configuration here. This will allow the parser to optimize
the resulting HashRange query."""

"""routed - If true, the cross collection join query will use each shard’s
hash range to determine the set of join keys to retrieve for that shard.
This parameter improves the performance of the cross-collection join, but
it depends on the local collection being routed by the to field. If this
parameter is not specified, the cross collection join query will try to
determine the correct value automatically."""

*Question 1*: Why overload terminology like "route" when these parameters
do NOT route AFAICT. Based on my reading of the code all they do is add a
hash_range fq parameter to the remote join query request. Filtering results
is not routing, so this fosters confusion. Is there reasoning behind this
or just happenstance?

*# Implied vs Actual behavior*

My reading of the code base is this: the hash_range parameter is always
populated with the "fromField" value. The routerField is only used to check
against the "toField" for equality to enable the hash_range parameter
usage, this is only done as a fall back if "routed" is not set.

It's a little strange to me that "routerField" is not used as a router
field, or even as a hash field. It is only used as a flag for "if a query
is joining to THIS field then use hash_range filter on the fromField" (or
at least that's how I read the code).

*Question 2:* Is my reading of the code correct? Can we try to update the
documentation to be more explicit about this?


*# Routing *

*Question 3:* Is there a reason why actual routing was not used? I'm not
familiar with the Solr code base, but it seems like it'd be nicer to
instead use existing routing behavior in this context instead of querying
all and filtering results. This seems like it would need 2 things: First,
the _route_ value from the current "local" request, and second, either the
local client (like how solrj does) or the remote "/export" handler would
need to recognize and handle this parameter. Is that obviously doable or
not doable? Trying to understand why that approach wasn't taken originally.


*# Hashing*

Here is the behavior touted in the docs for HashRangeQueryParser

.
"""In the cross collection join case, the hash range query parser is used
to ensure that each shard only gets the set of join keys that would end up
on that shard. This query parser uses the MurmurHash3_x86_32. This is the
same as the default hashing for the default composite ID router in Solr."""

The documentation mentions "CompositeID router", which we know is based on
prefixes (split on "!") being hashed and routed with the first/top 16 bits
of info (with the later 16 bits provided by the rest of the doc "id" on
inserts).

The CrossCollectionJoinQuery uses 16 bits from the current/local shard
range, which seems fine and good. However, the HashRangeQuery appears to hash
the entire field
.
So I'm struggling to understand how this would work, especially since the
join field and the "route" field are sourced from the same value. Either
the join field is a compositeId in which case the HashRangeQuery code
appears to be invalid, as it would not hash "A!B" the same as the actual
router would hash "A", or the join field is not a compositeId in which case
for it to work it would have to be the exact value as the actual
compositeId prefix field something like this doc: {"id":"A!B",
"myJoinField": "A"}. (Or maybe using "router.field=myJoinField" works
without the compositeId/"!" format?). And if the join field is not a
compositeId, then the only thing you could join on is the broad category
tenant/product/etc that is used as the compositeId prefix, which would
severely limit the use-case of the plugin, preventing joins on something
more akin to record-ids/foreign-keys, and only allowing you to narrow down
the results by what you know ahead of time to cram into the "v=" query
field.

*Question 4:* Not a specific question so much as "am I onto something here
or am I missing something and off base?"

Actually reading through the test code, now I see that my hypothesized "it
could only work if router key and join field are the same value" is
actually what is tested. The data is set-up

Re: Welcome Michael Gibney to the Solr PMC

[Noble Paul]

Welcome Michael

On Thu, Dec 22, 2022 at 5:28 AM David Smiley  wrote:

> Welcome Michael!
>
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> http://www.linkedin.com/in/davidwsmiley
>
>
> On Wed, Dec 21, 2022 at 12:11 PM Houston Putman 
> wrote:
>
> > I am pleased to announce that Michael Gibney has accepted the PMC's
> > invitation to join.
> >
> > Congratulations and welcome, Michael!
> >
>


-- 
-
Noble Paul

Re: Welcome Michael Gibney to the Solr PMC

[Anshum Gupta]

Congratulations and welcome, Michael!

On Wed, Dec 21, 2022 at 9:11 AM Houston Putman  wrote:

> I am pleased to announce that Michael Gibney has accepted the PMC's
> invitation to join.
>
> Congratulations and welcome, Michael!
>

Re: Welcome Michael Gibney to the Solr PMC

[Jan Høydahl]

Congrats and welcome Michael!

Jan Høydahl

> 21. des. 2022 kl. 18:11 skrev Houston Putman :
> 
> I am pleased to announce that Michael Gibney has accepted the PMC's
> invitation to join.
> 
> Congratulations and welcome, Michael!

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: Welcome Michael Gibney to the Solr PMC

[Michael Gibney]

Thank you for the invitation, and the warm welcome! It's an honor to
be a part of this community.

Michael

On Wed, Dec 21, 2022 at 4:06 PM Jan Høydahl  wrote:
>
> Congrats and welcome Michael!
>
> Jan Høydahl
>
> > 21. des. 2022 kl. 18:11 skrev Houston Putman :
> >
> > I am pleased to announce that Michael Gibney has accepted the PMC's
> > invitation to join.
> >
> > Congratulations and welcome, Michael!
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> For additional commands, e-mail: dev-h...@solr.apache.org
>

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: Welcome Michael Gibney to the Solr PMC

[Gus Heck]

Congratulations and Welcome :)

On Wed, Dec 21, 2022 at 4:41 PM Michael Gibney 
wrote:

> Thank you for the invitation, and the warm welcome! It's an honor to
> be a part of this community.
>
> Michael
>
> On Wed, Dec 21, 2022 at 4:06 PM Jan Høydahl  wrote:
> >
> > Congrats and welcome Michael!
> >
> > Jan Høydahl
> >
> > > 21. des. 2022 kl. 18:11 skrev Houston Putman :
> > >
> > > I am pleased to announce that Michael Gibney has accepted the PMC's
> > > invitation to join.
> > >
> > > Congratulations and welcome, Michael!
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> > For additional commands, e-mail: dev-h...@solr.apache.org
> >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> For additional commands, e-mail: dev-h...@solr.apache.org
>
>

-- 
http://www.needhamsoftware.com (work)
http://www.the111shift.com (play)

Re: Welcome Michael Gibney to the Solr PMC

[Mike Drob]

Welcome and well deserved!

On Wed, Dec 21, 2022 at 3:41 PM Michael Gibney 
wrote:

> Thank you for the invitation, and the warm welcome! It's an honor to
> be a part of this community.
>
> Michael
>
> On Wed, Dec 21, 2022 at 4:06 PM Jan Høydahl  wrote:
> >
> > Congrats and welcome Michael!
> >
> > Jan Høydahl
> >
> > > 21. des. 2022 kl. 18:11 skrev Houston Putman :
> > >
> > > I am pleased to announce that Michael Gibney has accepted the PMC's
> > > invitation to join.
> > >
> > > Congratulations and welcome, Michael!
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> > For additional commands, e-mail: dev-h...@solr.apache.org
> >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> For additional commands, e-mail: dev-h...@solr.apache.org
>
>

Re: Welcome Michael Gibney to the Solr PMC

[Tomás Fernández Löbbe]

Welcome Michael!

On Wed, Dec 21, 2022 at 1:55 PM Mike Drob  wrote:

> Welcome and well deserved!
>
> On Wed, Dec 21, 2022 at 3:41 PM Michael Gibney 
> wrote:
>
> > Thank you for the invitation, and the warm welcome! It's an honor to
> > be a part of this community.
> >
> > Michael
> >
> > On Wed, Dec 21, 2022 at 4:06 PM Jan Høydahl 
> wrote:
> > >
> > > Congrats and welcome Michael!
> > >
> > > Jan Høydahl
> > >
> > > > 21. des. 2022 kl. 18:11 skrev Houston Putman :
> > > >
> > > > I am pleased to announce that Michael Gibney has accepted the PMC's
> > > > invitation to join.
> > > >
> > > > Congratulations and welcome, Michael!
> > >
> > > -
> > > To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> > > For additional commands, e-mail: dev-h...@solr.apache.org
> > >
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> > For additional commands, e-mail: dev-h...@solr.apache.org
> >
> >
>

Re: Welcome Michael Gibney to the Solr PMC

[Koji Sekiguchi]

Welcome Michael!

Koji

2022年12月22日(木) 2:11 Houston Putman :

> I am pleased to announce that Michael Gibney has accepted the PMC's
> invitation to join.
>
> Congratulations and welcome, Michael!
>

Re: Welcome Michael Gibney to the Solr PMC

[Mikhail Khludnev]

Congratulations, Michael!

On Wed, Dec 21, 2022 at 8:11 PM Houston Putman  wrote:

> I am pleased to announce that Michael Gibney has accepted the PMC's
> invitation to join.
>
> Congratulations and welcome, Michael!
>


-- 
Sincerely yours
Mikhail Khludnev