Hi Ran,
GroupStep in OLTP does use "on the fly"-reduction.
https://github.com/apache/incubator-tinkerpop/blob/master/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/map/GroupStep.java#L176-L182
https://github.com/apache/incubator-tinkerpop/blob/master/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/util/ReducingBarrierStep.java#L89-L92
That is, if the amount of objects getting grouped becomes to large, a "midway"
reduction takes place.
Thoughts?,
Marko.
http://markorodriguez.com
On Feb 10, 2016, at 10:35 AM, Ran Magen <[email protected]> wrote:
> I'm not sure if its relevant for this ticket, but I have another use-case
> for combiners in an OLTP use-case.
> In Unipop (to which I'll hopefully have documentation very soon and be able
> to share with you guys) we have GroupStep optimizations. But since we can
> have results for the same step coming from different back-ends, we need
> some way to combine them to one result.
> If we had a simple way to combine the results, that would be great. If not,
> we'll just implement something by ourselves...
>
> On Mon, 8 Feb 2016 at 17:29 Marko A. Rodriguez (JIRA) <[email protected]>
> wrote:
>
>> Marko A. Rodriguez created TINKERPOP-1136:
>> ---------------------------------------------
>>
>> Summary: GroupStep OLAP does not combine, only reduces.
>> Key: TINKERPOP-1136
>> URL: https://issues.apache.org/jira/browse/TINKERPOP-1136
>> Project: TinkerPop
>> Issue Type: Improvement
>> Components: process
>> Affects Versions: 3.1.1-incubating
>> Reporter: Marko A. Rodriguez
>> Fix For: 3.2.0-incubating
>>
>>
>> Right now, {{GroupStep}} does not support a combiner. Unfortunately, this
>> is a necessary consequence of not knowing the semantics of the reduction
>> :|. This is analogous to why Spark has {{reduceByKey}} vs. {{groupByKey}}.
>> HOWEVER, if a {{Barrier}} is used for reduction in the {{reduceTraversal}}
>> and that {{Barrier}} is a {{ReducingBarrierStep}}, then a combiner can be
>> used. I suspect that 99% of the time, the {{Barrier}} is a
>> {{ReducingBarrierStep}}. Once TINKERPOP-1131 is merged, it should be
>> possible to make {{GroupStep}} more intelligent by doing an analysis of its
>> {{Barrier}}-step.
>>
>>
>>
>> --
>> This message was sent by Atlassian JIRA
>> (v6.3.4#6332)
>>
