Hi Rakesh, Yeah, the team aware that, they plan to merge the Trino plugin into the main repository at some point of time, depends on its maturity. The Hive plugin is not on current roadmap I think, but this is open for discussion.
Best regards, Shaofeng Shi 史少锋 Apache Kylin PMC, Apache Incubator PMC, Email: shaofeng...@apache.org Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html Join Kylin user mail group: user-subscr...@kylin.apache.org Join Kylin dev mail group: dev-subscr...@kylin.apache.org Rakesh Radhakrishnan <rake...@apache.org> 于2023年12月27日周三 14:06写道： > OK, that's really interesting. I hope you will add "Gluten-Trino > <https://github.com/oap-project/Gluten-Trino>" and "Gluten-Flink" as > sub-projects(plugin module) under the main Gluten project and follow a > single release cycle? > > Is it technically/architecturally feasible to develop a "Gluten-Hive" > plugin (as we know Hive uses Apache Tez DAG) as a long term plan ? > > Thanks, > Rakesh > > On Tue, Dec 26, 2023 at 4:30 PM ShaoFeng Shi <shaofeng...@apache.org> > wrote: > > > Hi Rakesh, > > > > Thanks for your comment. The Trino plugin is under PoC, not ready for > > widely use at this moment, so it is staged in another git repository: > > https://github.com/oap-project/Gluten-Trino > > Flink support is still in early discussion stage, need more collaboration > > in the future. > > > > Best regards, > > > > Shaofeng Shi 史少锋 > > Apache Kylin PMC, > > Apache Incubator PMC, > > Email: shaofeng...@apache.org > > > > Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html > > Join Kylin user mail group: user-subscr...@kylin.apache.org > > Join Kylin dev mail group: dev-subscr...@kylin.apache.org > > > > > > > > > > Rakesh Radhakrishnan <rake...@apache.org> 于2023年12月26日周二 11:24写道： > > > > > Thanks Shaofeng and team for the efforts! > > > > > > Its really interesting to see that Flink and Trino projects can get the > > > benefits of Gluten and boost their performance. Much appreciated, if > you > > > could share more on this integration, perf_numbers, > > pullrequest_reference, > > > jira_reference etc. Will Flink and Trino already have a plugin > mechanism > > in > > > place to seamlessly integrate with Gluten ? > > > > > > Thanks, > > > Rakesh > > > > > > On Sat, Dec 23, 2023 at 6:13 PM ShaoFeng Shi <shaofeng...@apache.org> > > > wrote: > > > > > > > Hi Enrico, > > > > > > > > This is a good question. Actually I asked the same to the team when > > they > > > > find me for coaching. > > > > > > > > As the proposal mentioned, Gluten is a middle layer between the > > JVM-based > > > > SQL engine and native libraries. It connects the two sides, and build > > > > common services like unified plan transformation, seamless native > > > > integration, clear JNI interfaces, etc. Spark is the first engine it > > > > supports now, but the framework and mechanism can also work for other > > > > engines like Trino, Flink SQL in the future (the proposal also > > mentioned > > > > this). Underlying Gluten, it can integrate with different libararies, > > > like > > > > Velox, Clickhouse and Arrow (not ready yet). So, its mission and > scope > > is > > > > different, that's why we put it as a new project. > > > > > > > > Best regards, > > > > > > > > Shaofeng Shi 史少锋 > > > > Apache Kylin PMC, > > > > Apache Incubator PMC, > > > > Email: shaofeng...@apache.org > > > > > > > > Apache Kylin FAQ: > > https://kylin.apache.org/docs/gettingstarted/faq.html > > > > Join Kylin user mail group: user-subscr...@kylin.apache.org > > > > Join Kylin dev mail group: dev-subscr...@kylin.apache.org > > > > > > > > > > > > > > > > > > > > Enrico Olivelli <eolive...@gmail.com> 于2023年12月22日周五 22:43写道： > > > > > > > > > Shaofeng, > > > > > > > > > > Il giorno ven 22 dic 2023 alle ore 13:59 ShaoFeng Shi > > > > > <shaofeng...@apache.org> ha scritto: > > > > > > > > > > > > Hi IPMC members, > > > > > > > > > > > > I would like to propose a new project to the ASF incubator - > > Gluten. > > > > > > > > > > > > Gluten[1] is a middle layer responsible for offloading Apache > Spark > > > SQL > > > > > > queries to native engines. This project aims to address the CPU > > > > > > computational bottleneck to offload SparkSQL operators to native > > > > engines > > > > > in > > > > > > data loading scenarios based on Apache Spark. > > > > > > > > > > Have you considered making this project a subproject of Apache > Spark > > ? > > > > > > > > > > Enrico > > > > > > > > > > > > > > > > > Here is the proposal - > > > > > > > > https://cwiki.apache.org/confluence/display/INCUBATOR/GlutenProposal > > > > > > > > > > > > I would be the Champion of the project. I will mentor and help > the > > > > > project > > > > > > through the incubator with Yu Li [l...@apache.org], Kent Yao [ > > > > > y...@apache.org] > > > > > > and Wenli Zhang [ovi...@apache.org] . > > > > > > > > > > > > We are open to hearing the feedback from the incubator. > > > > > > > > > > > > Best, > > > > > > Shaofeng Shi. > > > > > > > > > > > > [1] https://oap-project.github.io/gluten/ > > > > > > > > > > > > Best regards, > > > > > > > > > > > > Shaofeng Shi 史少锋 > > > > > > Apache Kylin PMC, > > > > > > Apache Incubator PMC, > > > > > > Email: shaofeng...@apache.org > > > > > > > > > > > > Apache Kylin FAQ: > > > > https://kylin.apache.org/docs/gettingstarted/faq.html > > > > > > Join Kylin user mail group: user-subscr...@kylin.apache.org > > > > > > Join Kylin dev mail group: dev-subscr...@kylin.apache.org > > > > > > > > > > > --------------------------------------------------------------------- > > > > > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org > > > > > For additional commands, e-mail: general-h...@incubator.apache.org > > > > > > > > > > > > > > > > > > > >
