Hi everyone, "Classloader" is a fair and valid point. I have shared this perspective in the past and discussed it with Nandor as well: I believe federation should utilize a "plugin" mechanism, similar to how persistence is handled.
By adopting this approach, each plugin (such as HMS) is in its own module, and would be responsible for managing its own dependencies and shading. Regards, JB On Wed, Mar 18, 2026 at 12:37 AM Yufei Gu <[email protected]> wrote: > Federation seems like the strongest candidate given the dependency surface. > For example, Trino-style classloader isolation could help when conflicts > arise (e.g., Hadoop or Guava across different federation implementations). > That said, I’d suggest introducing this only once we actually hit such > issues. Until then, keeping things simple with proper modularization may be > the better tradeoff. WDTY? > > Yufei > > > On Tue, Mar 17, 2026 at 2:00 PM Madhan Neethiraj <[email protected]> > wrote: > > > > we need to be careful about is proper modularization to control > > dependency scope growth. > > > > I share this concern; it is applicable for any extension like > authorizers. > > Having a built-in support for isolation of extension specific > dependencies > > will significantly reduce/eliminate complexities in handling potential > > library conflicts and avoid impact to Polaris core due to dependencies > > dragged by extensions. Trino's plugin model seems to be a good reference > - > > 1) and 2). > > > > Thanks, > > Madhan > > > > 1) https://trino.io/docs/current/installation/plugins.html > > 3) https://trino.io/docs/current/develop/spi-overview.html > > > > > > On 3/17/26, 1:30 PM, "Prashant Singh via dev" <[email protected] > > <mailto:[email protected]>> wrote: > > > > > > +1 to this. I would also recommend thinking about this from an interface > > POV to have BaseMetastoreCatalog (from Apache Iceberg) as a way to > > integrate, with a defined connection object that can feed the connection > to > > BQ or Glue (perhaps a standard key-value prep) > > and the dependencies can be added accordingly. > > > > > > Thanks, > > Prashant Singh > > > > > > > > > > On Tue, Mar 17, 2026 at 12:08 PM Dmitri Bourlatchkov <[email protected] > > <mailto:[email protected]>> > > wrote: > > > > > > > Hi Joy, > > > > > > I suppose Polaris will be the REST API + AuthZ layer on top of > > > BigQueryMetastoreCatalog, right? How do you envision this? > > > > > > In general, I think it's going to be a useful contribution. > > > > > > From my POV, the only technical issue we need to be careful about is > > > proper modularization to control dependency scope growth. I imagine > some > > > downstream projects may want to opt in/out of BigQueryMetastoreCatalog > > and > > > its transitive dependencies at their own discretion. > > > > > > Cheers, > > > Dmitri. > > > > > > On 2026/03/17 18:19:34 Joy wrote: > > > > Hi, > > > > > > > > I'm Joy. We use BigQueryMetastoreCatalog for Iceberg on GCP and want > to > > > use > > > > Polaris as a catalog gateway. I see federation currently supports > REST > > > and > > > > HMS. > > > > > > > > Would there be interest in supporting other native Iceberg catalogs > > like > > > > BQMS or Glue? I've been working on a prototype for BQMS following the > > HMS > > > > pattern and would be happy to contribute it if I'm successful. > > > > > > > > Regards, > > > > Joy > > > > > > > > > > > > > > > > > >
