I'm glad to see this proposal because people have been talking about working on an implementation of the Iceberg REST catalog spec for a long time. I don't think that it is a good idea to put an implementation in the Iceberg project itself, so it is great to see a project that intends to build one to meet that demand.
I'd like to volunteer to help out and mentor the project. I have a lot of context on the REST catalog spec from contributing to the design and client implementation, and I've helped both Parquet and Iceberg through incubation (which is why I talk about maintaining LICENSE and NOTICE so much). My take on the PPMC/committer list is that this seems like a reasonable choice. I'm also not worried that the project won't be able to attract a community given the size of the initial list. Ryan On Wed, Jul 31, 2024 at 1:02 PM Tyler Akidau <taki...@apache.org> wrote: > Hey folks, > > I wanted to give a little bit of additional context beyond what JB and > Jack have said so far in other threads. Everyone’s observations about the > level of community code contributions, the committer/PPMC list setup, > adjacency to other projects, etc. are spot on. The code has been pretty > much entirely delivered by Snowflake at this point. And the PPMC/committer > division in the proposal is atypical, but as Jack and JB called out, it’s > reflective of the collaborative community building that’s been happening > over the last two months; more on this below. > > From a code contribution perspective, we’ve largely been blocked on > getting a shareable repo up and running, which I admit took longer than > we’d hoped. That was primarily due to Snowflake internal logistics, which > as with any large company, is what it is at times. Now that we have that in > place, I expect to see more material code contributions rolling in over the > coming weeks. We’ve been having early discussions with the Dremio folks > about how Nessie features like catalog level versioning can be integrated > into Polaris, and once we align on a concrete design, Robert and JB and > crew will be diving in more deeply. Similarly, we’ve had early discussions > on integrations with other partners in the community, and now that the > codebase is fully public, it will be easier for us to make concrete > progress on turning those discussions into actual code contributions (e.g., > there's already some early Trino integration work happening [1].) > > From a community building perspective, in particular the concern that it > can be tough to build a community for a podling in a vacuum, I completely > agree. If you start a podling with no community in sight, you may be left > floundering and alone for quite some time. That’s why JB and I have spent > the last two months bootstrapping that process, finding stakeholders who > are interested in helping grown Polaris in some way, making sure we’re > directionally aligned on where we want the project to go, and identifying > specific individuals with both a vested interest in contribution and > experience helping grow and run Apache projects in the Apache way. A lot of > time, thought, and collaboration went into building this initial community > across a diverse set of stakeholders, and we wanted to reflect that in > calling out the proposed PPMC list separately. As JB said, we’re happy to > adjust the lists to something more standard if desired, but we believe the > story behind the lists is important in this case. > > From project overlap perspective, I just want to echo Jack’s take on > things: Polaris for now is fully focused on Iceberg, taking a depth first > approach, with the goal of implementing the entire Iceberg REST API spec > and helping push forward the state of the art in the Iceberg ecosystem for > features like governance that are highly important for all of our > collective user bases. It’s absolutely adjacent to Gravitino, but as others > have said, it feels to me that they are heading in somewhat different > directions overall. I also think there’s lots of empty space in the open > source catalog ecosystem in general at this point, with plenty of room for > both of these efforts to beneficially exist in parallel. And we are > absolutely open to discussing collaborations, with Gravitino, Amoro, or > anyone else; JB has highlighted the importance of this from the very > beginning of our Polaris conversations, and I completely agree. > > And lastly yes, any existing trademark issues should be fixed. I know > there was one batch discovered after the initial push that we were working > on fixing, but I'll go back and see if there are others we haven't > addressed (or if those fixes somehow just haven't made it out yet.) > > Thank you everyone for the feedback and enthusiasm. We appreciate it. :-) > > [1] https://github.com/polaris-catalog/polaris/pull/42 > > Cheers, > -Tyler > > On 2024/07/31 07:30:07 Justin Mclean wrote: > > Hi, > > > > I sent a reply earlier, but my email is acting up and looks like it > didn’t get through. I have some concerns with this proposal. > > > > In general, the incubator likes projects to have a code base and a small > community, I’m not seeing a community here. Trying to build one during > incubation can be difficult. We have recently rejected proposals in a > similar state, asking them to come back when they have more of a community > around the project. > > > > The PPMC/committer split is unusual. > > > > There seems to be little relation to people who have contributed to the > project and the initial committer list. A large number of the people > involved in commits (80+%) are from one vendor, with two exceptions, and > two others have made one or two minor commits of a couple of lines. > > > > Adding people to PPMC to help out also seems unusual, as that is the > mentor's job. > > > > In short, this seems to me (and I could be wrong) like a project mostly > from a single vendor, but the proposal has been made to make it look like > more people are involved. It may well be that these people will be > involved, but I’d prefer if the project was upfront about this and added > committers the usual way during incubation. > > > > In short, the initial commit list looks problematic to me. > > > > Kind Regards, > > Justin > > > > P.S. The repo landing page/readme has some ASF trademark issues that > would be good to address. > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org > > For additional commands, e-mail: general-h...@incubator.apache.org > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org > For additional commands, e-mail: general-h...@incubator.apache.org > > -- Ryan Blue Databricks
