osscm commented on PR #16961: URL: https://github.com/apache/iceberg/pull/16961#issuecomment-6069339896
While implementing and testing the SCALAR ref impl (#18187), and coming at it with a bit of vector/lance understanding, a thought: could we extend the spec so the framework also supports ranking-based indexes, not just pruning ones? That would open the gates to different kinds of ranking indexes (ANN over the vector type in #18297 being one), and let people solve that inside the Iceberg table itself instead of standing up something separate - building on what you've already added here. I'm planning to try ANN on top of the secondary-index framework and will share what I learn. Maybe will be a followup PR once this is merged to extend it; for now just flagging it early, and curious how we are thinking about the ranking index type. cc @rdblue @RussellSpitzer @pvary @huaxingao -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
