gaoj0017 commented on PR #14078: URL: https://github.com/apache/lucene/pull/14078#issuecomment-2644556568
After Elastic’s last round of replies, the Elastic team reached us for clarification on the issues via zoom meetings. In the meetings, they promised to fix the misattribution, so we suspended our reply and waited for their amendment. Recently, Elastic made some edits on the [BBQ blog](https://www.elastic.co/search-labs/blog/better-binary-quantization-lucene-elasticsearch) and this pull request. However, (1) the updated blog still positions BBQ as a new method (despite that BBQ majorly follows our RaBitQ method with some differences in the implementation only), and (2) the updated PR does not position our extended RaBitQ method as a prior art of their method (despite that the method is similar to our extended RaBitQ method at its core and our method is a prior art). As academic researchers, we feel tired of writing emails asking for proper attribution round after round, and hence we would like to speak out for ourselves through this channel here. Point 1 in our reply: This pull request is a direct follow-up to [the preceding pull request](https://github.com/apache/lucene/pull/13651) of the so-called “BBQ” method. Thus, this point should not be ignored and excluded from the discussion. We emphasize again that all the key features of the so-called “BBQ” method follow our RaBitQ method, with only minor modifications. Point 2 in our last reply: For the community’s information, we would like to provide the following timeline. - On 16-Sep-2024, we published our [extended RaBitQ paper](https://arxiv.org/abs/2409.09913) along with codes. - On 25-Oct-2024, we circulated the extended RaBitQ paper to Elastic as we knew they were implementing RaBitQ in Lucene. They replied positively that they will eagerly read our new paper. At that time, we didn’t expect that they would announce a variant of our RaBitQ method as their own innovation later. In addition, it is worth highlighting that our extended RaBitQ method is the first to achieve reasonable accuracy with 1-bit and 2-bit binary/scalar quantization in the literature by then. - On 11-Nov-2024, Elastic announced the BBQ method as their new innovation (despite that BBQ is simply our RaBitQ method with some differences in implementation). - On 22-Nov-2024, we expressed our concerns over the BBQ method to Elastic via email. Elastic replied that they would need internal discussions and would get in touch soon. - On 14-Dec-2024, we had not received any replies from Elastic and Elastic kept marketing the so-called “BBQ” method as their own innovation in several blogs and press releases. We therefore expressed our concerns over the BBQ method to the Lucene community (https://github.com/apache/lucene/pull/13651). - On 19/20-Dec-2024, Elastic announced the OSQ method in this PR and [this blog](https://www.elastic.co/search-labs/blog/scalar-quantization-optimization). They claimed to have unlocked the performance gain for various bits. However, they ignored the fact that our extended RaBitQ method can already handle various bits, despite that the method was shared with Elastic on 25 Oct 2024 and published on arXiv in Sep 2024. - 14-Dec-2024 to 07-Jan-2025, Elastic and we had a few rounds of communications via this channel at GitHub, which are visible to the public. The last response was from Elastic on 07-Jan-2025, and we stopped replying here because Elastic reached out to us after their response and proposed to have communications via other channels (email and meetings) and we agreed with the hope that Elastic would fix the acknowledgement problem. - 13-Jan-2025 and 22-Jan-2025, Elastic and we had two Zoom meetings and email communications and discussed how to provide sufficient acknowledgements to RaBitQ and its extension. Afterwards, Elastic made some edits on [the BBQ blog](https://www.elastic.co/search-labs/blog/better-binary-quantization-lucene-elasticsearch) and this PR. However, (1) in the updated blog, Elastic still positions BBQ as a new methodology and does not explicitly specify that the core ideas of BBQ follow those of RaBitQ and (2) in this PR, Elastic still positions OSQ as a parallel work of our extended RaBitQ method (despite its core ideas are similar to our extended RaBitQ method and our method is a prior part and published months ago). In addition, Elastic shared a doc with us explaining the differences between OSQ and extended RaBitQ, but our understanding is that the core ideas of two methods, i.e., trying different parameters for scalar quantization on a per-vector basis, are similar. - 05-Feb-2025, this is when Elastic contacted us via <ins>Email</ins> lastly (as of today), saying that they think the acknowledgement of our RaBitQ and extended RaBitQ (based on the updated blog and PR) is sufficient. - 08-Feb-2025 (today), this is when we replied to Elastic via <ins>Email</ins> lastly, updating them we shall speak out for us in other channels (after getting tired of writing emails round after round). In summary, our requirements are two-fold: (1) Elastic should position BBQ as a variant of RaBitQ but not as a new innovation; and (2) Elastic should acknowledge our extended RaBitQ method as a prior art, which has covered similar ideas of the OSQ method and been published months earlier, but not a parallel work. As for future communications, we may not be able to respond to every future response from Elastic because we feel they can keep arguing and changing their arguing points - in this thread, Elastic have changed their attribution of OSQ several times (from the inspiration of RaBitQ to LVQ and SCANN, and to their own earlier PR). We should have been focusing on research more as academic researchers. No matter how they would argue, they cannot deny the following two points: 1. <ins>**The so-called BBQ majorly follows our RaBitQ method;** </ins> 2. <ins>**The OSQ method (introduced in this PR) has its major idea similar to our extended RaBitQ method and our extended RaBitQ method is a prior art which achieves good accuracy at 1-bit/2-bit binary/scalar quantization for the first time.**</ins> We would also consider reaching out to more channels to speak out for ourselves if necessary. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
