Re: [PR] Fix IntegerOverflow exception in postings encoding as group-varint [lucene]

2024-05-17 Thread via GitHub
easyice commented on PR #13376: URL: https://github.com/apache/lucene/pull/13376#issuecomment-2118037371 Backport completed and added an entry under 9.10.1 Bug Fixes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] Fix IntegerOverflow exception in postings encoding as group-varint [lucene]

2024-05-17 Thread via GitHub
easyice commented on PR #13376: URL: https://github.com/apache/lucene/pull/13376#issuecomment-2117910603 Okay, I will backport to 9.10/branch_9x. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

Re: [PR] Fix IntegerOverflow exception in postings encoding as group-varint [lucene]

2024-05-17 Thread via GitHub
jpountz commented on PR #13376: URL: https://github.com/apache/lucene/pull/13376#issuecomment-2117901339 +1 to a bugfix release -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific commen

Re: [PR] Fix IntegerOverflow exception in postings encoding as group-varint [lucene]

2024-05-17 Thread via GitHub
jpountz commented on PR #13376: URL: https://github.com/apache/lucene/pull/13376#issuecomment-2117901620 Can you backport to the 9.10 branch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

Re: [PR] Fix IntegerOverflow exception in postings encoding as group-varint [lucene]

2024-05-17 Thread via GitHub
easyice merged PR #13376: URL: https://github.com/apache/lucene/pull/13376 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apa

Re: [PR] Fix IntegerOverflow exception in postings encoding as group-varint [lucene]

2024-05-17 Thread via GitHub
easyice commented on PR #13376: URL: https://github.com/apache/lucene/pull/13376#issuecomment-2117847222 > Does the exception happen because the remainder part of a postings list (after all length 128 blocks are done), which we now encode with GroupVInt, had a docID delta that was >= 1<<30,

Re: [PR] Fix IntegerOverflow exception in postings encoding as group-varint [lucene]

2024-05-17 Thread via GitHub
mikemccand commented on PR #13376: URL: https://github.com/apache/lucene/pull/13376#issuecomment-2117812216 It's hard for me to tell what the expected user impact here is? Does the exception happen because the remainder part of a postings list (after all length 128 blocks are done), which

Re: [PR] Fix IntegerOverflow exception in postings encoding as group-varint [lucene]

2024-05-17 Thread via GitHub
easyice commented on code in PR #13376: URL: https://github.com/apache/lucene/pull/13376#discussion_r1605011950 ## lucene/core/src/java/org/apache/lucene/store/DataOutput.java: ## @@ -328,9 +328,12 @@ public void writeSetOfStrings(Set set) throws IOException { /** * Enco

Re: [PR] Fix IntegerOverflow exception in postings encoding as group-varint [lucene]

2024-05-17 Thread via GitHub
easyice commented on code in PR #13376: URL: https://github.com/apache/lucene/pull/13376#discussion_r1604988872 ## lucene/core/src/java/org/apache/lucene/util/GroupVIntUtil.java: ## @@ -118,6 +120,13 @@ private static int numBytes(int v) { return Integer.BYTES - (Integer.nu

Re: [PR] Fix IntegerOverflow exception in postings encoding as group-varint [lucene]

2024-05-17 Thread via GitHub
easyice commented on code in PR #13376: URL: https://github.com/apache/lucene/pull/13376#discussion_r1604980640 ## lucene/test-framework/src/java/org/apache/lucene/tests/store/BaseDirectoryTestCase.java: ## @@ -1442,6 +1442,19 @@ public void testListAllIsSorted() throws IOExcept

Re: [PR] Fix IntegerOverflow exception in postings encoding as group-varint [lucene]

2024-05-17 Thread via GitHub
jpountz commented on code in PR #13376: URL: https://github.com/apache/lucene/pull/13376#discussion_r1604452451 ## lucene/test-framework/src/java/org/apache/lucene/tests/store/BaseDirectoryTestCase.java: ## @@ -1442,6 +1442,19 @@ public void testListAllIsSorted() throws IOExcept

Re: [PR] Fix IntegerOverflow exception in postings encoding as group-varint [lucene]

2024-05-17 Thread via GitHub
easyice commented on PR #13376: URL: https://github.com/apache/lucene/pull/13376#issuecomment-2117348590 I pushed the requested changes, @jpountz . No rush, just wanted to let you know. -- This is an automated message from the Apache Git Service. To respond to the message, please log on t

Re: [PR] Fix IntegerOverflow exception in postings encoding as group-varint [lucene]

2024-05-17 Thread via GitHub
easyice commented on PR #13376: URL: https://github.com/apache/lucene/pull/13376#issuecomment-2116962504 That's also a good idea! by this approach we can make `writeGroupVInts `/`readGroupVInt` use positive only. it's actually handled as an unsigned integer, so we don't need to consider the

Re: [PR] Fix IntegerOverflow exception in postings encoding as group-varint [lucene]

2024-05-17 Thread via GitHub
jpountz commented on PR #13376: URL: https://github.com/apache/lucene/pull/13376#issuecomment-2116909274 Thanks for looking into it! Your approach works, but I'm tempted to fix it the other way around, by no longer checking if values are in the expected range with `Math.toIntExact` but rath

Re: [PR] Fix IntegerOverflow exception in postings encoding as group-varint [lucene]

2024-05-16 Thread via GitHub
easyice commented on PR #13376: URL: https://github.com/apache/lucene/pull/13376#issuecomment-2115450214 The essence of this issue is how to deal with the integer value with the sign bit as 1 (like this integer overflow case). We have two options. * The first is pass a negative long valu

Re: [PR] Fix IntegerOverflow exception in postings encoding as group-varint [lucene]

2024-05-16 Thread via GitHub
easyice commented on PR #13376: URL: https://github.com/apache/lucene/pull/13376#issuecomment-2114744868 This change keeps the input values of `writeGroupVInts` explained as integer, instead of a big number greater than Integer.MAX_VALUE -- This is an automated message from the Apache Gi

[PR] Fix IntegerOverflow exception in postings encoding as group-varint [lucene]

2024-05-16 Thread via GitHub
easyice opened a new pull request, #13376: URL: https://github.com/apache/lucene/pull/13376 Closes: https://github.com/apache/lucene/issues/13373 This exception occurs because a negative integer value stores as positive long. In line 376, after a long value `<< 1`, if the sign bit