[
https://issues.apache.org/jira/browse/LUCENE-10605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
sun wuqiang updated LUCENE-10605:
---------------------------------
Description:
ArrayUtil.{*}oversize{*}(int minTargetSize, int bytesPerElement)
This method is used to calculate the optimal length of an array during
expansion.
According to current logic,in order to avoid space waste caused by *object
alignment gap.* In *32-bit* JVM,the array length will select the numbers(the
+current optional+ columns) in the table below. But the results weren't perfect.
For example, if I want to expand byte[2], I call the method oversize(2,1) to
get the size of the next array, which returns 8.
But Byte [8] is not the best result.
Since byte[8] and byte[12] use the same memory space (both are 24 bytes due to
alignment gap),
So it's best to return 12 here.
See the table below.
!image-2022-06-09-08-26-36-528.png!
I used *jol-core* to calculate object alignment gap
{code:java}
<dependency>
<groupId>org.openjdk.jol</groupId>
<artifactId>jol-core</artifactId>
<version>0.16</version>
<scope>compile</scope>
</dependency> {code}
Execute the following code:
{code:java}
System.out.println(ClassLayout.parseInstance(new byte[6]).toPrintable()); {code}
!image-2022-06-08-21-24-57-674.png!
To further verify that the tool's results are correct, I wrote the following
code to infer how much space the array of different lengths actually occupies
based on when the OOM occursThe conclusion is consistent with jol-core.
{code:java}
// -Xms16m -Xmx16m
// Used to infer the memory space occupied
// by the length of various arrays
public static void main(String[] args) {
byte[][] arr = new byte[1024 * 1024][];
for (int i = 0; i < arr.length; i++) {
if (i % 100 == 0) {
System.out.println(i);
}
// According to OOM occurrence time
// in 32-bit JVM,
// Arrays range in length from 5 to 12,
// occupying the same amount of memory
arr[i]=new byte[5];
}
} {code}
*new byte[5]* and *new byte[12]* use the same amount of memory
----
In addition +*- XX: ObjectAlignmentInBytes*+ should also affect the return
value of this method. But I don't know whether it is necessary to do this
function. If necessary, I will modify it together. Thank you very much!
was:
ArrayUtil.{*}oversize{*}(int minTargetSize, int bytesPerElement)
This method is used to calculate the optimal length of an array during
expansion.
According to current logic,in order to avoid space waste caused by *object
alignment gap.* In *32-bit* JVM,the array length will select the numbers(the
+current optional+ columns) in the table below. But the results weren't perfect.
See the table below.
!image-2022-06-08-20-50-27-712.png!
I used *jol-core* to calculate object alignment gap
{code:java}
<dependency>
<groupId>org.openjdk.jol</groupId>
<artifactId>jol-core</artifactId>
<version>0.16</version>
<scope>compile</scope>
</dependency> {code}
Execute the following code:
{code:java}
System.out.println(ClassLayout.parseInstance(new byte[6]).toPrintable()); {code}
!image-2022-06-08-21-24-57-674.png!
To further verify that the tool's results are correct, I wrote the following
code to infer how much space the array of different lengths actually occupies
based on when the OOM occursThe conclusion is consistent with jol-core.
{code:java}
// -Xms16m -Xmx16m
// Used to infer the memory space occupied
// by the length of various arrays
public static void main(String[] args) {
byte[][] arr = new byte[1024 * 1024][];
for (int i = 0; i < arr.length; i++) {
if (i % 100 == 0) {
System.out.println(i);
}
// According to OOM occurrence time
// in 32-bit JVM,
// Arrays range in length from 5 to 12,
// occupying the same amount of memory
arr[i]=new byte[5];
}
} {code}
*new byte[5]* and *new byte[12]* use the same amount of memory
----
In addition +*- XX: ObjectAlignmentInBytes*+ should also affect the return
value of this method. But I don't know whether it is necessary to do this
function. If necessary, I will modify it together. Thank you very much!
> fix error in 32bit jvm object alignment gap calculation
> -------------------------------------------------------
>
> Key: LUCENE-10605
> URL: https://issues.apache.org/jira/browse/LUCENE-10605
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/other
> Affects Versions: 9.2
> Environment: jdk 7 32-bit
> jdk 8 32-bit
> Reporter: sun wuqiang
> Priority: Trivial
> Attachments: image-2022-06-08-20-50-27-712.png,
> image-2022-06-08-21-24-57-674.png, image-2022-06-09-08-25-55-289.png,
> image-2022-06-09-08-26-36-528.png
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> ArrayUtil.{*}oversize{*}(int minTargetSize, int bytesPerElement)
> This method is used to calculate the optimal length of an array during
> expansion.
>
> According to current logic,in order to avoid space waste caused by *object
> alignment gap.* In *32-bit* JVM,the array length will select the numbers(the
> +current optional+ columns) in the table below. But the results weren't
> perfect.
> For example, if I want to expand byte[2], I call the method oversize(2,1) to
> get the size of the next array, which returns 8.
> But Byte [8] is not the best result.
> Since byte[8] and byte[12] use the same memory space (both are 24 bytes due
> to alignment gap),
> So it's best to return 12 here.
> See the table below.
> !image-2022-06-09-08-26-36-528.png!
>
> I used *jol-core* to calculate object alignment gap
> {code:java}
> <dependency>
> <groupId>org.openjdk.jol</groupId>
> <artifactId>jol-core</artifactId>
> <version>0.16</version>
> <scope>compile</scope>
> </dependency> {code}
>
> Execute the following code:
> {code:java}
> System.out.println(ClassLayout.parseInstance(new byte[6]).toPrintable());
> {code}
>
> !image-2022-06-08-21-24-57-674.png!
>
> To further verify that the tool's results are correct, I wrote the following
> code to infer how much space the array of different lengths actually occupies
> based on when the OOM occursThe conclusion is consistent with jol-core.
> {code:java}
> // -Xms16m -Xmx16m
> // Used to infer the memory space occupied
> // by the length of various arrays
> public static void main(String[] args) {
> byte[][] arr = new byte[1024 * 1024][];
> for (int i = 0; i < arr.length; i++) {
> if (i % 100 == 0) {
> System.out.println(i);
> }
> // According to OOM occurrence time
> // in 32-bit JVM,
> // Arrays range in length from 5 to 12,
> // occupying the same amount of memory
> arr[i]=new byte[5];
> }
> } {code}
> *new byte[5]* and *new byte[12]* use the same amount of memory
> ----
>
> In addition +*- XX: ObjectAlignmentInBytes*+ should also affect the return
> value of this method. But I don't know whether it is necessary to do this
> function. If necessary, I will modify it together. Thank you very much!
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
