uschindler commented on PR #12681:
URL: https://github.com/apache/lucene/pull/12681#issuecomment-1763361313
> > This also makes this test reproducible from random seed regardless of
the hardware, as `SPECIES_PREFERRED` is not used at all in tests. From a test
perspective, it is like a forbidden-api.
>
> I am mostly ok, but initially I stumbled on this. I was expecting that all
tests get slow. But luckily we run tests without C2 so by default for all test
except the one using the testMode, we use the scalar impl.
>
> I'd like to add some extra safety. If the provider class detects that any
of those properties are given at command line and at same time it is not in
Test mode, it should scream loud in logger and use default impl. The problem is
that Jenkins uses different GC and does not force Client mode, so it would take
very long as all tests run using fixed bit size.
>
> I have a small change to implement this, are you ok? Basically I would
read the system props in the provider code.
Here is my idea, I can commit it if you agree. It basically moves the system
properties to the main `VectorizationProvider` (as optional). The
`SecurityException` is also handled there.
If the system properties are detected, it complains on logger and uses
default provider, unless testMode is enabled. This leads to the following:
- TestVectorUtilSupport uses the Panama provider
- All other tests fallback to default provider (they do by default due to
client VM, but on jenkins they won't be slow)
```patch
.../vectorization/VectorizationProvider.java | 48
++++++++++++++++++++--
.../vectorization/PanamaVectorUtilSupport.java | 16 ++------
2 files changed, 48 insertions(+), 16 deletions(-)
diff --git
a/lucene/core/src/java/org/apache/lucene/internal/vectorization/VectorizationProvider.java
b/lucene/core/src/java/org/apache/lucene/internal/vectorization/VectorizationProvider.java
index 3192af72b2c..9c41b8dfa6d 100644
---
a/lucene/core/src/java/org/apache/lucene/internal/vectorization/VectorizationProvider.java
+++
b/lucene/core/src/java/org/apache/lucene/internal/vectorization/VectorizationProvider.java
@@ -25,8 +25,12 @@ import java.security.AccessController;
import java.security.PrivilegedAction;
import java.util.Locale;
import java.util.Objects;
+import java.util.Optional;
+import java.util.OptionalInt;
import java.util.Set;
+import java.util.function.Predicate;
import java.util.logging.Logger;
+import java.util.stream.Stream;
import org.apache.lucene.util.SuppressForbidden;
import org.apache.lucene.util.VectorUtil;
@@ -39,6 +43,37 @@ import org.apache.lucene.util.VectorUtil;
*/
public abstract class VectorizationProvider {
+ static final OptionalInt TESTS_VECTOR_SIZE;
+ static final Optional<Boolean> TESTS_FORCE_INTEGER_VECTORS;
+
+ static {
+ var vs = OptionalInt.empty();
+ try {
+ vs =
+ Stream.ofNullable(System.getProperty("tests.vectorsize"))
+ .filter(Predicate.not(String::isEmpty))
+ .mapToInt(Integer::parseInt)
+ .findAny();
+ } catch (
+ @SuppressWarnings("unused")
+ SecurityException se) {
+ // ignored
+ }
+ TESTS_VECTOR_SIZE = vs;
+ var enforce = Optional.<Boolean>empty();
+ try {
+ enforce =
+
Optional.ofNullable(System.getProperty("tests.forceintegervectors"))
+ .filter(Predicate.not(String::isEmpty))
+ .map(Boolean::valueOf);
+ } catch (
+ @SuppressWarnings("unused")
+ SecurityException se) {
+ // ignored
+ }
+ TESTS_FORCE_INTEGER_VECTORS = enforce;
+ }
+
/**
* Returns the default instance of the provider matching vectorization
possibilities of actual
* runtime.
@@ -87,9 +122,16 @@ public abstract class VectorizationProvider {
"Java vector incubator module is not readable. For optimal
vector performance, pass '--add-modules jdk.incubator.vector' to enable Vector
API.");
return new DefaultVectorizationProvider();
}
- if (!testMode && isClientVM()) {
- LOG.warning("C2 compiler is disabled; Java vector incubator API
can't be enabled");
- return new DefaultVectorizationProvider();
+ if (!testMode) {
+ if (TESTS_VECTOR_SIZE.isPresent() ||
TESTS_FORCE_INTEGER_VECTORS.isPresent()) {
+ LOG.warning(
+ "Vector bitsize and/or integer vectors enforcement; using
default vectorization provider outside of testMode");
+ return new DefaultVectorizationProvider();
+ }
+ if (isClientVM()) {
+ LOG.warning("C2 compiler is disabled; Java vector incubator API
can't be enabled");
+ return new DefaultVectorizationProvider();
+ }
}
try {
// we use method handles with lookup, so we do not need to deal
with setAccessible as we
diff --git
a/lucene/core/src/java20/org/apache/lucene/internal/vectorization/PanamaVectorUtilSupport.java
b/lucene/core/src/java20/org/apache/lucene/internal/vectorization/PanamaVectorUtilSupport.java
index dad428845b7..22ae585bd13 100644
---
a/lucene/core/src/java20/org/apache/lucene/internal/vectorization/PanamaVectorUtilSupport.java
+++
b/lucene/core/src/java20/org/apache/lucene/internal/vectorization/PanamaVectorUtilSupport.java
@@ -57,10 +57,7 @@ final class PanamaVectorUtilSupport implements
VectorUtilSupport {
// default to platform supported bitsize
int vectorBitSize = VectorShape.preferredShape().vectorBitSize();
// but allow easy overriding for testing
- try {
- vectorBitSize = Integer.getInteger("tests.vectorsize", vectorBitSize);
- } catch (SecurityException ignored) {
- }
+ vectorBitSize =
VectorizationProvider.TESTS_VECTOR_SIZE.orElse(vectorBitSize);
INT_SPECIES = VectorSpecies.of(int.class,
VectorShape.forBitSize(vectorBitSize));
VECTOR_BITSIZE = INT_SPECIES.vectorBitSize();
FLOAT_SPECIES = INT_SPECIES.withLanes(float.class);
@@ -76,15 +73,8 @@ final class PanamaVectorUtilSupport implements
VectorUtilSupport {
// hotspot misses some SSE intrinsics, workaround it
// to be fair, they do document this thing only works well with
AVX2/AVX3 and Neon
boolean isAMD64withoutAVX2 = Constants.OS_ARCH.equals("amd64") &&
VECTOR_BITSIZE < 256;
- boolean hasFastIntegerVectors = isAMD64withoutAVX2 == false;
- try {
- hasFastIntegerVectors =
- Boolean.parseBoolean(
- System.getProperty(
- "tests.forceintegervectors",
Boolean.toString(hasFastIntegerVectors)));
- } catch (SecurityException ignored) {
- }
- HAS_FAST_INTEGER_VECTORS = hasFastIntegerVectors;
+ HAS_FAST_INTEGER_VECTORS =
+
VectorizationProvider.TESTS_FORCE_INTEGER_VECTORS.orElse(isAMD64withoutAVX2 ==
false);
}
@Override
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
