Hi, Little background before I put forward my question is, as we know NB uses JDK swing classes to render its components and JDK uses its own rendering pipeline be it directx on windows, xrender on linux and opengl on mac to render. Now, we are creating a new rendering pipeline for JDK on macos to support Metal pipeline since Apple has deprecated opengl pipeline going forward. So, we need to test IDE performance for metal compared to currently used opengl pipeline. These pipeline is transparent to NB users/developers and they will continue to use the same JDK swing api but internally the swing components will use Metal as opposed to opengl APIs on macos. So, we need to test whether our newly created pipeline will behave same performance wise compared to current opengl. we have run other benchmark suite and now we need to test IDE performance.
Given that background, we chanced upon these tests in NB https://github.com/apache/netbeans/tree/master/java/performance.java/test/qa-functional/src/org/netbeans/performance/j2se which is supposed to test 3 categories, MeasureJ2SEActionsTest - open/close file, create/open/close project, typing in editor, code completion invocation MeasureJ2SEDialogsTest - tests dialog and windows MeasureJ2SEMenusTest - tests main menus and contextual menus by comparing the time taken for the response against pre-defined benchmark and mark the test-case success or failed based on whether it takes more or less time than benchmark time. We tried running jdk15+ binary on NB12.0 on macos 10.14.6 and we saw the test is just giving pass/fail remark Question is : Is there any way we can get the individual test time for each test by modifying any NB file instead of just test pass/fail? Also, we saw that many tests are erroring out with following error Forked Java VM exited abnormally. Please note the time in the report does not reflect the time until the VM exit. junit.framework.AssertionFailedError at jdk.internal.reflect.GeneratedMethodAccessor90.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.util.Vector.forEach(Vector.java:1358) at jdk.internal.reflect.GeneratedMethodAccessor90.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at jdk.internal.reflect.GeneratedMethodAccessor90.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at jdk.internal.reflect.GeneratedMethodAccessor90.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.util.Vector.forEach(Vector.java:1358) at jdk.internal.reflect.GeneratedMethodAccessor90.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at jdk.internal.reflect.GeneratedMethodAccessor90.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at org.netbeans.core.execution.RunClassThread.run(RunClassThread.java:128) Is there any way NB can be modified to get rid of these failures? Regards Prasanta Note: We have already used test.use.jdk.javac=true permit.jdk9.builds=true showoutput="yes" requires.nb.javac=false in performance.java/project.properties
