haridsv commented on PR #7256: URL: https://github.com/apache/hbase/pull/7256#issuecomment-3243771617
> One issue i see is since the tests run like a black box it;s hard to know what exactly failed in comparison to before. Wrapping everything in cluster tests makes it a bigger black box reducing observability even further Each test logs the asserts to the output.txt file so I tried to get some insights from it and what I see is that the total number of assertions went up from 1716 to 1764. The prior number of 1716 probably includes a couple of duplicate assertions as I noticed that sftchange_test_cluster.rb while having a standalone Java test class was also getting picked up by TestShell as it was missing from exclusion list. Here is the output from run on master: ``` $ time mvn -Dmaven.test.failure.ignore=true -pl hbase-shell -Pjacoco test -Dtest=TestAdminShell,TestAdminShell2,TestChangeSftShell,TestListTablesShell,TestQuotasShell,TestReplicationShell,TestRSGroupShell,TestShell,TestTableShell,TestShellNoCluster ... [INFO] Tests run: 10, Failures: 0, Errors: 0, Skipped: 0 ... real 9m2.101s user 41m20.080s sys 5m33.809s $ grep -E '\d+ tests, \d+ assertions' ~/src/apache/hbase/hbase-shell/target/surefire-reports/*-output.txt /Users/hdara/src/apache/hbase/hbase-shell/target/surefire-reports/org.apache.hadoop.hbase.client.TestAdminShell-output.txt:92 tests, 235 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications /Users/hdara/src/apache/hbase/hbase-shell/target/surefire-reports/org.apache.hadoop.hbase.client.TestAdminShell2-output.txt:40 tests, 76 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications /Users/hdara/src/apache/hbase/hbase-shell/target/surefire-reports/org.apache.hadoop.hbase.client.TestChangeSftShell-output.txt:2 tests, 2 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications /Users/hdara/src/apache/hbase/hbase-shell/target/surefire-reports/org.apache.hadoop.hbase.client.TestListTablesShell-output.txt:2 tests, 2 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications /Users/hdara/src/apache/hbase/hbase-shell/target/surefire-reports/org.apache.hadoop.hbase.client.TestQuotasShell-output.txt:16 tests, 62 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications /Users/hdara/src/apache/hbase/hbase-shell/target/surefire-reports/org.apache.hadoop.hbase.client.TestReplicationShell-output.txt:34 tests, 354 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications /Users/hdara/src/apache/hbase/hbase-shell/target/surefire-reports/org.apache.hadoop.hbase.client.TestRSGroupShell-output.txt:7 tests, 51 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications /Users/hdara/src/apache/hbase/hbase-shell/target/surefire-reports/org.apache.hadoop.hbase.client.TestShell-output.txt:417 tests, 674 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications /Users/hdara/src/apache/hbase/hbase-shell/target/surefire-reports/org.apache.hadoop.hbase.client.TestTableShell-output.txt:86 tests, 260 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications ``` Output from HBASE-29558 branch: ``` $ time mvn -Dmaven.test.failure.ignore=true -pl hbase-shell -Pjacoco test -Dtest=TestListTablesShell,TestQuotasShell,TestReplicationShell,TestRSGroupShell,TestShell,TestTableShell,TestShellNoCluster ... [INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0 ... real 8m18.859s user 27m40.352s sys 3m41.428s $ grep -E '\d+ tests, \d+ assertions' ~/src/apache/hbase/hbase-shell/target/surefire-reports/*-output.txt /Users/hdara/src/apache/hbase/hbase-shell/target/surefire-reports/org.apache.hadoop.hbase.client.TestListTablesShell-output.txt:2 tests, 2 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications /Users/hdara/src/apache/hbase/hbase-shell/target/surefire-reports/org.apache.hadoop.hbase.client.TestQuotasShell-output.txt:16 tests, 62 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications /Users/hdara/src/apache/hbase/hbase-shell/target/surefire-reports/org.apache.hadoop.hbase.client.TestReplicationShell-output.txt:34 tests, 354 assertions, 0 failu res, 0 errors, 0 pendings, 0 omissions, 0 notifications /Users/hdara/src/apache/hbase/hbase-shell/target/surefire-reports/org.apache.hadoop.hbase.client.TestRSGroupShell-output.txt:7 tests, 51 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications /Users/hdara/src/apache/hbase/hbase-shell/target/surefire-reports/org.apache.hadoop.hbase.client.TestShell-output.txt:549 tests, 985 assertions, 0 failures, 0 err ors, 0 pendings, 0 omissions, 0 notifications /Users/hdara/src/apache/hbase/hbase-shell/target/surefire-reports/org.apache.hadoop.hbase.client.TestShellNoCluster-output.txt:8 tests, 50 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications /Users/hdara/src/apache/hbase/hbase-shell/target/surefire-reports/org.apache.hadoop.hbase.client.TestTableShell-output.txt:86 tests, 260 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications ``` > do you mind posting before and after of number of tests/classes run.? By test classes, I am assuming you are referring to the Ruby files loaded and run. Loaded tests on master (total 20, excluding a duplicate): ``` $ sed -En -e "s/.*loading test file '(.*)'\./\1/p" ~/src/apache/hbase/hbase-shell/target/surefire-reports/*-output.txt admin_test.rb admin2_test.rb sftchange_shell_test.rb list_tables_test.rb quotas_test.rb replication_admin_test.rb rsgroup_shell_test.rb balancer_utils_test.rb hbase_test.rb security_admin_test.rb taskmonitor_test.rb visibility_labels_admin_test.rb commands_test.rb converter_test.rb formatter_test.rb list_locks_test.rb list_procedures_test.rb noninteractive_test.rb sftchange_shell_test.rb shell_test.rb table_test.rb ``` Loaded tests on HBASE-29558 (total 23): ``` $ gsed -En -e "s/.*loading test file '(.*)'\./\1/p" ~/src/apache/hbase/hbase-shell/target/surefire-reports/*-output.txt list_tables_test.rb quotas_test.rb replication_admin_test.rb rsgroup_shell_test.rb admin2_test_cluster.rb admin_test_cluster.rb balancer_utils_test_cluster.rb hbase_test_cluster.rb security_admin_test_cluster.rb taskmonitor_test_cluster.rb visibility_labels_admin_test_cluster.rb commands_test_cluster.rb converter_test_cluster.rb formatter_test_cluster.rb general_test_cluster.rb list_locks_test_cluster.rb list_procedures_test_cluster.rb noninteractive_test_cluster.rb sftchange_test_cluster.rb connection_test_no_cluster.rb list_regions_test_no_cluster.rb quotas_test_no_cluster.rb table_test.rb ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
