Hi Eric,
I am sorry I did not state the question clear enough. Let me first say that
tests does not depend on each other, and running with or without –parallel
does not make much of difference.
Here is how I set the tests . This is a Fortran code, it does not return.
ADD_TEST command is a call to CMAKE, which (i) executes the Fortran code, and
(ii) executes a python script, a metrics of how close the solution is to the
reference data. I have a PASS_REGULAR_EXPRESSION property, passRegex,
associated with the test. When a test fails, the output is that the required
regular expression ${passRegex} is not found. A test and the python script are
attached. Please let me know if more information is needed.
Thanks in advance,
Ted
CMAKE_MINIMUM_REQUIRED(VERSION 2.8)
project(project_name Fortran )
enable_testing()
set (SRUN_COMMAND /bin/sh -c "cd ${ CMAKE_CURRENT_BINARY_DIR } &&
$<TARGET_FILE:my_exec> ${PROJECT_NAME}")
add_custom_target(${PROJECT_NAME}_srun COMMAND ${SRUN_COMMAND})
set(TARGET_NAME ${PROJECT_NAME}_serial)
add_test(NAME ${TARGET_NAME}
WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}
COMMAND ${CMAKE_COMMAND}
-D "PRE_DELETE_OUTPUT_FILES:BOOL=true"
-D "OUTPUT_FILES:STRING=${files_to_delete}"
-D "PROGRAM1:STRING=${SRUN_COMMAND}"
-D
"PROGRAM2:STRING=python;${assert_case};${hs_work_file};${hs_gold_file};0"
-P "run_and_assert.cmake")
set(passRegex "CALCULATION TERMINATED BECAUSE STOP TIME ACHIEVED" "ctest:
SUCCESS")
set_property(TEST ${TARGET_NAME}
PROPERTY PASS_REGULAR_EXPRESSION "${passRegex}")
/> cat run_and_assert.cmake
…
EXECUTE_PROCESS(COMMAND ${PROGRAM1}
RESULT_VARIABLE error
ERROR_FILE err.log
OUTPUT_FILE out.log
)
IF(error)
MESSAGE(FATAL_ERROR "Executing \"${PROGRAM1}\" FAIED! error=${error}")
ENDIF()
EXECUTE_PROCESS(COMMAND ${PROGRAM2}
RESULT_VARIABLE status)
IF(status)
MESSAGE(SEND_ERROR "Executing \"${PROGRAM2}\" FAILED! status=${status}")
ENDIF()
From: Eric Noulard [mailto:eric.noul...@gmail.com]
Sent: Wednesday, May 17, 2017 10:54 PM
To: T.Sariyski <tzsari...@gmail.com>
Cc: CMake ML <cmake@cmake.org>
Subject: Re: [CMake] CTest question
We lack information on how your test are failing. If they all timeout then may
be you should setup a proper timeout for the test. The actual execution speed
of a test may depend on the load of your machine or the network (depending on
what the test are actually doing).
This may be even more sensible if your test are run in parallel. Do you run
ctest with --parallel option?
Do your tests depends on each other? If so did you setup test dependencies?
Le 17 mai 2017 10:59 PM, "T.Sariyski" <tzsari...@gmail.com
<mailto:tzsari...@gmail.com> > a écrit :
Hi,
I wonder what is the reason that sometimes ctest ends with numerous ‘Failed’
and ‘Timeout’, which ‘ctest –rerun-failed’ reports ‘Passed’. At times the
number of these false failures is about one third. I guess that the problem is
in the way I set my tests, but I have no clue where to look, or how to ‘debug’
the ctest run.
Any advice is highly appreciated.
Thanks,
Ted
--
Powered by www.kitware.com <http://www.kitware.com>
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Kitware offers various services to support the CMake community. For more
information on each offering, please visit:
CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
--
Powered by www.kitware.com
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Kitware offers various services to support the CMake community. For more
information on each offering, please visit:
CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
