Edit report at https://bugs.php.net/bug.php?id=60285&edit=1
ID: 60285 Comment by: reeze dot xia at gmail dot com Reported by: tyr...@php.net Summary: setting REPORT_EXIT_STATUS for make test won't work Status: Open Type: Bug Package: Testing related PHP Version: trunk-SVN-2011-11-13 (SVN) Block user comment: N Private report: N New Comment: Yes, I got the same problem, I use Travis to test my extension, but make test didn't report failed test correctly. http://travis-ci.org/#!/reeze/php-sundown/jobs/1670528 That commit ignore the status code, the lastest version add another command after run-test: rm $(top_builddir)/tmp-php.ini; \ This makes it never return exit code. -------------------------- Previous Comments: ------------------------------------------------------------------------ [2011-11-13 18:45:24] tyr...@php.net I just checked, it was introduced way back in 2003: http://svn.php.net/viewvc?view=revision&revision=135844 ------------------------------------------------------------------------ [2011-11-13 18:35:57] tyr...@php.net Description: ------------ when setting up the ci environment, I noticed, that passing the REPORT_EXIT_STATUS environment variable to run-tests.php works, but passing the same thing to make test won't. after some debugging with Hannes, we figured out, that the test target in the Makefile overuses the error suppression operator(http://sunsite.ualberta.ca/Documentation/Gnu/make- 3.79/html_chapter/make_5.html#SEC48). if you check the test target (http://svn.php.net/viewvc/php/php- src/trunk/Makefile.global?view=markup#l88) you can see that the whole target is a one-liner prefixed with - which means that anything can fail in the target, it will still report success. :/ the same pattern is used for many of the targets. Could somebody look into this? Test script: --------------- REPORT_EXIT_STATUS=1 make test; echo $?; Expected result: ---------------- should print non-zero if there are failed tests. Actual result: -------------- always prints 0 ------------------------------------------------------------------------ -- Edit this bug report at https://bugs.php.net/bug.php?id=60285&edit=1
