Hi Don, On Thu, Apr 20, 2006 at 02:38:46PM -0700, Don Armstrong wrote: > Package: libemail-valid-perl > Version: 0.15-3 > > While failure to build when a network is not attached is a serious > issue, it's also not ok to build successfully when tests fail, even > those that do not require the network. > > Please separate out the tests that require the network from those that > do, and only run the tests that require the network if the network > (and the hosts tested) actually exist/are reachable.
Do you have some suggestion for a reliable way to test the difference between a failure due to the network being unavailable (even temporarily) and a failure of the test itself to connect to it for some other reason. This seems a bit like the halting problem. I agree it would be better if autobuilds failed if the basic test suite fails, but any test for a remote network service is going to be inherently unreliable -- and 'fails to build because aol is down' doesn't seem like the sort of thing I want to hear about my packages... Even testing for it just a moment prior doesn't really help, though it improves our odds a little. The compromise at present, is the uploader gets to check the buildd logs to see if the test suite actually succeeded suitably. In either case, this is probably something we should bug upstream with, even if that means submitting an improved test suite to them. Because we do want the tests to succeed before it gets to us, and that is the only way to be sure. If people have ideas (and patches :-) on how to improve this usefully it may well be worth pursuing. cheers, Ron -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
