Hi. Fist about the things that look strange now.
21.12.2012, 01:30, "Juan Jose Garcia-Ripoll" <[email protected]>: > Indeed. There are reappearances which are some mysterious and I believe there > is some non-reproducibility of some results. Non-reproducibility is quite rare. If test case uses random values, or written incorrectly. > And there are sometimes when I can hardly interpret the tables generated. > See for > instance http://ecls.sourceforge.net/reports-generated/ecl/ecl-diff-lisp-to-c-20121207231628.html > > which comes right > after http://ecls.sourceforge.net/reports-generated/ecl/ecl-diff-lisp-to-c-20121206231812.html > I have the impression that the code that generates the tables is still > somehow broken. I reviewed this more carefully and now I understand what it is. ECL 5b7258e8 has the only difference ECL ab933fa5 - additional failed test case of usocked. Here are the usocket logs of these ECLs: http://cl-test-grid.appspot.com/blob?key=860701 http://cl-test-grid.appspot.com/blob?key=882267 This table has only single row because it is build by first computing set-exclusive-or between test results of two ECL versions, and then laying-out the result into a table. Each failed test case is are separate result item, therefore set-exclusive-or returns a set of a single element - the failed usocket test case on 5b7258e8. > There are also sometimes when cl-test-grid reports OK and this is not so. > See http://cl-test-grid.appspot.com/blob?key=854116 I believe this is another > source of "regressions" cl-test-grid works correnctly here, that's a problem of this particular system vs Quicklisp. I've build this version of ECL and tried (ql:quickload :buildnode). The result is ;; Error while compiling /home/testgrid/quicklisp/dists/quicklisp/software/slime-20121125-cvs/swank-ecl.lisp: ;; COMPILE-FILE returned NIL. ;; Aborting. ;; (:BUILDNODE) No errors is signaled. This happens because :buildnode depens on :swank. swank.asd uses swank-loader::init, which calls (abort) in case compilation is failed. But ql:quicload installs ABORT restart wich silently aborts the system load, without signalling any errors. I've asked in Quicklisp mailing list how to think about this case: https://groups.google.com/forum/?fromgroups=#!topic/quicklisp/rbyog9Hn7xY Now about the testing procedure. It only makes sence to compare test resutls when we change only in one dimension of environment. If we compare two ECL versions, then we keep the same quicklisp. If we what to compare two quicklisp versions, we consider only the same versions of lisp implementations. When I tested ABCL, I had "base version" as I described previously. When new quicklis is out, I've retested old release, the "base version" and ensured the absence of regressions holds on the new quicklisp too. This means in such testing we need 3 binaries available: - old release - the base version - current head I will try perform this procedure running tests from my machine. Will first compare 12.12.1 with the current head. If you will like how it works, we could then setup it for you. Best regards, - Anton ------------------------------------------------------------------------------ LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial Remotely access PCs and mobile devices and provide instant support Improve your efficiency, and focus on delivering more value-add services Discover what IT Professionals Know. Rescue delivers http://p.sf.net/sfu/logmein_12329d2d _______________________________________________ Ecls-list mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/ecls-list
