>From "Practices of an Agile Developer" by Venkat Subramaniam and Andy Hunt, c. 2006, page 90:
<quote> You're already writing unit tests to exercise your code. Whenever you modify or refactor your code, you exercise your test cases before you check in the code. All you have to do now is exercise your test cases on each supported platform or environment. If your application is expected to run on different operating systems (MacOS, Linux, Windows, etc.), *** you need to test on all of them *** [My emphasis.]. If you expect your application to work on different versions of the Java Virtual Machine (VM) or .NET common language runtime (CLR), *** you need to test that as well *** [My emphasis.]. </quote> I firmly agree that this -- in principle -- must be done. But how on earth can a solo developer, perhaps hoping to make a buck or two, with very limited resources, accomplish manual (much less continuous, automated) integration testing on all reasonable environment possibilities? Just the possibilities for Windows with their constant, unending (albeit, necessary) string of security updates boggles my imagination! One of the many beauties of Python is its portability across many platforms, but it is not perfect. And surprises can occur, even if it is not Python's fault. I *thought* I was finally finished with supporting Python 2.4 at work, and could start using features available in Python 2.6. While testing some new code where I had starting using "with ..." to take care of closing files, I started receiving erratic crashes. Sure enough the traceback pointed to the sections of code using the "with ..." blocks. Huh?!? After investigation, I found that when my IS and commercial vendor friends installed the second computational server for the planning software we use, they did *not* install the *exact* same environment. The OS version was the same, but not all of the other ancillary software was the same. In this instance, they went backwards with Python and installed Python 2.4 on the new server. Since which server gets used is out of the user's control, sometimes my tests ran on one server, sometimes on the other. And since I am stuck doing manual testing, these kinds of things are especially hard to catch! The only way I will ever be able to automate testing at work will be to write my own custom testing framework that can bridge the "automation gap" between the software we use with its proprietary scripting language which does not support automated testing of any sort and the Python that I find myself using more and more, where I have the potential to automate. But the bottom line is -- at work -- the core data I need is in the proprietary environment and after the needed processing with Python it has to be translated back into the proprietary environment's scripting language and run there. The above was just a single practical example (Rant? Sorry!). But even for stuff I do at home, even with using some complex combination of virtual environments to test all relevant OS's (Assuming I could even afford copies WinXP, WinVista, Win7, Win10, etc., with all their many permutations.), how can automated, continuous integration testing be accomplished *in practice* for the severely resource-constrained solo developer? -- boB _______________________________________________ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
