> You're already writing unit tests to exercise your code. Whenever you > modify or refactor your code, you exercise your test cases before you > check in the code. All you have to do now is exercise your test cases > on each supported platform or environment. > > If your application is expected to run on different operating systems > (MacOS, Linux, Windows, etc.), *** you need to test on all of them *** > [My emphasis.]. If you expect your application to work on different > versions of the Java Virtual Machine (VM) or .NET common language > runtime (CLR), *** you need to test that as well *** [My emphasis.]. > </quote> > > I firmly agree that this -- in principle -- must be done. But how on > earth can a solo developer, perhaps hoping to make a buck or two, with
I as a solo developer have been unable to achieve this task too; but once my software is closer to complete that is when I look to do integration testing. > very limited resources, accomplish manual (much less continuous, > automated) integration testing on all reasonable environment > possibilities? Just the possibilities for Windows with their > constant, unending (albeit, necessary) string of security updates > boggles my imagination! You can always download the free test versions of windows; that is what I do and then I throw them on a virtual machine so as to not have to suck up that cost. But dealing with what security updates break is a different story all together. > > One of the many beauties of Python is its portability across many > platforms, but it is not perfect. And surprises can occur, even if it > is not Python's fault. I *thought* I was finally finished with > supporting Python 2.4 at work, and could start using features > available in Python 2.6. While testing some new code where I had > starting using "with ..." to take care of closing files, I started > receiving erratic crashes. Sure enough the traceback pointed to the > sections of code using the "with ..." blocks. Huh?!? After > investigation, I found that when my IS and commercial vendor friends > installed the second computational server for the planning software we > use, they did *not* install the *exact* same environment. The OS > version was the same, but not all of the other ancillary software was > the same. In this instance, they went backwards with Python and > installed Python 2.4 on the new server. Since which server gets used > is out of the user's control, sometimes my tests ran on one server, > sometimes on the other. And since I am stuck doing manual testing, > these kinds of things are especially hard to catch! The only way I > will ever be able to automate testing at work will be to write my own > custom testing framework that can bridge the "automation gap" between > the software we use with its proprietary scripting language which does > not support automated testing of any sort and the Python that I find > myself using more and more, where I have the potential to automate. > But the bottom line is -- at work -- the core data I need is in the > proprietary environment and after the needed processing with Python it > has to be translated back into the proprietary environment's scripting > language and run there. Yes in my virtual environments I have over time been diligently writing a testing automation tool; that starts when the OS boots; then looks to a specific server for a specific directory to download the code to be tested. But it is very rough; the key is to start small and write code that can be re-used. But I am still missing an android and IOS client so I have to test those manually. > > The above was just a single practical example (Rant? Sorry!). But > even for stuff I do at home, even with using some complex combination > of virtual environments to test all relevant OS's (Assuming I could > even afford copies WinXP, WinVista, Win7, Win10, etc., with all their > many permutations.), how can automated, continuous integration testing > be accomplished *in practice* for the severely resource-constrained > solo developer? Good Luck! _______________________________________________ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
