Whew! What a trip this bug has been! Danny was exactly right here:
> is exactly the sort of thing I'd expect if two threads were > contending for the same resource, so let's see if the bug has to do > with this. This bug drove me nuts. (it's a short drive) So I started wrapping my file openings in lock.acquire(), lock.release() and nothing seemed to change. Imagine my discouragement. So I knuckled down and began to prepare some code for posting here, documenting, cleaning, generating a pleasant log file, ... While doing this, I became more and more convinced that my code is good. Then it hit me. Maybe I should upgrade to 2.4. Maybe I don't have thread-safety happening in my current situation. Maybe I wasn't planning for threads when I compiled 2.3. The new installation MySQL-python tells me that the mysqlclient library is mostly, but not guaranteed, thread-safe and I should use mysqlclient_s. That didn't exist before. So I bring up the latest of everything and things are much worse. My updates to mysql are ignored without an error. I can't run *anything* Now, at that point, on Thursday night, I was so frustrated and discouraged. I was thinking that I should start over in C, that python is a great teaching language, as originally intended, but for production work, forget it. I was so wrong! I don't know how Charlie, my boss and my son, found this, because I googled and googled and couldn't find it, even after I knew what I was looking for. But, the new MySQL-python has changed its behavior with respect to InnoDB tables. This is some relatively new MySQL table type that allows foreign key constraints -- and upon which we totally depend. Now I must do a commit after each update or insert. Then things started to clear up. Adding locks started to make a difference. It turns out that the shared resource causing all my trouble was, indeed, the operating system's file descriptor generation process. I wrapped the same lock around file openings, socket creations, popen calls, and MySQL connectings and my troubles disappeared. However, when, for curiosity, I ran my stuff under 2.3 again, it worked! So I must have missed it that the first few locks did indeed help my situation. It's easy to miss because if any file-descriptor-dependent operation is left outside the lock, it wrecks havoc in all file-descriptor-dependent operations. So the errors looked the same at first glance. But everything is groovy-to-the-max now. I have my MTA deferring messages because of MySQL's "Too many connections" and the python has stepped out of the performance picture for being so fast, so so fast. Or so it looks today. And thank you Alan, for this reminder: > As was said in a recent thread, mixing file objects and > file descriptors is a bad idea. The OS can get confused, > let alone the programmer! Yeh. Gee. I think I started that thread too. Speaking about confused! Thank you so much again, Alan and Danny and other tutors. Knowing that I have this list to turn to got me through this. Marilyn _______________________________________________ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
