Reading binary data
OK so here is my task. I want to get at the data stored in /var/account/pacct, which stores process accounting data, so that I can make it into a more human understandable format then what the program sa can do. The thing is, its in a binary format and an example program that reads some data from the file is done in C using a struct defined in sys/acct.h. http://www.linuxjournal.com/articles/lj/0104/6144/6144l2.html So I was wondering how can I do the same thing, but in python? I'm still learning so please be gentle. David -- http://mail.python.org/mailman/listinfo/python-list
Re: Reading binary data
Thanks but the C Struct describing the data doesn't match up with the
list on the module-struct page.
this is the acct.h file
#ifndef _SYS_ACCT_H
#define _SYS_ACCT_H 1
#include
#define __need_time_t
#include
#include
__BEGIN_DECLS
#define ACCT_COMM 16
/*
comp_t is a 16-bit "floating" point number with a 3-bit base 8
exponent and a 13-bit fraction. See linux/kernel/acct.c for the
specific encoding system used.
*/
typedef u_int16_t comp_t;
struct acct
{
char ac_flag; /* Accounting flags. */
u_int16_t ac_uid; /* Accounting user ID. */
u_int16_t ac_gid; /* Accounting group ID. */
u_int16_t ac_tty; /* Controlling tty. */
u_int32_t ac_btime; /* Beginning time. */
comp_t ac_utime;/* Accounting user time. */
comp_t ac_stime;/* Accounting system time. */
comp_t ac_etime;/* Accounting elapsed time. */
comp_t ac_mem; /* Accounting average memory usage. */
comp_t ac_io; /* Accounting chars transferred. */
comp_t ac_rw; /* Accounting blocks read or written.
*/
comp_t ac_minflt; /* Accounting minor pagefaults. */
comp_t ac_majflt; /* Accounting major pagefaults. */
comp_t ac_swaps;/* Accounting number of swaps. */
u_int32_t ac_exitcode; /* Accounting process exitcode. */
char ac_comm[ACCT_COMM+1]; /* Accounting command name. */
char ac_pad[10];/* Accounting padding bytes. */
};
enum
{
AFORK = 0x01, /* Has executed fork, but no exec. */
ASU = 0x02, /* Used super-user privileges. */
ACORE = 0x08, /* Dumped core. */
AXSIG = 0x10/* Killed by a signal. */
};
#define AHZ 100
/* Switch process accounting on and off. */
extern int acct (__const char *__filename) __THROW;
__END_DECLS
#endif /* sys/acct.h */
What are u_ini16_t and comp_t? And what about the enum section?
--
http://mail.python.org/mailman/listinfo/python-list
Re: Understanding other people's code
> Literally any idea will help, pen and paper, printing off all the code and doing some sort of highlighting session - anything! > I keep reading bits of code and thinking "well where the hell has that been defined and what does it mean" to find it was inherited from 3 modules up the chain. > I really need to get a handle on how exactly all this slots together! Any techniques,tricks or methodologies that people find useful would be much appreciated. I'd highly recommend Eclipse with PyDev, unless you have some strong reason not to. That's what I use, and it saves pretty much all of those "what's this thing?" problems, as well as lots of others... DC -- http://mail.python.org/mailman/listinfo/python-list
how: embed + extend to control my running app?
i'd like my app to be "available" to python while it's running. for example, say my app is "FooBar.app". when my FooBar.app is running, now there is a python interface available to python, and the user can write python scripts to make use of it. with their scripts, they can control my running application when FooBar.app is NOT running, perhaps making use of any of the python functions of "FooBar.app" would either return an error, or possibly launch "FooBar.app"? or do nothing since it's not running? can boost::python help with this? i've never worked with extending or embedding python, so any help would be super great -- http://mail.python.org/mailman/listinfo/python-list
Re: how: embed + extend to control my running app?
i'm targeting Mac and Windows. Let's skip the thing about "it should work when my app isn't running", just assume it's going to be embedded, no pipes or sockets necessary. For Mac, I understand i need to "create" (?) a python.dylib, but i find no directions for that at the expected location: http://docs.python.org/2/extending/embedding.html is there some wiki page explaining how to create this for use in MacOS / Xcode? Now for Windows: same thing, i think i must create a .dll, right? Is there a tutorial for that? After that, i can link to these items, then in my C++ app, just #include "Python.h" and i've covered step 1. -- http://mail.python.org/mailman/listinfo/python-list
Re: how: embed + extend to control my running app?
well, umm, gosh, now i feel quite silly. that was easy. okay that's done.
next: i'd like to redirect the output of any "print" statements to my C
function:
> voidLog(const unsigned char *utf8_cstrP);
on the mac, python output sys.stdout goes into the debug console if you're in
the debugger, and to the "console app" if not. On windows, i don't think it
goes anywhere at all?
So: i really want it to go to my own log file (via my Log() function). now,
can i specify "please output to this FILE*" ?, i looked at all the python c
headers but found nothing about redirecting the output.
I see "PySys_GetFile()" which will get what it points to, but what i want is a
"PySys_SetFile()" so i can set it.
the only alternative seems to be:
> PyObject*logObjectP = create ???;
>
> ERR(PySys_SetObject("stdout", logObjectP));
if that's the only way, how to create the logObjectP such that it redirects the
write() python function to my Log() C function?
i tried this:
const char *s_printFunc =
"import sys\n"
"class CustomPrint():\n"
" def __init__(self):\n"
" self.old_stdout=sys.stdout\n"
"\n"
" def write(self, text):\n"
" self.old_stdout.write('foobar')\n"
" text = text.rstrip()\n"
" if len(text) == 0:\n"
" return\n"
" self.old_stdout.write('custom Print--->' + text +
'\n')\n";
OSStatusCPython_PreAlloc(const char *utf8Z)
{
OSStatuserr = noErr;
PyCompilerFlags flags;
PyObject*logObjectP = NULL;
Py_SetProgramName(const_cast(utf8Z));
Py_Initialize();
flags.cf_flags = PyCF_SOURCE_IS_UTF8;
logObjectP = Py_CompileStringFlags(s_printFunc,
"CustomPrint", Py_single_input, &flags);
ERR_NULL(logObjectP, tsmUnsupScriptLanguageErr);
if (!err) {
ERR(PySys_SetObject("stdout", logObjectP));
ERR(PySys_SetObject("stderr", logObjectP));
Py_DECREF(logObjectP);
}
return err;
}
voidCPython_PostDispose()
{
Py_Finalize();
}
voidCPython_Test()
{
PyRun_SimpleString(
"from time import time, ctime\n"
"print 'Today is', ctime(time())\n");
}
-
and when i run CPython_Test(), there is no output at all. If i comment out the
entire Py_CompileStringFlags() line, then the output works fine (going to
stdout as expected), so i'm not sure what i'm doing wrong
--
http://mail.python.org/mailman/listinfo/python-list
Re: how: embed + extend to control my running app?
i don't use stdout in my own code, my code goes to my own log file. i want the output from any python code to go to my existing log file, so log statements from my app and any python code are intermingled in that one file. my updated code is here, which now bridges my python print function to my C function: http://karaoke.kjams.com/wiki/Python but it seems that my custom "s_printFunc" is never called ? -- http://mail.python.org/mailman/listinfo/python-list
Re: how: embed + extend to control my running app?
> http://karaoke.kjams.com/wiki/Python nevermind, i got it, it's working now (see link for code) -- http://mail.python.org/mailman/listinfo/python-list
Re: how: embed + extend to control my running app?
> > Now for Windows: same thing, i think i must create a .dll, right? > you should already have a python.dll in your installation i can find "python27.lib" in the "libs" folder, but there is no "python27_d.lib", and there is no "python27.dll" in the DLLs folder? are there instructions for creating (or finding) these for Windows? -- http://mail.python.org/mailman/listinfo/python-list
Re: how: embed + extend to control my running app?
update: okay so the python27.dll is in /windows/system32 so ignore that i've set my include directory correct, so i can compile i've set my "additional libraries" directory to the "libs" directory (where the ".lib" files are. (note: NOT including "Lib" directory, cuz that's full of .py files and folders) (note: NOT including "DLLs" directory, cuz, why would i?) No need to specify "additional dependencies" for the .lib file, cuz the pyconfig.h file does that. but there is no "python27_d.dll" anywhere to be found, so i hacked pyconfig.h to get rid of the "_d". so it all compiles. but it won't link: LNK2001: unresolved external symbol __imp___Py_RefTotal LNK2001: unresolved external symbol __imp___Py_NoneStruct LNK2019: unresolved external symbol __imp__PyArg_ParseTuple LNK2019: unresolved external symbol __imp__PyFloat_FromDouble LNK2019: unresolved external symbol __imp__PyString_FromString LNK2019: unresolved external symbol __imp__PyRun_SimpleStringFlags LNK2019: unresolved external symbol __imp__Py_InitModule4TraceRefs LNK2019: unresolved external symbol __imp__Py_Initialize LNK2019: unresolved external symbol __imp__Py_SetProgramName LNK2019: unresolved external symbol __imp__Py_Finalize LNK2019: unresolved external symbol __imp__PyRun_SimpleFileExFlags what, pray tell, am i doing wrong? *hopeful face* -- http://mail.python.org/mailman/listinfo/python-list
Re: how: embed + extend to control my running app?
Okay the link problem was solved: i had installed a 64bit python and my app is 32bit. i'm using ActivePython installer from here: http://www.activestate.com/activepython/downloads it seems that now the problem is that this does not install the _d versions of the .lib. :( does anyone know how to get or create the _d version of the .lib out of the ActivePtyon installation? -- http://mail.python.org/mailman/listinfo/python-list
how to package embedded python?
what must i include in my app package if i'm embedding python? i tried including *everything* in the "DLLs" directory, but my app still crashes as soon as i attempt to initialize python. this is on a system that does not have python installed, as most of my users won't have it. is it actually a requirement that they first install python? (cuz it does work then) -- http://mail.python.org/mailman/listinfo/python-list
embedded python and threading
in my app i initialize python on the main thread, then immediately call PyEval_SaveThread() because i do no further python stuff on the main thread. then, for each script i want to run, i use boost::threads to create a new thread, then on that thread i "ensure" the GIL, do my stuff, then release it. so, to test concurrency, on my first background thread, i do an infinite loop that just logs "i'm alive", then calls sleep(0.25) so that thread continues to run forever (with it's GIL ensured) according to the doc: "In order to emulate concurrency of execution, the interpreter regularly tries to switch threads" so i figure i can run another thread that does a single print statement: > ensure gil > print my thing > release gil and this DOES run. however, after releasing it's gil, i guess the interpeter gets back to the first back thread, but then has this error immediately: 9: Traceback (most recent call last): 9: File "", line 70, in ? 9: File "", line 55, in main 9: AttributeError: 'builtin_function_or_method' object has no attribute 'sleep' suddenly the sleep module has been unloaded?? huh? i thought the thread state had been preserved? -- http://mail.python.org/mailman/listinfo/python-list
Re: how to package embedded python?
does nobody know how to do this? does nobody know where proper documentation on this is? -- http://mail.python.org/mailman/listinfo/python-list
Re: embedded python and threading
okay, i have simplified it: here is the code == import time def main(): while True: print "i'm alive" time.sleep(0.25) #- if __name__ == "__main__": main() == the new error is: == 9: Traceback (most recent call last): 9: File "", line 10, in ? 9: File "", line 6, in main 9: AttributeError: 'builtin_function_or_method' object has no attribute 'sleep' == -- http://mail.python.org/mailman/listinfo/python-list
Re: embedded python and threading
no, there is no "time.py" anywhere (except perhaps as the actual python library originally imported) did you understand that the function works perfectly, looping as it should, up until the time i run a second script on a separate thread? -- http://mail.python.org/mailman/listinfo/python-list
Re: embedded python and threading
DOH! as my second thread, i had been using a sample script that i had copy-pasted without much looking at it. guess what? it prints the time. and yes, it did "from time import time", which explains it all. thanks for the hints here, that helped me figure it out! -- http://mail.python.org/mailman/listinfo/python-list
embedding: how to create an "idle" handler to allow user to kill scripts?
in my C++ app, on the main thread i init python, init threads, then call
PyEval_SaveThread(), since i'm not going to do any more python on the main
thread.
then when the user invokes a script, i launch a preemptive thread
(boost::threads), and from there, i have this:
static int CB_S_Idle(void *in_thiz) {
CT_RunScript*thiz((CT_RunScript *)in_thiz);
return thiz->Idle();
}
int Idle()
{
int resultI = 0;
OSStatuserr = noErr;
ERR(i_taskRecP->MT_UpdateData(&i_progData));
if (err) {
resultI = -1;
}
ERR(ScheduleIdleCall());
return err;
}
int ScheduleIdleCall()
{
int resultI(Py_AddPendingCall(CB_S_Idle, this));
CFAbsoluteTime timeT(CFAbsoluteTimeGetCurrent());
SuperString str; str.Set(timeT, SS_Time_LOG);
Logf("$$$ Python idle: (%d) %s\n", resultI, str.utf8Z());
return resultI;
}
virtual OSStatusoperator()(OSStatus err) {
ScPyGILStatesc;
ERR(ScheduleIdleCall());
ERR(PyRun_SimpleString(i_script.utf8Z()));
return err;
}
so, my operator() gets called, and i try to schedule an Idle call, which
succeeds, then i run my script. however, the CB_S_Idle() never gets called?
the MT_UpdateData() function returns an error if the user had canceled the
script
must i schedule a run-loop on the main thread or something to get it to be
called?
--
http://mail.python.org/mailman/listinfo/python-list
Re: how to package embedded python?
nooobody knw the trouble a s... -- http://mail.python.org/mailman/listinfo/python-list
Re: embedding: how to create an "idle" handler to allow user to kill scripts?
Okay, i'm really surprised nobody knows how to do this. and frankly i'm amazed at the utter lack of documentation. but i've figured it out, and it's all working beautifully. if you want the code, go here: http://karaoke.kjams.com/wiki/Python -- http://mail.python.org/mailman/listinfo/python-list
Re: how to package embedded python?
yes, i've looked there, and all over google. i'm quite expert at embedding at this point. however nowhere i have looked has had instructions for "this this is how you package up your .exe with all the necessary python modules necessary to actually run on a user's system that does not have python installed". on mac, it's trivial: all macs come with python, there is nothing i need to include with my app and it "just works" on windows: if you don't include the proper DLLs and/or whatnot, then the app will complain about missing DLLs on startup. What DLLs must i include? where are the instructions? -- http://mail.python.org/mailman/listinfo/python-list
Re: how to package embedded python?
okay, well that might turn out to be useful, except i don't quite know how to use it, and there are no "from scratch" instructions. i managed to download "py2exe-0.6.9.zip" and unzip it, but how does one "install" this package? (yes, still a newb at that) then, once installed, how do i say "include the entire world" instead of just "mymodule" ? cuz the point of embedding python on my app is that the end-user can run any script at all, not just one module. -- http://mail.python.org/mailman/listinfo/python-list
Getting a TimedRotatingFileHandler not to put two dates in the same file?
We have a TimedRotatingFileHandler with when='midnight'. This works great, splitting the log information across files by date, as long as the process is actually up at midnight. But now the users have noticed that if the process isn't up at midnight, they can end up with lines from two (or I guess potentially more) dates in the same log file. Is there some way to fix this, either with cleverer arguments into the TimedRotatingFileHandler, or by some plausible subclassing of it or its superclass? Or am I misinterpreting the symptoms somehow? Tx much! DC -- http://mail.python.org/mailman/listinfo/python-list
Re: problem with ThreadingTCPServer Handler
> jorge > I'm programming a server that most send a message to each client > connected to it and nothing else. this is obviously a base of what i > want to do. the thing is, I made a class wich contains the Handler class > for the ThreadingTCPServer and starts the server but i don't know how > can i access the message variable contained in the class from the > Handler since I have not to instance the Handler by myself. The information about the request is in attributes of the Handler object. >From the socketserver docs ( http://docs.python.org/library/socketserver.html ): RequestHandler.handle() This function must do all the work required to service a request. The default implementation does nothing. Several instance attributes are available to it; the request is available as self.request; the client address asself.client_address; and the server instance as self.server, in case it needs access to per-server information. If that's what you meant by "the message variable contained in the class". If, on the other hand, you meant that you want to pass some specific data into the handler about what it's supposed to be doing, I've generally stashed that in the server, since the handler can see the server via self.server. DC -- http://mail.python.org/mailman/listinfo/python-list
Re: Getting a TimedRotatingFileHandler not to put two dates in the same file?
> [email protected] > > On 10/23/2012 11:23 AM, David M Chess wrote: > > We have a TimedRotatingFileHandler with when='midnight' > > You give us no clue what's in this class, or how it comes up with the > filenames used. Sorry if I was unclear. This isn't my own subclass of TimedRotatingFileHandler or anything, this is the bog-standard logging.handlers.TimedRotatingFileHandler I'm talking about. So all clues about what's in the class, and how it comes up with the filenames used, is available at http://docs.python.org/library/logging.handlers.html#timedrotatingfilehandler :) The specific Python version involved here is Python 2.6.6 (r266:84297, Aug 24 2010, 18:46:32), to the extent that that matters... >> This works great, splitting the log information across files by date, as >> long as the process is actually up at midnight. >> >> But now the users have noticed that if the process isn't up at midnight, >> they can end up with lines from two (or I guess potentially more) dates in >> the same log file. >> >> Is there some way to fix this, either with cleverer arguments into the >> TimedRotatingFileHandler, or by some plausible subclassing of it or its >> superclass? Tx, DC http://mail.python.org/mailman/listinfo/python-list
Re: Getting a TimedRotatingFileHandler not to put two dates in the same file?
> [email protected] > Something like: > Does a log file exist? -> No -> First run; create log file & continue > | > Yes > | > Read backwards looking for date change, copy lines after change > to new file, delete from old file. Yep, I'm concluding that also. It just wasn't clear to me from the documentation whether or not the existing TimedRotatingFileHandler had any "at startup, see if we missed any rollovers, and do them now if so" function, or if there was some known variant that does. The answer, apparently, being "nope". :) Shouldn't be that hard to write, so that's probably what we'll do. DC --- -- http://mail.python.org/mailman/listinfo/python-list
A lock that prioritizes acquire()s?
Okay, next silly question. :)
We have a very simple multi-threaded system where a request comes in,
starts running in a thread, and then (zero, one, or two times per request)
gets to a serialization point, where the code does:
with lock:
do_critical_section_stuff_that_might_take_awhile()
and then continues.
Which is almost the same as:
lock.acquire()
try:
do_critical_section_stuff_that_might_take_awhile()
finally:
lock.release()
Now we discover that It Would Be Nice if some requests got priority over
others, as in something like:
lock.acquire(importance=request.importance)
try:
do_critical_section_stuff_that_might_take_awhile()
finally:
lock.release()
and when lock.release() occurs, the next thread that gets to run is one of
the most important ones currently waiting in acquire() (that's the
exciting new thing).
Other requirements are that the code to do this be as simple as possible,
and that it not mess anything else up. :)
My first thought was something like a new lock-ish class that would do
roughly:
class PriorityLock(object):
def __init__(self):
self._lock = threading.Lock()
self._waiter_map = {} # maps TIDs to importance
def acquire(self,importance=0):
this_thread = threading.currentThread()
self._waiter_map[this_thread] = importance # I want in
while True:
self._lock.acquire()
if ( max( self._waiter_map.values())<=importance ): # we win
del self._waiter_map[this_thread] # not waiting anymore
return # return with lock acquired
self._lock.release() # We are not most impt: release/retry
def release(self):
self._lock.release()
(Hope the mail doesn't garble that too badly.)
Basically the acquire() method just immediately releases and tries again
if it finds that someone more important is waiting.
I think this is semantically correct, as long as the underlying lock
implementation doesn't have starvation issues, and it's nice and simple,
but on the other hand it looks eyerollingly inefficient.
Seeking any thoughts on other/better ways to do this, or whether the
inefficiency will be too eyerolling if we get say one request per second
with an average service time a bit under a second but maximum service time
well over a second, and most of them are importance zero, but every (many)
seconds there will be one or two with higher importance.
Tx,
DC
---
--
http://mail.python.org/mailman/listinfo/python-list
Re: A lock that prioritizes acquire()s?
Lovely, thanks for the ideas! I remember considering having release() pick the next thread to notify, where all the waiters were sitting on separate Conditions or whatever; not sure why I didn't pursue it to the end. Probably distracted by something shiny; or insufficient brainpower. :) DC -- -- http://mail.python.org/mailman/listinfo/python-list
bus errors when the network interface is reset?
We have a system running Python 2.6.6 under RHEL 6.1. A bunch of processes spend most of their time sitting in a BaseHTTPServer.HTTPServer waiting for requests. Last night an update pushed out via xcat whimsically restarted all of the network interfaces, and at least some of our processes died with bus errors (i.e. no errors or exceptions reflected up to the Python level, just a crash). This is just my initial looking into this. Seeking opinions of the form, say: Yeah, that happens, don't reset the network interfaces. Yeah, that happens, and you can prevent the crash by doing X in your OS. Yeah, that happens, and you can prevent the crash by doing X in your Python code. That wouldn't happen if you upgraded S to version V. That sounds like a new bug and/or more information is needed; please provide copious details including at least X, Y, and Z. Any thoughts or advice greatly appreciated. DC David M. Chess IBM Watson Research Center -- http://mail.python.org/mailman/listinfo/python-list
Re: Py3.3 unicode literal and input()
> If you (the programmer) want a function that asks the user to enter a > literal at the input prompt, you'll have to write a post-processing for > it, which looks for prefixes, for quotes, for backslashes, etc., and > encodes the result. There very well may be such a decoder in the Python > library, but input does nothing of the kind. As it says at the end of eval() (which you definitely don't want to use here due to side effects): See ast.literal_eval() for a function that can safely evaluate strings with expressions containing only literals. DC -- http://mail.python.org/mailman/listinfo/python-list
Re: authentication for xmlrpc via cgi
[EMAIL PROTECTED] writes: > I'm using python 2.2 (hopefully we'll be upgrading our system to 2.3 > soon) and I'm trying to prototype some xml-rpc via cgi functionality. > If I override the Transport class on the xmlrpclib client and add some > random header like "Junk", then when I have my xmlrpc server log it's > environment when running, I see the HTTP_JUNK header. If I do this > with AUTHORIZATION, the header is not found. > > Does this ring a bell for anyone? Am I misunderstanding how to use > this header? I'm guessing that Apache might be eating this header, but > I don't know why. By default, Apache does eat that. It's a compile time default; the Apache developers think it's a security hole. Here's a note about it: http://httpd.apache.org/dev/apidoc/apidoc_SECURITY_HOLE_PASS_AUTHORIZATION.html >From what I can see, this is still true in Apache 2. -- |>|\/|< /--\ |David M. Cooke |cookedm(at)physics(dot)mcmaster(dot)ca -- http://mail.python.org/mailman/listinfo/python-list
Re: Underscores in Python numbers
Peter Hansen <[EMAIL PROTECTED]> writes: > Steven D'Aprano wrote: >> Dealing with numeric literals with lots of digits is >> a real (if not earth-shattering) human interface problem: it is hard for >> people to parse long numeric strings. > > I'm totally unconvinced that this _is_ a real problem, if we define > "real" as being even enough to jiggle my mouse, let alone shattering the > planet. > > What examples does anyone have of where it is necessary to define a > large number of large numeric literals? Isn't it the case that other > than the odd constants in various programs, defining a large number of > such values would be better done by creating a data file and parsing > it? One example I can think of is a large number of float constants used for some math routine. In that case they usually be a full 16 or 17 digits. It'd be handy in that case to split into smaller groups to make it easier to match with tables where these constants may come from. Ex: def sinxx(x): "computes sin x/x for 0 <= x <= pi/2 to 2e-9" a2 = -0.1 4 a4 = 0.00833 33315 a6 = -0.00019 84090 a8 = 0.0 27526 a10= -0.0 00239 x2 = x**2 return 1. + x2*(a2 + x2*(a4 + x2*(a6 + x2*(a8 + x2*a10 (or least that's what I like to write). Now, if I were going to higher precision, I'd have more digits of course. -- |>|\/|< /--\ |David M. Cooke |cookedm(at)physics(dot)mcmaster(dot)ca -- http://mail.python.org/mailman/listinfo/python-list
Re: idea of building python module using pyrex
"[EMAIL PROTECTED]" <[EMAIL PROTECTED]> writes: > For example stastical module like commulative probability function for > t distribution, or other numerical module which incorporate looping to > get the result. > > I found that pyrex is very helpfull when dealing with looping > things. Pyrex is indeed quite helpful. If you're interested in statistical distributions, you'll want to look at the scipy.stats module in scipy (http://www.scipy.org/), which has lots (including the t distribution). In SciPy, we use Pyrex for the random-number generator module scipy.random. It's actually used to wrap some C code, but it does the job well. -- |>|\/|< /------\ |David M. Cooke |cookedm(at)physics(dot)mcmaster(dot)ca -- http://mail.python.org/mailman/listinfo/python-list
Re: date/time
Thomas Guettler <[EMAIL PROTECTED]> writes:
> Am Wed, 05 Jan 2005 15:08:37 +0100 schrieb Nader Emami:
>
>> L.S.,
>>
>> Could somebody help me how I can get the next format of date
>> from the time module?
>
> I don't understand your question. Do you want to have the next day?
>
> 20041231 --> 20050101 ?
>
> You can do it like this:
> - parse the string with time.strptime
> - timetuple[2]+=1
> - mktime(timetuple) # --> secs
> - strftime(localtime(secs))
Or using the datetime module:
import time, datetime
tt = time.strptime('20041231', '%Y%m%d')
t = datetime.date.fromtimestamp(time.mktime(tt))
# now in a easier-to-handle form than the time tuple
t += datetime.timedelta(days=1)
print t.strftime('%Y%m%d')
--
|>|\/|<
/--\
|David M. Cooke
|cookedm(at)physics(dot)mcmaster(dot)ca
--
http://mail.python.org/mailman/listinfo/python-list
Re: why not datetime.strptime() ?
Joshua Spoerri <[EMAIL PROTECTED]> writes:
> Skip Montanaro pobox.com> writes:
>> josh> Shouldn't datetime have strptime?
>> If someone wants to get their feet wet with extension module
>> programming
>> this might be a good place to start. Mostly, I think nobody who has
>> needed/wanted it so far has the round tuits available to spend on the
>> task.
>
> OK, it was pretty straightforward. Thanks for the direction.
>
> To whom should I send the patch (attached)?
Submit it to the patch tracker on sourceforge.
But first, some constructive criticism:
> --- Modules/datetimemodule.c.orig 2003-10-20 10:34:46.0 -0400
> +++ Modules/datetimemodule.c 2005-01-10 20:58:38.884823296 -0500
> @@ -3774,6 +3774,32 @@
> return result;
> }
>
> +/* Return new datetime from time.strptime(). */
> +static PyObject *
> +datetime_strptime(PyObject *cls, PyObject *args)
> +{
> + PyObject *result = NULL, *obj, *module;
> + const char *string, *format;
> +
> + if (!PyArg_ParseTuple(args, "ss:strptime", &string, &format))
> + return NULL;
> + if ((module = PyImport_ImportModule("time")) == NULL)
> + return NULL;
> + obj = PyObject_CallMethod(module, "strptime", "ss", string, format);
> + Py_DECREF(module);
You don't check for errors: an exception being thrown by
PyObject_CallMethod will return obj == NULL.
If there's a module in sys.path called time that overrides the stdlib
time, things will fail, and you should be able to catch that.
> + result = PyObject_CallFunction(cls, "iii",
> + PyInt_AsLong(PySequence_GetItem(obj, 0)),
> + PyInt_AsLong(PySequence_GetItem(obj, 1)),
> + PyInt_AsLong(PySequence_GetItem(obj, 2)),
> + PyInt_AsLong(PySequence_GetItem(obj, 3)),
> + PyInt_AsLong(PySequence_GetItem(obj, 4)),
> + PyInt_AsLong(PySequence_GetItem(obj, 5)),
> + PyInt_AsLong(PySequence_GetItem(obj, 6)));
Are you positive those PySequence_GetItem calls will succeed? That
they will return Python integers?
> + Py_DECREF(obj);
> + return result;
> +}
> +
> /* Return new datetime from date/datetime and time arguments. */
> static PyObject *
> datetime_combine(PyObject *cls, PyObject *args, PyObject *kw)
> @@ -4385,6 +4411,11 @@
>PyDoc_STR("timestamp -> UTC datetime from a POSIX timestamp "
> "(like time.time()).")},
>
> + {"strptime", (PyCFunction)datetime_strptime,
> + METH_VARARGS | METH_CLASS,
> + PyDoc_STR("strptime -> new datetime parsed from a string"
> +"(like time.strptime()).")},
> +
> {"combine", (PyCFunction)datetime_combine,
>METH_VARARGS | METH_KEYWORDS | METH_CLASS,
>PyDoc_STR("date, time -> datetime with same date and time fields")},
It probably would help to add some documentation to add to the
datetime module documentation.
--
|>|\/|<
/--\
|David M. Cooke
|cookedm(at)physics(dot)mcmaster(dot)ca
--
http://mail.python.org/mailman/listinfo/python-list
Re: Python3: on removing map, reduce, filter
Steven Bethard <[EMAIL PROTECTED]> writes: > Some timings to verify this: > > $ python -m timeit -s "def square(x): return x*x" "map(square, range(1000))" > 1000 loops, best of 3: 693 usec per loop > > $ python -m timeit -s "[x*x for x in range(1000)]" > 1000 loops, best of 3: 0.0505 usec per loop Maybe you should compare apples with apples, instead of oranges :-) You're only running the list comprehension in the setup code... $ python2.4 -m timeit -s "def square(x): return x*x" "map(square, range(1000))" 1000 loops, best of 3: 464 usec per loop $ python2.4 -m timeit "[x*x for x in range(1000)]" 1000 loops, best of 3: 216 usec per loop So factor of 2, instead of 13700 ... -- |>|\/|< /--\ |David M. Cooke |cookedm(at)physics(dot)mcmaster(dot)ca -- http://mail.python.org/mailman/listinfo/python-list
Re: readline, rlcompleter
[EMAIL PROTECTED] writes:
> This a case where the documentation is lacking. The standard library
> documentation
> (http://www.python.org/dev/doc/devel/lib/module-rlcompleter.html) gives
> this example
> try:
> import readline
> except ImportError:
> print "Module readline not available."
> else:
> import rlcompleter
> readline.parse_and_bind("tab: complete")
>
> but I don't find a list of recognized key bindings. For instance, can I
> would
> like to bind shift-tab to rlcompleter, is that possible? Can I use
> function
> keys? I did various attempt, but I did not succed :-(
> Is there any readline-guru here with some good pointers?
> Michele Simionato
Basically, you could use any key sequence that is sent to the terminal. So
shift-tab is out (that's not sent as a key to any terminal program).
Function keys would have to be specified as the key sequence sent by a
function key ("\e[11~" for F1, for instance).
Have a look at the readline info page, or the man page. The syntax of
readline.parse_and_bind is the same as that of an inputrc file.
--
|>|\/|<
/--\
|David M. Cooke
|cookedm(at)physics(dot)mcmaster(dot)ca
--
http://mail.python.org/mailman/listinfo/python-list
Re: OT: MoinMoin and Mediawiki?
Paul Rubin <http://[EMAIL PROTECTED]> writes: > Alexander Schremmer <[EMAIL PROTECTED]> writes: >> > lists of incoming links to wiki pages, >> >> It does. > > Huh? I don't see those. How does it store them, that's resilient > across crashes? Or does it just get wedged if there's a crash? Most Wiki implementations (MoinMoin included) have this, by using a search. Usually, following the original Wiki (http://c2.com/cgi/wiki) model, you get at it by clicking on the title of the page. Searching instead of indexing makes it very resilient :-) -- |>|\/|< /------\ |David M. Cooke |cookedm(at)physics(dot)mcmaster(dot)ca -- http://mail.python.org/mailman/listinfo/python-list
Re: Pyrex-0.9.3: definition mismatch with distutils of Python24
[EMAIL PROTECTED] (Martin Bless) writes: > Now that I've got my extension building machine using the VC++ Toolkit > 2003 up and running I'm keen on using Pyrex (Pyrex-0.9.3, > Python-2.4.0). > > But the definition of the swig_sources() method seems to have changed. > > When I try to build the examples from Pyrex I get a TypeError: > > > c:\Pyrex-0.9.3\Demos> python Setup.py build_ext --inplace > running build_ext > building 'primes' extension > [...] > File "C:\Python24\lib\distutils\command\build_ext.py", line 442, in > build_extension > sources = self.swig_sources(sources, ext) > TypeError: swig_sources() takes exactly 2 arguments (3 given) > > > I can see that Pyrex.Distutils.build_ext.py subclasses > distutils.command.build_ext.build_ext, and the number of arguments of > the swig_sources method seems to have changed. > > Pyrex uses: > > def swig_sources (self, sources): > > whereas the distutils use: > > def swig_sources (self, sources, extension): > > If I just add the "extension" arg to the Pyrex definitions everything > seems to work. But I have to admit that I don't really know what I'm > doing here and I feel sorry I can't contribute more than just > reporting the error. Yep, that's it. Greg must know now, it's been reported a few times. You'll want to change it to def swig_sources(self, sources, extension=None): so that if you use an older python it won't complain about missing arguments. -- |>|\/|< /--\ |David M. Cooke |cookedm(at)physics(dot)mcmaster(dot)ca -- http://mail.python.org/mailman/listinfo/python-list
Re: pickling extension class
harold fellermann <[EMAIL PROTECTED]> writes:
> Hi all,
>
> I have a problem pickling an extension class. As written in the
> Extending/Embedding Manual, I
> provided a function __reduce__ that returns the appropreate tuple.
> This seams to work fine,
> but I still cannot pickle because of the following error:
>
> >>> from model import hyper
> >>> g = hyper.PeriodicGrid(4,4,1)
> >>> g.__reduce__()
> (,(4.,4.,1.))
> >>> import pickle
> >>> pickle.dump(g,file("test","w"))
> Traceback (most recent call last):
>File "pickle_test.py", line 5, in ?
> pickle.dump(g,file("test","w"))
>File "/sw/lib/python2.4/pickle.py", line 1382, in dump
> Pickler(file, protocol, bin).dump(obj)
>File "/sw/lib/python2.4/pickle.py", line 231, in dump
> self.save(obj)
>File "/sw/lib/python2.4/pickle.py", line 338, in save
> self.save_reduce(obj=obj, *rv)
>File "/sw/lib/python2.4/pickle.py", line 414, in save_reduce
> save(func)
>File "/sw/lib/python2.4/pickle.py", line 293, in save
> f(self, obj) # Call unbound method with explicit self
>File "/sw/lib/python2.4/pickle.py", line 760, in save_global
> raise PicklingError(
> pickle.PicklingError: Can't pickle : it's
> not found as hyper.PeriodicGrid
> >>> dir(hyper)
> ['Dir', 'Neighbors', 'PeriodicGrid', 'PeriodicPos', '__doc__',
> '__file__', '__name__', 'refcount']
> >>> hyper.PeriodicGrid
>
^
I think that's your error. The extension type is declared to be
hyper.PeriodicGrid, where it actually is model.hyper.PeriodicGrid
(because hyper is in the model package).
Pickle stores g.__class__.__module__ (which is "hyper") and
g.__class__.__name__ (="PeriodicGrid") to find the class object for
reimporting, and on unpickling, tries to do __import__("hyper"), which
fails.
The tp_name slot of your extension type should be "model.hyper.PeriodicGrid".
--
|>|\/|<
/--\
|David M. Cooke
|cookedm(at)physics(dot)mcmaster(dot)ca
--
http://mail.python.org/mailman/listinfo/python-list
Re: MMTK Install Problem
Justin Lemkul <[EMAIL PROTECTED]> writes: > Hello All, > > I am hoping that someone out there will be able to help me. During the > "build" phase of MMTK installation, I receive the following series of errors: > > $ python setup.py build > running build > running build_py > running build_ext > building 'lapack_mmtk' extension > gcc -fno-strict-aliasing -Wno-long-double -no-cpp-precomp -mno-fused-madd > -fno-common > -dynamic -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -DLIBM_HAS_ERFC > -DEXTENDED_TYPES -IInclude > -I/System/Library/Frameworks/Python.framework/Versions/ > 2.3/include/python2.3 -c Src/lapack_mmtk.c -o > build/temp.darwin-7.7.0-Power_Macintosh > -2.3/Src/lapack_mmtk.o > Src/lapack_mmtk.c:2:33: Numeric/arrayobject.h: No such file or directory Always look at the first error :-) GCC is awful for, when it can't find an include file, saying it can't, then spewing millions of error messages afterwards that are a direct result of not having stuff declared. In this case, it's obvious that you don't have Numeric installed correctly; the header files should be picked from one of the directories specified by the -I flags in the gcc invocation above. > I am attempting the install on a Mac OS X v10.3 with Python v2.3, NumPy > v23.1, > and SciPy v2.4.3 (You mean ScientificPython, not SciPy, right? Scipy is at 0.3.2) How did you install Numeric? The newest version is 23.7. It should be real easy to upgrade to that, as that version picks up Apple's vecLib framework for the linear algebra routines. Just do the usual 'python setup.py build', 'sudo python setup.py install'. That should put the header files where the MMTK installation expects them. -- |>|\/|< /--\ |David M. Cooke |cookedm(at)physics(dot)mcmaster(dot)ca -- http://mail.python.org/mailman/listinfo/python-list
Re: LinearAlgebra incredibly slow for eigenvalue problems
"drife" <[EMAIL PROTECTED]> writes: > Hello, > > I need to calculate the eigenvectors and eigenvalues for a 3600 X 3600 > covariance matrix. > > The LinearAlgebra package in Python is incredibly slow to perform the > above calculations (about 1.5 hours). This in spite of the fact that > I have installed Numeric with the full ATLAS and LAPACK libraries. > > Also note that my computer has dual Pentium IV (3.1 GHz) processors > with 2Gb ram. > > Every Web discussion I have seen about such issues indicates that > one can expect huge speed ups if one compiles and installs Numeric > linked against the ATLAS and LAPACK libraries. Are you *sure* that Numeric is linked against these? > Even more perplexing is that the same calculation takes a mere 7 min > in Matlab V6.5. Matlab uses both ATLAS and LAPACK. > > Moreover, the above calculation takes the same amount of time for > Numeric to complete with --and-- without ATLAS and PACK. I am certain > that I have done the install correctly. This is good evidence that Numeric *isn't* linked to them. If you're on a Linux system, you can check with ldd: [EMAIL PROTECTED] ldd /usr/lib/python2.3/site-packages/Numeric/lapack_lite.so liblapack.so.3 => /usr/lib/atlas/liblapack.so.3 (0x002a95677000) libblas.so.3 => /usr/lib/atlas/libblas.so.3 (0x002a95e55000) libg2c.so.0 => /usr/lib/libg2c.so.0 (0x002a96721000) libpthread.so.0 => /lib/libpthread.so.0 (0x002a96842000) libc.so.6 => /lib/libc.so.6 (0x002a96957000) libm.so.6 => /lib/libm.so.6 (0x002a96b96000) /lib64/ld-linux-x86-64.so.2 => /lib64/ld-linux-x86-64.so.2 (0x00552000) You can see that lapack and blas (the Atlas versions) are linked to the lapack_lite.so. To install Numeric using Lapack: - remove the build/ directory in your Numeric sources, so you don't any old binaries - edit setup.py and follow the comments on using Lapack (you need to comment out a few lines, and set some directories) Also set use_dotblas to 1. - do the 'python setup.py build', 'python setup.py install' dance. -- |>|\/|< /--\ |David M. Cooke |cookedm(at)physics(dot)mcmaster(dot)ca -- http://mail.python.org/mailman/listinfo/python-list
Re: LinearAlgebra incredibly slow for eigenvalue problems
"drife" <[EMAIL PROTECTED]> writes:
> Hi David,
>
> I performed the above check, and sure enough, Numeric
> is --not-- linked to the ATLAS libraries.
>
> I followed each of your steps outlined above, and Numeric
> still is not linking to the ATLAS libraries.
>
> My setup.py file is attached below.
> # delete all but the first one in this list if using your own LAPACK/BLAS
> sourcelist = [os.path.join('Src', 'lapack_litemodule.c')]
> # set these to use your own BLAS;
>
> library_dirs_list = ['/d2/lib/atlas']
> libraries_list = ['lapack', 'ptcblas', 'ptf77blas', 'atlas', 'g2c']
>
> # set to true (1), if you also want BLAS optimized
> matrixmultiply/dot/innerproduct
> use_dotblas = 1
> include_dirs = ['/d2/include']
This all look right (assuming you've got the right stuff in /d2).
When it compiles, does it look like it's actually doing the linking?
After doing python setup.py build, you can run ldd on the libraries in
the build directory (something like build/lib.linux-i386-2.3/lapack_lite.so).
If that's linked, then it's not being installed right.
You don't have a previous Numeric installation that's being picked up
instead of the one you're trying to install, do you?
At the interpreter prompt, check that
>>> import Numeric
>>> Numeric.__file__
gives you something you're expecting, and not something else.
--
|>|\/|<
/--\
|David M. Cooke
|cookedm(at)physics(dot)mcmaster(dot)ca
--
http://mail.python.org/mailman/listinfo/python-list
Re: Is there a way to determine -- when parsing -- if a word contains a builtin name or other imported system module name?
Casey Hawthorne <[EMAIL PROTECTED]> writes: > Is there a way to determine -- when parsing -- if a word contains a > builtin name or other imported system module name? > > Like "iskeyword" determines if a word is a keyword! Look in the keyword module; there is actually an "iskeyword" function there :) For modules, sys.modules is a dictionary of the modules that have been imported. -- |>|\/|< /------\ |David M. Cooke |cookedm(at)physics(dot)mcmaster(dot)ca -- http://mail.python.org/mailman/listinfo/python-list
Re: [ANN] Python 2.4 Quick Reference available
"Pete Havens" <[EMAIL PROTECTED]> writes: > The is awesome! Thanks. I did notice one thing while reading it. In the > "File Object" section, it states: > > "Created with built-in functions open() [preferred] or its alias > file()." > > ...this seems to be the opposite of the Python documentation: > > "The file() constructor is new in Python 2.2. The previous spelling, > open(), is retained for compatibility, and is an alias for file()." Except if you look at the current development docs (http://www.python.org/dev/doc/devel/lib/built-in-funcs.html) it says """ The file() constructor is new in Python 2.2 and is an alias for open(). Both spellings are equivalent. The intent is for open() to continue to be preferred for use as a factory function which returns a new file object. The spelling, file is more suited to type testing (for example, writing "isinstance(f, file)"). """ ... which more accurately reflects what I believe the consensus is about the usage of open vs. file. -- |>|\/|< /--\ |David M. Cooke |cookedm(at)physics(dot)mcmaster(dot)ca -- http://mail.python.org/mailman/listinfo/python-list
Re: How to write python plug-ins for your own python program?
Simon Wittber <[EMAIL PROTECTED]> writes:
>> You mean like 'import'? :)
>
> That's how I would do it. It's the simplest thing, that works.
>
> exec("import %s as plugin" % pluginName)
> plugin.someMethod()
>
> where pluginName is the name of the python file, minus the ".py" extension.
You'd better hope someone doesn't name their plugin
'os; os.system("rm -rf /"); import sys'
Use __import__ instead.
--
|>|\/|<
/--\
|David M. Cooke
|cookedm(at)physics(dot)mcmaster(dot)ca
--
http://mail.python.org/mailman/listinfo/python-list
Re: binutils "strings" like functionality?
"cjl" <[EMAIL PROTECTED]> writes:
> Fredrik Lundh wrote:
>
>> something like this could work:
>>
>> import re
>>
>> text = open(file, "rb").read()
>>
>> for m in re.finditer("([\x20-\x7f]{4,})[\n\0]", text):
>> print m.start(), repr(m.group(1))
>
> Hey...that worked. I actually modified:
>
> for m in re.finditer("([\x20-\x7f]{4,})[\n\0]", text):
>
> to
>
> for m in re.finditer("([\x20-\x7f]{4,})", text):
>
> and now the output is nearly identical to 'strings'. One problem
> exists, in that if the binary file contains a string
> "monkey/chicken/dog/cat" it is printed as "mokey//chicken//dog//cat",
> and I don't know enough to figure out where the extra "/" is coming
> from.
Are you sure it's monkey/chicken/dog/cat, and not
monkey\chicken\dog\cat? The later one will print monkey\\chicken...
because of the repr() call.
Also, you probably want it as [\x20-\x7e] (the DEL character \x7f
isn't printable). You're also missing tabs (\t).
The GNU binutils string utility looks for \t or [\x20-\x7e].
--
|>|\/|<
/--\
|David M. Cooke
|cookedm(at)physics(dot)mcmaster(dot)ca
--
http://mail.python.org/mailman/listinfo/python-list
Re: SAX parsing problem
anon <[EMAIL PROTECTED]> writes: > So I've encountered a strange behavior that I'm hoping someone can fill > me in on. i've written a simple handler that works with one small > exception, when the parser encounters a line with '&' in it, it > only returns the portion that follows the occurence. > > For example, parsing a file with the line : > mykeysome%20&%20value > > results in getting "%20value" back from the characters method, rather > than "some%20&%20value". > > After looking into this a bit, I found that SAX supports entities and > that it is probably believing the & to be an entity and processing > it in some way that i'm unware of. I'm using the default > EntityResolver. Are you sure you're not actually getting three chunks: "some%20", "&", and "%20value"? The xml.sax.handler.ContentHandler.characters method (which I presume you're using for SAX, as you don't mention!) is not guaranteed to get all contiguous character data in one call. Also check if .skippedEntity() methods are firing. -- |>|\/|< /--\ |David M. Cooke |cookedm(at)physics(dot)mcmaster(dot)ca -- http://mail.python.org/mailman/listinfo/python-list
Re: how to handle repetitive regexp match checks
Matt Wette <[EMAIL PROTECTED]> writes:
> Over the last few years I have converted from Perl and Scheme to
> Python. There one task that I do often that is really slick in Perl
> but escapes me in Python. I read in a text line from a file and check
> it against several regular expressions and do something once I find a match.
> For example, in perl ...
>
> if ($line =~ /struct {/) {
>do something
> } elsif ($line =~ /typedef struct {/) {
>do something else
> } elsif ($line =~ /something else/) {
> } ...
>
> I am having difficulty doing this cleanly in python. Can anyone help?
>
> rx1 = re.compile(r'struct {')
> rx2 = re.compile(r'typedef struct {')
> rx3 = re.compile(r'something else')
>
> m = rx1.match(line)
> if m:
>do something
> else:
>m = rx2.match(line)
>if m:
> do something
>else:
> m = rx3.match(line)
> if m:
> do something
> else:
> error
I usually define a class like this:
class Matcher:
def __init__(self, text):
self.m = None
self.text = text
def match(self, pat):
self.m = pat.match(self.text)
return self.m
def __getitem__(self, name):
return self.m.group(name)
Then, use it like
for line in fo:
m = Matcher(line)
if m.match(rx1):
do something
elif m.match(rx2):
do something
else:
error
--
|>|\/|<
David M. Cooke
cookedm(at)physics(dot)mcmaster(dot)ca
--
http://mail.python.org/mailman/listinfo/python-list
Re: Distutils vs. Extension header files
Mike Meyer <[EMAIL PROTECTED]> writes: > I've got a package that includes an extension that has a number of > header files in the directory with the extension. They are specified > as "depends = [...]" in the Extension class. However, Distutils > doesn't seem to do anything with them. > > If I do an sdist, the include files aren't added to the tarball. > > If I do a bdist_rpm, the source files get copied into the build > directory and the build starts, but the header files aren't copied > with the source file, so the build fails with a missing header file. > > I find it hard to believe that this is a bug in distutils, so I'd > appreciate it if someone could tell me what I'm doing wrong. vincent has the solution (you need to specify them in MANIFEST.in), but I'll add my 2 cents. depends = [...] is used in building (it's like dependencies in make). If one of those files change, distutils will rebuild the extension. But that's all distutils does with it. It's braindead including stuff in the source distribution, including depends, data files, and other stuff you'd think it would do. When in doubt, add it to MANIFEST.in. -- |>|\/|< /--\ |David M. Cooke |cookedm(at)physics(dot)mcmaster(dot)ca -- http://mail.python.org/mailman/listinfo/python-list
Re: Namespaces and the timeit module
Roy Smith <[EMAIL PROTECTED]> writes:
> I'm playing with the timeit module, and can't figure out how to time a
> function call. I tried:
>
> def foo ():
> x = 4
> return x
>
> t = timeit.Timer ("foo()")
> print t.timeit()
>
> and quickly figured out that the environment the timed code runs under
> is not what I expected:
>
> Traceback (most recent call last):
> File "./d.py", line 10, in ?
> print t.timeit()
> File "/usr/local/lib/python2.3/timeit.py", line 158, in timeit
> return self.inner(it, self.timer)
> File "", line 6, in inner
> NameError: global name 'foo' is not defined
>
> In fact, trying to time "print dir()" gets you:
>
> ['_i', '_it', '_t0', '_timer']
>
> It seems kind of surprising that I can't time functions. Am I just not
> seeing something obvious?
Like the documentation for Timer? :-)
class Timer([stmt='pass' [, setup='pass' [, timer=]]])
You can't use statements defined elsewhere, you have to define them in
the setup arguments (as a string). Like this:
define_foo = '''
def foo():
x = 4
return x
'''
t = timeit.Timer("foo()" setup=define_foo)
print t.timeit()
One common idiom I've seen is to put your definition of foo() in a
module (say x.py), then, from the command line:
$ python -m timeit -s 'from x import foo' 'foo()'
(the -m is for python 2.4 to run the timeit module; use the full path
to timeit.py instead for earlier pythons)
Alternatively, the examples for the timeit module has another way to
time functions defined in a module.
--
|>|\/|<
/--\
|David M. Cooke
|cookedm(at)physics(dot)mcmaster(dot)ca
--
http://mail.python.org/mailman/listinfo/python-list
Re: getattr() woes
[EMAIL PROTECTED] (Aahz) writes:
> In article <[EMAIL PROTECTED]>,
> Thomas Rast <[EMAIL PROTECTED]> wrote:
>>
>>class dispatcher:
>># ...
>>def __getattr__(self, attr):
>>return getattr(self.socket, attr)
>>
>>>>> import asyncore
>>>>> class Peer(asyncore.dispatcher):
>>... def _get_foo(self):
>>... # caused by a bug, several stack levels deeper
>>... raise AttributeError('hidden!')
>>... foo = property(_get_foo)
>>...
>
> You're not supposed to use properties with classic classes.
Even if dispatcher was a new-style class, you still get the same
behaviour (or misbehaviour) -- Peer().foo still raises AttributeError
with the wrong message.
A simple workaround is to put a try ... except AttributeError block in
his _get_foo(), which would re-raise with a different error that
wouldn't be caught by getattr. You could even write a property
replacement for that:
>>> class HiddenAttributeError(Exception):
... pass
>>> def robustprop(fget):
... def wrapped_fget(self):
... try:
... return fget(self)
... except AttributeError, e:
... raise HiddenAttributeError(*e.args)
... return property(fget=wrapped_fget)
Ideally, I think the better way is if getattr, when raising
AttributeError, somehow reused the old traceback (which would point
out the original problem). I don't know how to do that, though.
--
|>|\/|<
/--\
|David M. Cooke
|cookedm(at)physics(dot)mcmaster(dot)ca
--
http://mail.python.org/mailman/listinfo/python-list
Re: Passing arguments to python from URL
Casey Bralla <[EMAIL PROTECTED]> writes:
> I've got a python cgi-bin application which produces an apache web page. I
> want to pass arguments to it on the URL line, but the parameters are not
> getting passed along to python properly.
>
> I've been using sys.argv to pick up command line arguments, and it works
> fine when I call the python program from the command line. Unfortunately,
> when I pass data to the program from the URL, many of the parameters are
> being clobbered and **NOT** passed to python.
>
> For example: "http://www.nobody.com/cgi-bin/program.py?sort=ascending"; only
> passes the parameter "/usr/lib/cgi-bin/program.py".
This is expected.
> However, "http://www.nobody.com/cgi-bin/program.py?sort%20ascending"; passes
> a 2-place tuple of ("/usr/lib/cgi-bin/program.py", "sort
> ascending").
I don't know why this actually works, it's not (AFAIK) defined behaviour.
> Somehow, adding the "=" in the argument list prevents **ANY** parameters
> from being passed to python. I could re-write the python program to work
> around this, but I sure would like to understand it first.
You're going to have to rewrite. CGI scripts get their arguments
passed to them through the environment, not on the command line.
QUERY_STRING, for instance, will hold the query string (the stuff
after the ?).
Use Python's cgi module to make things easier on yourself; the
documentation has a good overview:
http://www.python.org/doc/2.4/lib/module-cgi.html
In this case, your script would look something like this:
import cgi
form = cgi.FieldStorage()
if form.getvalue('sort') == 'ascending':
... sort in ascending order ...
etc.
--
|>|\/|<
/--\
|David M. Cooke
|cookedm(at)physics(dot)mcmaster(dot)ca
--
http://mail.python.org/mailman/listinfo/python-list
Re: Using python to extend a python app
dataangel <[EMAIL PROTECTED]> writes: > I'm writing a python app that works as a replacement for the menu that > comes with most minimalist wms when you right click the root window. > It's prettier and written completely in python. > > I'd like to provide hooks or some system so that people can write > their own extensions to the app, for example adding fluxbox options, > and then fluxbox users can choose to use that extension. But I'm not > sure how to implement it. > > Right now the best idea I have is to have all desired extensions in a > folder, import each .py file in that folder as a module using > __import__, and then call some predetermined method, say "start", and > pass it the menu as it exists so far so they can add to it, > start(menu). This seems kind of hackish. That looks pretty reasonable, and easy. There have been some recent threads (in the past month or so) on plugins, so you might want to search the archives. Most of it's revolved around not using exec :-) I just had a look at pyblosxom (one program that I know that uses plugins), and it uses exactly this approach, with some extra frills: looking in subdirectories, for instance. -- |>|\/|< /--\ |David M. Cooke |cookedm(at)physics(dot)mcmaster(dot)ca -- http://mail.python.org/mailman/listinfo/python-list
Re: breaking up is hard to do
"bbands" <[EMAIL PROTECTED]> writes: > For example I have a class named Indicators. If I cut it out and put it > in a file call Ind.py then "from Ind import Indicators" the class can > no longer see my globals. This is true even when the import occurs > after the config file has been read and parsed. Don't use globals? Or put all the globals into a separate module, which you import into Ind and into whatever uses Ind. Putting the globals into a separate namespace (module, class, class instance, whatever) also makes it easier to know what is a global :-) -- |>|\/|< /------\ |David M. Cooke |cookedm(at)physics(dot)mcmaster(dot)ca -- http://mail.python.org/mailman/listinfo/python-list
Re: itertools to iter transition
Steven Bethard <[EMAIL PROTECTED]> writes: > Terry Reedy wrote: >>>But if classmethods are intended to provide alternate constructors >> But I do not remember that being given as a reason for >> classmethod(). But I am not sure what was. > > Well I haven't searched thoroughly, but I know one place that it's > referenced is in descrintro[1]: > > "Factoid: __new__ is a static method, not a class method. I initially > thought it would have to be a class method, and that's why I added the > classmethod primitive. Unfortunately, with class methods, upcalls > don't work right in this case, so I had to make it a static method > with an explicit class as its first argument. Ironically, there are > now no known uses for class methods in the Python distribution (other > than in the test suite). Not true anymore, of course (it was in 2.2.3). In 2.3.5, UserDict, tarfile and some the Mac-specific module use classmethod, and the datetime extension module use the C version (the METH_CLASS flag). And staticmethod (and METH_STATIC) aren't used at all in 2.3 or 2.4 :-) [if you ignore __new__] -- |>|\/|< /--\ |David M. Cooke |cookedm(at)physics(dot)mcmaster(dot)ca -- http://mail.python.org/mailman/listinfo/python-list
Re: numeric module
[EMAIL PROTECTED] writes: > Hello, > What's the problem with this code? I get the following error message: > > File "test.py", line 26, in test > print tbl[wi][bi] > IndexError: index must be either an int or a sequence > > ---code snippet > > from Numeric import * > tbl = zeros((32, 16)) > > def test(): > > val = testme() > wi = val >> 4 > bi = val & 0xFL [above changed to use val instead of crc, as you mentioned in another post] > print wi > print bi > print tbl[wi][bi] tbl[wi][bi] would be indexing the bi'th element of whatever tbl[wi] returns. For Numeric arrays, you need tbl[wi,bi] Now, you'll have another problem as Terry Reedy mentioned: the indices (in Numeric) need to be Python ints, not longs. You could rewrite your test() function as def test(): val = testme() wi = int(val >> 4) bi = int(val & 0xF) print wi print bi print tbl[wi,bi] and that'll work. -- |>|\/|< /--\ |David M. Cooke |cookedm(at)physics(dot)mcmaster(dot)ca -- http://mail.python.org/mailman/listinfo/python-list
Re: numeric module
"coffeebug" <[EMAIL PROTECTED]> writes: > I cannot import "numarray" and I cannot import "numeric" using python > 2.3.3 numarray and Numeric are separate modules available at http://numpy.sourceforge.net/ If you're doing anything numerical in Python, you'll want them :-) -- |>|\/|< /------\ |David M. Cooke |cookedm(at)physics(dot)mcmaster(dot)ca -- http://mail.python.org/mailman/listinfo/python-list
Re: Gnuplot.py and, _by far_, the weirdest thing I've ever seen on my computer
"syd" <[EMAIL PROTECTED]> writes:
> I don't even know where to begin. This is just bizarre. I just picked
> up the Gnuplot.py module (a light interface to gnuplot commands) and
> was messing around with it today.
>
> I've got a tiny script, but it only works from the command line about
> half the time! In the python interpreter, 100%. Ipython, 100%. I'm
> not kidding.
>
> #!/bin/env python
> import Gnuplot
> g = Gnuplot.Gnuplot(debug=1)
> g.title('A simple example')
> g('set data style linespoints')
> g('set terminal png small color')
> g('set output "myGraph.png"')
> g.plot([[0,1.1], [1,5.8], [2,3.3], [3,100]])
>
> Here's just one example -- it does not work, then it works. It seems
> totally random. It will work a few times, then it won't for a few
> times...
>
> bash-2.05b$ ./myGnu.py
> gnuplot> set title "A simple example"
> gnuplot> set data style linespoints
> gnuplot> set terminal png small color
> gnuplot> set output "myGraph.png"
> gnuplot> plot '/tmp/tmp5LXAow' notitle
>
> gnuplot> plot '/tmp/tmp5LXAow' notitle
> ^
> can't read data file "/tmp/tmp5LXAow"
> line 0: util.c: No such file or directory
>
> bash-2.05b$ ./myGnu.py
> gnuplot> set title "A simple example"
> gnuplot> set data style linespoints
> gnuplot> set terminal png small color
> gnuplot> set output "myGraph.png"
> gnuplot> plot '/tmp/tmpHMTkpL' notitle
>
> (and it makes the graph image just fine)
>
> I mean what the hell is going on? My permissions on /tmp are wide open
> (drwxrwxrwt). It does the same thing when I run as root. And it
> _always_ works when I use the interpreter or interactive python.
>
> Any clues would be greatly appreciated. I'm baffled.
What's your OS? Python version? Gnuplot.py version (I assume 1.7)?
Put a 'import sys; print sys.version' in there to make sure /bin/env
is using the same python as you expect it to.
It looks like any temporary file it's writing to is deleted too early.
Have a look at gp_unix.py in the Gnuplot source. There's some
customization options that might be helpful. In particular, I'd try
import Gnuplot
Gnuplot.GnuplotOpts.prefer_fifo_data = 0
... then the data will be save to a temporary file instead of piped
through a fifo.
Alternatively, try
Gnuplot.GnuplotOpts.prefer_inline_data = 1
... then no file will be used.
[I don't use Gnuplot myself; this is just what I came up with after a
few minutes of looking at it]
--
|>|\/|<
/--\
|David M. Cooke
|cookedm(at)physics(dot)mcmaster(dot)ca
--
http://mail.python.org/mailman/listinfo/python-list
Re: richcmpfunc semantics
harold fellermann <[EMAIL PROTECTED]> writes:
> Thank you Greg,
>
> I figured most of it out in the meantime, myself. I only differ
> from you in one point.
>
>>> What has to be done, if the function is invoked for an operator
>>> I don't want to define?
>>
>> Return Py_NotImplemented. (Note that's return, *not* raise.)
>
> I used
>
> PyErr_BadArgument();
> return NULL;
>
> instead. What is the difference between the two and which one
> is to prefer.
If you do it your way you're a bad neighbour: If your object is the
first one (left-hand side) of the operator, it will prevent the other
object from handling the case if it can. This is the same advice as
for all of the other operators (__add__, etc.)
Consider the pure-python version:
class A:
def __init__(self, what_to_do='return'):
self.what_to_do = what_to_do
def __eq__(self, other):
print 'A.__eq__'
if self.what_to_do == 'return':
return NotImplemented
else:
raise Exception
class B:
def __eq__(self, other):
print 'B.__eq__'
return True
>>> a = A('return')
>>> b = B()
>>> a == b
A.__eq__
B.__eq__
True
>>> b == a
B.__eq__
True
>>> a == a
A.__eq__
A.__eq__
A.__eq__
A.__eq__
True
So the B class handles the case where A doesn't know what to do. Also
note the last case, where Python falls back on id() comparisions to
determine equality.
Now, compare with this:
>>> a = A('raise')
>>> b = B()
>>> a == b
A.__eq__
Traceback (most recent call last):
File "", line 1, in ?
File "x.py", line 9, in __eq__
raise Exception
Exception
>>> b == a
B.__eq__
True
>>> a == a
A.__eq__
Traceback (most recent call last):
File "", line 1, in ?
File "x.py", line 9, in __eq__
raise Exception
Exception
So now comparing A and B objects can fail. If you *know* A and B
objects can't be compared for equality, it'd be ok to raise a
TypeError, but that should be after a type test.
> Also, do you need to increment the reference count
> of Py_NotImeplemented before returning it?
Yes; it's a singleton like Py_None.
--
|>|\/|<
/--\
|David M. Cooke
|cookedm(at)physics(dot)mcmaster(dot)ca
--
http://mail.python.org/mailman/listinfo/python-list
Re: curious problem with large numbers
Chris Fonnesbeck <[EMAIL PROTECTED]> writes: > I have been developing a python module for Markov chain Monte Carlo > estimation, in which I frequently compare variable values with a very > large number, that I arbitrarily define as: > > inf = 1e1 > > However, on Windows (have tried on Mac, Linux) I get the following behaviour: > >>>> inf = 1e1 >>>> inf > 1.0 > > while I would have expected: > > 1.#INF > > Smaller numbers, as expected, yield: > >>>> inf = 1e100 >>>> inf > 1e+100 > > Obviously, I cannot use the former to compare against large (but not > infinite) numbers, at least not to get the result I expect. Has anyone > seen this behaviour? I don't do Windows, so I can't say this will work, but try >>> inf = 1e308*2 I think your problem is how the number is being parsed; perhaps Windows is punting on all those zeros? Computing the infinity may or may not work, but it gets around anything happening in parsing. Alternatively, if you have numarray installed (which you should probably consider if you're doing numerical stuff ;-) you could use >>> import numarray.ieeespecial >>> numarray.ieeespecial.plus_inf inf (there's minus_inf, nan, plus_zero, and minus_zero also) -- |>|\/|< /--\ |David M. Cooke |cookedm(at)physics(dot)mcmaster(dot)ca -- http://mail.python.org/mailman/listinfo/python-list
Re: Threads and variable assignment
Gregory Bond <[EMAIL PROTECTED]> writes: > I've had a solid hunt through the (2.3) documentation but it seems > silent on this issue. > > I have an problem that would naturally run as 2 threads: One monitors > a bunch of asyncrhonous external state and decides if things are > "good" or "bad". The second thread processes data, and the processing > depends on the "good" or "bad" state at the time the data is processed. > > Sort of like this: > > Thread 1: > > global isgood > while 1: > wait_for_state_change() > if new_state_is_good(): > isgood = 1 > else: > isgood = 0 > > Thread 2: > > s = socket() > s.connect(...) > f = s.makefile() > while 1: > l = f.readline() > if isgood: > print >> goodfile, l > else: > print >> badfile, l You said that the processing depends on the good or bad state at the time the data is processed: I don't know how finely-grained your state changes will be in thread 1, but it doesn't seem that thread 2 would notice at the right time. If the socket blocks reading a line, the state could change i > What guarantees (if any!) does Python make about the thread safety of > this construct? Is it possible for thread 2 to get an undefined > variable if it somehow catches the microsecond when isgood is being > updated by thread 1? It won't be undefined, but it's possible that (in thread 1) between the "if new_state_is_good()" and the setting of isgood that thread 2 will execute, so if new_state_is_good() was false, then it could still write the line to the goodfile. It really depends on how often you have state changes, how often you get (full) lines on your socket, and how much you care that the correct line be logged to the right file. If you needed this to be robust, I'd either: - Try to rewrite wait_for_status_change()/new_state_is_good() to be asynchronous, particularly if wait_for_status_change() is blocking on some file or socket object. This way you could hook it into asynchat/asyncore or Twisted without any threads. - Or, if you need to use threads, use a Queue.Queue object where timestamps w/ state changes are pushed on in thread 1, and popped off and analysed before logging in thread 2. (Or something; this just popped in my head.) -- |>|\/|< /--\ |David M. Cooke |cookedm(at)physics(dot)mcmaster(dot)ca -- http://mail.python.org/mailman/listinfo/python-list
Re: python/svn issues....
"bruce" <[EMAIL PROTECTED]> writes: > hi... > > in trying to get viewcvs up/running, i tried to do the following: > > [EMAIL PROTECTED] viewcvs-0.9.2]# python > Python 2.3.3 (#1, May 7 2004, 10:31:40) > [GCC 3.3.3 20040412 (Red Hat Linux 3.3.3-7)] on linux2 > Type "help", "copyright", "credits" or "license" for more information. >>>> import svn.repos > Traceback (most recent call last): > File "", line 1, in ? > File > "/dar/tmp/subversion-1.1.4-0.1.1.fc2.rf-root/usr/lib/python2.3/site-packages > /svn/repos.py", line 19, in ? > File > "/dar/tmp/subversion-1.1.4-0.1.1.fc2.rf-root/usr/lib/python2.3/site-packages > /svn/fs.py", line 28, in ? > File > "/dar/tmp/subversion-1.1.4-0.1.1.fc2.rf-root/usr/lib/python2.3/site-packages > /libsvn/fs.py", line 4, in ? > ImportError: /usr/lib/libsvn_fs_base-1.so.0: undefined symbol: db_create This looks like a problem when Subversion was built: this library was not linked against the Berkeley DB libraries. You can check what's linked using ldd, and see the unresolved symbols with ldd -r. For instance, on my AMD64 Debian system, $ ldd -r /usr/lib/libsvn_fs_base-1.so.0 libsvn_delta-1.so.0 => /usr/lib/libsvn_delta-1.so.0 (0x002a95696000) libsvn_subr-1.so.0 => /usr/lib/libsvn_subr-1.so.0 (0x002a9579f000) libaprutil-0.so.0 => /usr/lib/libaprutil-0.so.0 (0x002a958c8000) libldap.so.2 => /usr/lib/libldap.so.2 (0x002a959e) liblber.so.2 => /usr/lib/liblber.so.2 (0x002a95b19000) libdb-4.2.so => /usr/lib/libdb-4.2.so (0x002a95c27000) libexpat.so.1 => /usr/lib/libexpat.so.1 (0x002a95e05000) libapr-0.so.0 => /usr/lib/libapr-0.so.0 (0x002a95f29000) librt.so.1 => /lib/librt.so.1 (0x002a9604e000) libm.so.6 => /lib/libm.so.6 (0x002a96155000) libnsl.so.1 => /lib/libnsl.so.1 (0x002a962dc000) libpthread.so.0 => /lib/libpthread.so.0 (0x002a963f2000) libc.so.6 => /lib/libc.so.6 (0x002a96506000) libdl.so.2 => /lib/libdl.so.2 (0x002a96746000) libcrypt.so.1 => /lib/libcrypt.so.1 (0x002a96849000) libresolv.so.2 => /lib/libresolv.so.2 (0x002a9697c000) libsasl2.so.2 => /usr/lib/libsasl2.so.2 (0x002a96a91000) libgnutls.so.11 => /usr/lib/libgnutls.so.11 (0x002a96ba8000) /lib64/ld-linux-x86-64.so.2 => /lib64/ld-linux-x86-64.so.2 (0x00552000) libtasn1.so.2 => /usr/lib/libtasn1.so.2 (0x002a96d1b000) libgcrypt.so.11 => /usr/lib/libgcrypt.so.11 (0x002a96e2b000) libgpg-error.so.0 => /usr/lib/libgpg-error.so.0 (0x002a96f77000) libz.so.1 => /usr/lib/libz.so.1 (0x002a9707b000) If it doesn't look like that, then I'd say your Subversion package was built badly. You may also want to run ldd on your svn binary, to see what libraries it pulls in. -- |>|\/|< /--\ |David M. Cooke |cookedm(at)physics(dot)mcmaster(dot)ca -- http://mail.python.org/mailman/listinfo/python-list
Re: python/svn issues....
"bruce" <[EMAIL PROTECTED]> writes: > david... > > thanks for the reply... > > it's starting to look as though the actual /usr/lib/libdb-4.2.so from the > rpm isn't exporting any of the symbols... > > when i do: > nm /usr/lib/libdb-4.2.so | grep db_create > > i get > nm: /usr/lib/libdb-4.2.so: no symbols > > which is strange... because i should be getting the db_create symbol... > > i'll try to build berkeley db by hand and see what i get... > > if you could try the 'nm' command against your berkely.. i'd appreciate you > letting me know what you get.. Not surprising; plain 'nm' doesn't work for me on shared libraries. I need to use 'nm -D'. In that case, I get a db_create (or rather, a versioned form, db_create_4002). Running 'nm -D -g' on the libsvn_fs_base library shows it uses the same db_create_4002 function. -- |>|\/|< /--\ |David M. Cooke http://arbutus.physics.mcmaster.ca/dmc/ |[EMAIL PROTECTED] -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there a package with convolution and related methods?
Charles Krug <[EMAIL PROTECTED]> writes: > List: > > Is there a Python package with Convolution and related methods? > > I'm working on modeling some DSP processes in Python. I've rolled one > up, but don't feel much like reinventing the wheel, especially if > there's already something like "Insanely Efficient FFT for Python" > already. > > Thanks You most certainly want to look at the numerical python packages Numeric and numarray (http://numeric.scipy.org/) for array manipulations, and scipy (http://scipy.org) has wraps for FFTW (Fast Fourier Transform in the West). -- |>|\/|< /------\ |David M. Cooke |cookedm(at)physics(dot)mcmaster(dot)ca -- http://mail.python.org/mailman/listinfo/python-list
PyArg_ParseTuple() when the type could be anything?
I'd like to be able to use PyArg_ParseTuple() in a generic way.
for example, i'd like to have all commands start with 1 integer parameter, and
this "commandID" will inform me of what parameters come next (via LUT).
knowing that i can then call ParseTuple again with the proper parameters.
like this:
if (PyArg_ParseTuple(args, "i|", &commandID)) {
switch (commandID) {
case cmd_with_str: {
const char *strZ = NULL;
if (PyArg_ParseTuple(args, "is", &commandID, &strZ)) {
// do something with string
}
break;
}
case cmd_with_float: {
float valF = -1;
if (PyArg_ParseTuple(args, "if", &commandID, &valF)) {
// do something with float
}
break;
}
}
}
is there a way to achieve this? the "i|" at the start is not working
--
http://mail.python.org/mailman/listinfo/python-list
Re: PyArg_ParseTuple() when the type could be anything?
i was able to get what i wanted by simply iterating over the tupile instead of using ParseTupile, then just query the type, then convert the type to C and move on to the next. totally great, now i can pass N different argument types to a single function, and have the C side deal gracefully with whatever types are sent. -- http://mail.python.org/mailman/listinfo/python-list
Re: Raw_input with readline in a daemon thread makes terminal text disappear
Hi all, This is an old thread, but I'm having the same behavior in my terminal when I run some code but kill the process in the terminal (Ctrl-C). The code has two prime suspects (from a simple google search): 1. Creates ssh port forward via the subprocess module (http://unix.stackexchange.com/questions/4740/screen-remote-login-failure-an d-disappearing-text) 2. Using the getpass module (raw_input?) Calling "$ reset" brings back the disappearing text, so I'm just wondering if this issue has been addressed and if so, what should I be doing that I'm not. Thank you, Dave W. Response to post: http://mail.python.org/pipermail/python-list/2009-October/554784.html I'm getting input for a program while it's running by using raw_input in a loop in separate thread. This works except for the inconvenience of not having a command history or the use of backspace etc. That can be solved by loading the readline module; however, it results in a loss of visible access to the terminal when the program ends: nothing is echoed to the screen and the history is invisible (although it is there - hitting return executes whatever should be there normally). The only way to get it back is to close the terminal and open a new one. Here is minimal code that reproduces the problem (python 2.5 on Linux): from threading import Thread import readline get_input = Thread(target=raw_input) get_input.setDaemon(True) get_input.start() If the thread is not set to daemon mode, there is no such problem (don't know why this makes a difference), but in the real program, it needs to be a daemon or it hangs the exit waiting for more input. Any suggestions appreciated. Thanks, John -- http://mail.python.org/mailman/listinfo/python-list
can't get utf8 / unicode strings from embedded python
note everything works great if i use Ascii, but:
in my utf8-encoded script i have this:
> print "frøânçïé"
in my embedded C++ i have this:
PyObject* CPython_Script::print(PyObject *args)
{
PyObject*resultObjP = NULL;
const char *utf8_strZ = NULL;
if (PyArg_ParseTuple(args, "s", &utf8_strZ)) {
Log(utf8_strZ, false);
resultObjP = Py_None;
Py_INCREF(resultObjP);
}
return resultObjP;
}
Now, i know that my Log() can print utf8 (has for years, very well debugged)
but what it *actually* prints is this:
> print "frøânçïé"
--> frøânçïé
another method i use looks like this:
> kj_commands.menu("控件", "同步滑帧", "全局无滑帧")
or
> kj_commands.menu(u"控件", u"同步滑帧", u"全局无滑帧")
and in my C++ i have:
SuperString ScPyObject::GetAs_String()
{
SuperString str;
if (PyUnicode_Check(i_objP)) {
#if 1
// method 1
{
ScPyObject
utf8Str(PyUnicode_AsUTF8String(i_objP));
str = utf8Str.GetAs_String();
}
#elif 0
// method 2
{
UTF8Char*uniZ = (UTF8Char
*)PyUnicode_AS_UNICODE(i_objP);
str.assign(&uniZ[0],
&uniZ[PyUnicode_GET_DATA_SIZE(i_objP)], kCFStringEncodingUTF16);
}
#else
// method 3
{
UTF32VeccharVec(32768);
CF_ASSERT(sizeof(UTF32Vec::value_type) == sizeof(wchar_t));
PyUnicodeObject *uniObjP = (PyUnicodeObject
*)(i_objP);
Py_ssize_t
sizeL(PyUnicode_AsWideChar(uniObjP, (wchar_t *)&charVec[0], charVec.size()));
charVec.resize(sizeL);
charVec.push_back(0);
str.Set(SuperString(&charVec[0]));
}
#endif
} else {
str.Set(uc(PyString_AsString(i_objP)));
}
Log(str.utf8Z());
return str;
}
for the string, "控件", i get:
--> 控件
for the *unicode* string, u"控件", Methods 1, 2, and 3, i get the same thing:
--> 控件
okay so what am i doing wrong???
--
http://mail.python.org/mailman/listinfo/python-list
Re: can't get utf8 / unicode strings from embedded python
> I see you are using Python 2 correct > Firstly, in Python 2, the compiler assumes that the source code is encoded in > ASCII gar, i must have been looking at doc for v3, as i thought it was all assumed to be utf8 > # -*- coding: utf-8 -*- okay, did that, still no change > you need to use u" ... " delimiters for Unicode, otherwise the results you > get are completely arbitrary and depend on the encoding of your terminal. okay, well, i'm on a mac, and not using "terminal" at all. but if i were, it would be utf8 but it's still not flying :( > For example, if I set my terminal encoding to IBM-850 okay how do you even do that? this is not an interactive session, this is embedded python, within a C++ app, so there's no terminal. but that is a good question: all the docs say "default encoding" everywhere (as in "If string is a Unicode object, this function computes the default encoding of string and operates on that"), but fail to specify just HOW i can set the default encoding. if i could just say "hey, default encoding is utf8", i think i'd be done? > So change the line of code to: > print u"frøânçïé" okay, sure... but i get the exact same results > Those two changes ought to fix the problem, but if they don't, try setting > your terminal encoding to UTF-8 as well well, i'm not sure what you mean by that. i don't have a terminal here. i'm logging to a utf8 log file (when i print) > but what it *actually* prints is this: > >print "frøânçïé" > --> fr√∏√¢n√ß√Ø√© >It's hard to say what *exactly* is happening here, because you don't explain >how the python print statement somehow gets into your C++ Log code. Do I guess >right that it catches stdout? yes, i'm redirecting stdout to my own custom print class, and then from that function i call into my embedded C++ print function >If so, then what I expect is happening is that Python has read in the source >code of >print "~" >with ~ as a bunch of junk bytes, and then your terminal is displaying >those junk bytes according to whatever encoding it happens to be using. >Since you are seeing this: >fr√∏√¢n√ß√Ø√© >my guess is that you're using a Mac, and the encoding is set to the MacRoman >encoding. Am I close? you hit the nail on the head there, i think. using that as a hint, i took this text "fr√∏√¢n√ß√Ø√©" and pasted that into a "macRoman" document, then *reinterpreted* it as UTF8, and voala: "frøânçïé" so, it seems that i AM getting my utf8 bytes, but i'm getting them converted to macRoman. huh? where is macRoman specified, and how to i change that to utf8? i think that's the missing golden ticket -- http://mail.python.org/mailman/listinfo/python-list
Re: can't get utf8 / unicode strings from embedded python
> What _are_ you using? i have scripts in a file, that i am invoking into my embedded python within a C++ program. there is no terminal involved. the "print" statement has been redirected (via sys.stdout) to my custom print class, which does not specify "encoding", so i tried the suggestion above to set it: static const char *s_RedirectScript = "import " kEmbeddedModuleName "\n" "import sys\n" "\n" "class CustomPrintClass:\n" " def write(self, stuff):\n" " " kEmbeddedModuleName "." kCustomPrint "(stuff)\n" "class CustomErrClass:\n" " def write(self, stuff):\n" " " kEmbeddedModuleName "." kCustomErr "(stuff)\n" "sys.stdout = CustomPrintClass()\n" "sys.stderr = CustomErrClass()\n" "sys.stdout.encoding = 'UTF-8'\n" "sys.stderr.encoding = 'UTF-8'\n"; but it didn't help. I'm still getting back a string that is a utf-8 string of characters that, if converted to "macRoman" and then interpreted as UTF8, shows the original, correct string. who is specifying macRoman, and where, and how do i tell whoever that is that i really *really* want utf8? -- http://mail.python.org/mailman/listinfo/python-list
Re: can't get utf8 / unicode strings from embedded python
i'm sorry this is so confusing, let me try to re-state the problem in as clear
a way as i can.
I have a C++ program, with very well tested unicode support. All logging is
done in utf8. I have conversion routines that work flawlessly, so i can assure
you there is nothing wrong with logging and unicode support in the underlying
program.
I am embedding python 2.7 into the program, and extending python with routines
in my C++ program.
I have a script, encoded in utf8, and *marked* as utf8 with this line:
# -*- coding: utf-8 -*-
In that script, i have inline unicode text. When I pass that text to my C++
program, the Python interpreter decides that these bytes are macRoman, and
handily "converts" them to unicode. To compensate, i must "convert" these
"macRoman" characters encoded as utf8, back to macRoman, then "interpret" them
as utf8. In this way i can recover the original unicode.
When i return a unicode string back to python, i must do the reverse so that
Python gets back what it expects.
This is not related to printing, or sys.stdout, it does happen with that too
but focusing on that is a red-herring. Let's focus on just passing a string
into C++ then back out.
This would all actually make sense IF my script was marked as being "macRoman"
even tho i entered UTF8 Characters, but that is not the case.
Let's prove my statements. Here is the script, *interpreted* as MacRoman:
http://karaoke.kjams.com/screenshots/bugs/python_unicode/script_as_macroman.png
and here it is again *interpreted* as utf8:
http://karaoke.kjams.com/screenshots/bugs/python_unicode/script_as_utf8.png
here is the string conversion code:
SuperString ScPyObject::GetAs_String()
{
SuperString str;// underlying format of
SuperString is unicode
if (PyUnicode_Check(i_objP)) {
ScPyObject utf8Str(PyUnicode_AsUTF8String(i_objP));
str = utf8Str.GetAs_String();
} else {
const UTF8Char *bytes_to_interpetZ =
uc(PyString_AsString(i_objP));
// the "Set" call *interprets*, does not *convert*
str.Set(bytes_to_interpetZ, kCFStringEncodingUTF8);
// str is now unicode characters which *represent*
macRoman characters
// so *convert* these to actual macRoman
// fyi: Update_utf8 means "convert to this encoding and
// store the resulting bytes in the variable named "utf8"
str.Update_utf8(kCFStringEncodingMacRoman);
// str is now unicode characters converted from macRoman
// so *reinterpret* them as UTF8
// FYI, we're just taking the pure bytes that are stored
in the utf8 variable
// and *interpreting* them to this encoding
bytes_to_interpetZ = str.utf8().c_str();
str.Set(bytes_to_interpetZ, kCFStringEncodingUTF8);
}
return str;
}
PyObject* PyString_FromString(const SuperString& str)
{
SuperString localStr(str);
// localStr is the real, actual unicode string
// but we must *interpret* it as macRoman, then take these
"macRoman" characters
// and "convert" them to unicode for Python to "get it"
const UTF8Char *bytes_to_interpetZ = localStr.utf8().c_str();
// take the utf8 bytes (actual utf8 prepresentation of string)
// and say "no, these bytes are macRoman"
localStr.Set(bytes_to_interpetZ, kCFStringEncodingMacRoman);
// okay so now we have unicode of MacRoman characters (!?)
// return the underlying utf8 bytes of THAT as our string
return PyString_FromString(localStr.utf8Z());
}
And here is the results from running the script:
18: ---
18: Original string: frøânçïé
18: converting...
18: it worked: frøânçïé
18: ---
18: ---
18: Original string: 控件
18: converting...
18: it worked: 控件
18: ---
Now the thing that absolutely utterly baffles me (if i'm not baffled enough) is
that i get the EXACT same results on both Mac and Windows. Why do they both
insist on interpreting my script's bytes as MacRoman?
--
http://mail.python.org/mailman/listinfo/python-list
Re: can't get utf8 / unicode strings from embedded python
fair enough. I can provide further proof of strangeness.
here is my latest script: this is saved on disk as a UTF8 encoded file, and
when viewing as UTF8, it shows the correct characters.
==
# -*- coding: utf-8 -*-
import time, kjams, kjams_lib
def log_success(msg, successB, str):
if successB:
print msg + " worked: " + str
else:
print msg + "failed: " + str
def do_test(orig_str):
cmd_enum = kjams.enum_cmds()
print "---"
print "Original string: " + orig_str
print "converting..."
oldstr = orig_str;
newstr = kjams_lib.do_command(cmd_enum.kScriptCommand_Unicode_Test,
oldstr)
log_success("first", oldstr == newstr, newstr);
oldstr = unicode(orig_str, "UTF-8")
newstr = kjams_lib.do_command(cmd_enum.kScriptCommand_Unicode_Test,
oldstr)
newstr = unicode(newstr, "UTF-8")
log_success("second", oldstr == newstr, newstr);
oldstr = unicode(orig_str, "UTF-8")
oldstr.encode("UTF-8")
newstr = kjams_lib.do_command(cmd_enum.kScriptCommand_Unicode_Test,
oldstr)
newstr = unicode(newstr, "UTF-8")
log_success("third", oldstr == newstr, newstr);
print "---"
def main():
do_test("frøânçïé")
do_test("控件")
#-
if __name__ == "__main__":
main()
==
and the latest results:
20: ---
20: Original string: frøânçïé
20: converting...
20: first worked: frøânçïé
20: second worked: frøânçïé
20: third worked: frøânçïé
20: ---
20: ---
20: Original string: 控件
20: converting...
20: first worked: 控件
20: second worked: 控件
20: third worked: 控件
20: ---
now, given the C++ source code, this should NOT work, given that i'm doing some
crazy re-coding of the bytes.
so, you see, it does not matter whether i pass "unicode" strings or regular
"strings", they all translate to the same, weird macroman.
for completeness, here is the C++ code that the script calls:
===
case kScriptCommand_Unicode_Test: {
pyArg = iterP.NextArg_OrSyntaxError();
if (pyArg.get()) {
SuperString str = pyArg.GetAs_String();
resultObjP = PyString_FromString(str);
}
break;
}
===
--
http://mail.python.org/mailman/listinfo/python-list
Re: can't get utf8 / unicode strings from embedded python
i got it!! OMG! so sorry for the confusion, but i learned a lot, and i can
share the result:
the CORRECT code *was* what i had assumed. the Python side has always been
correct (no need to put "u" in front of strings, it is known that the bytes are
utf8 bytes)
it was my "run script" function which read in the file. THAT was what was
"reinterpreting" the utf8 bytes as macRoman (on both platforms). correct code
below:
SuperString ScPyObject::GetAs_String()
{
SuperString str;
if (PyUnicode_Check(i_objP)) {
ScPyObject utf8Str(PyUnicode_AsUTF8String(i_objP));
str = utf8Str.GetAs_String();
} else {
// calling "uc" on this means "assume this is utf8"
str.Set(uc(PyString_AsString(i_objP)));
}
return str;
}
PyObject* PyString_FromString(const SuperString& str)
{
return PyString_FromString(str.utf8Z());
}
--
http://mail.python.org/mailman/listinfo/python-list
Re: can't get utf8 / unicode strings from embedded python
i am already doing (3), and all is working perfectly. bytestring literals are fine, i'm not sure what this trouble is that you speak of. note that i'm not using PyRun_AnyFile(), i'm loading the script myself, assumed as utf8 (which was my original problem, i had assumed it was macRoman), then calling PyRun_SimpleString(). it works flawlessly now, on both mac and windows. -- http://mail.python.org/mailman/listinfo/python-list
Re: can't get utf8 / unicode strings from embedded python
I am very sorry that I have offended you to such a degree you feel it necessary to publicly eviscerate me. Perhaps I could have worded it like this: "So far I have not seen any troubles including unicode characters in my strings, they *seem* to be fine for my use-case. What kind of trouble has been seen with this by others?" Really, I wonder why you are so angry at me for having made a mistake? I'm going to guess that you don't have kids. -- http://mail.python.org/mailman/listinfo/python-list
Re: can't get utf8 / unicode strings from embedded python
Thank you for your thoughtful and thorough response. I now understand much better what you (and apparently the others) were warning me against and I will certainly consider that moving forward. I very much appreciate your help as I learn about python and embedding and all these crazy encoding problems. > What do kids have to do with this? When a person has children, they quickly learn that the best way to deal with some one who seems to be not listening or having a tantrum: show understanding and compassion, restraint and patience, as you, in the most neutral way that you can, gently bit firmly guide said person back on track. You learn that if you instead express your frustration at said person, that it never, ever helps the situation, and only causes more hurt to be spread around to the very people you are ostensibly attempting to help. > Are you an adult or a child? Perhaps my comment was lost in translation, but this is rather the question that I was obliquely asking you. *wink right back* In any case I thank you for your help, which has in fact been quite great! My demo script is working, and I know now to properly advise my script writers regarding how to properly encode strings. -- http://mail.python.org/mailman/listinfo/python-list
Re: can't find win32api from embedded pyrun call
I find i'm having this problem, but the solution you found isn't quite specific
enough for me to be able to follow it.
I'm embedding Python27 in my app. I have users install ActivePython27 in order
to take advantage of python in my app, so the python installation can't be
touched as it's on a user's machine.
When I attempt to do:
>import win32api
i get this:
>Traceback (most recent call last):
> File "startup.py", line 5, in
>ImportError: DLL load failed: The specified module could not be found.
I someone suggested i manually load the dependent libraries in the correct
order, like this:
>import pywintypes
>import pythoncom
>import win32api
but then i get this:
>Traceback (most recent call last):
> File "startup.py", line 3, in
> File "C:\Python27\lib\site-packages\win32\lib\pywintypes.py", line 124, in
>
>__import_pywin32_system_module__("pywintypes", globals())
> File "C:\Python27\lib\site-packages\win32\lib\pywintypes.py", line 64, in
> __import_pywin32_system_module__
>import _win32sysloader
ImportError: DLL load failed: The specified module could not be found.
the ultimate goal here is actually to do this:
>from win32com.client.gencache import EnsureDispatch
which currently yields:
>Traceback (most recent call last):
> File "startup.py", line 3, in
> File "C:\Python27\lib\site-packages\win32com\__init__.py", line 5, in
>
>import win32api, sys, os
>ImportError: DLL load failed: The specified module could not be found.
So, if anyone has any idea, that would be super duper great. thanks so much!
notes: my paths are definitely set correctly
--
https://mail.python.org/mailman/listinfo/python-list
Re: can't find win32api from embedded pyrun call
note that when the script is called, i DO see this in the output window: > 'kJams 2 Debug.exe': Loaded 'C:\Python27\Lib\site-packages\win32\win32api.pyd' > 'kJams 2 Debug.exe': Loaded 'C:\Windows\SysWOW64\pywintypes27.dll' > 'kJams 2 Debug.exe': Unloaded > 'C:\Python27\Lib\site-packages\win32\win32api.pyd' > 'kJams 2 Debug.exe': Unloaded 'C:\Windows\SysWOW64\pywintypes27.dll' -- https://mail.python.org/mailman/listinfo/python-list
Re: can't find win32api from embedded pyrun call
the problem was: ActivePython does not install debug libraries, so you must link with release libraries in your project. but if you run the debug version, you're linking against debug libraries which conflict with the ones linked to by python. "fixed" by running the release version. basically, it's not possible to debug with ActivePython due to ActiveState not including debug libs. grr -- https://mail.python.org/mailman/listinfo/python-list
Re: Cannot run a single MySQLdb execute....
Νίκος Γκρ33κ : >> What paramstyle are you using? > >Yes it is Chris, but i'am not sure what exactly are you asking me. >Please if you cna pout it even simper for me, thank you. For instance: >>> import MySQLdb >>> MySQLdb.paramstyle 'format' FWIW and HTH, DC -- http://mail.python.org/mailman/listinfo/python-list
Re: Functional vs. Object oriented API
> Roy Smith > As part of our initial interview screen, we give applicants some small > coding problems to do. One of the things we see a lot is what you could > call "Java code smell". This is our clue that the person is really a > Java hacker at heart who just dabbles in Python but isn't really fluent. > ... > It's not just LongVerboseFunctionNamesInCamelCase(). Nor is it code > that looks like somebody bought the Gang of Four patterns book and is > trying to get their money's worth out of the investment. The real dead > giveaway is when they write classes which contain a single static method > and nothing else. I may have some lingering Java smell myself, although I've been working mostly in Python lately, but my reaction here is that's really I don't know "BASIC smell" or something; a class that contains a single static method and nothing else isn't wonderful Java design style either. > That being said, I've noticed in my own coding, it's far more often that > I start out writing some functions and later regret not having initially > made it a class, than the other way around. That's as true in my C++ > code as it is in my Python. Definitely. > Once you start having state (i.e. data) and behavior (i.e. functions) in > the same thought, then you need a class. If you find yourself passing > the same bunch of variables around to multiple functions, that's a hint > that maybe there's a class struggling to be written. And I think equally to the point, even if you have only data, or only functions, right now, if the thing in question has that thing-like feel to it :) you will probably find yourself with both before you're done, so you might as well make it a class now... DC -- http://mail.python.org/mailman/listinfo/python-list
problem adding list values
Hi all,
I am fairly new to Python and trying to figure out a syntax error
concerning lists and iteration through the same. What I am trying to do is
sum a list of float values and store the sum in a variable for use later.
The relevant code looks like this -
def getCredits():
""" This function asks the user to input any credits not shown on their
bank statement
It returns the sum(converted to float) of the entered credits """
global credits
credlist = []
credits = 0.0
temp = 0.0
print "Now you need to enter any credits not shown on your bank statement
\n"
print "Please enter a zero (0) once all credits have been entered \n"
raw_input("Hit 'Enter' to continue \n")
temp = float(raw_input("Please enter the first credit \n"))
while temp != 0:
credlist.append(temp)
temp = float(raw_input("Please enter the next credit \n"))
i = 0
for i in credlist:
credits += credlist[i]
i = i + 1
return credits
And the syntax error I get is this -
Traceback (most recent call last):
File "./BankReconciler_Rev1.py", line 129, in ?
main()
File "./BankReconciler_Rev1.py", line 116, in main
getCredits()
File "./BankReconciler_Rev1.py", line 60, in getCredits
credits += credlist[i]
TypeError: list indices must be integers
If anyone can point me in the right direction, I would greatly appreciate
it.
Thanks in advance
--
http://mail.python.org/mailman/listinfo/python-list
Re: Coding style
"Carl Banks" <[EMAIL PROTECTED]> writes: > Patrick Maupin wrote: >> PTY wrote: >> >> > It looks like there are two crowds, terse and verbose. I thought terse >> > is perl style and verbose is python style. BTW, lst = [] was not what >> > I was interested in :-) I was asking whether it was better style to >> > use len() or not. >> >> It's not canonical Python to use len() in this case. From PEP 8: >> >> - For sequences, (strings, lists, tuples), use the fact that empty >> sequences are false. >> >> Yes: if not seq: >>if seq: >> >> No: if len(seq) >> if not len(seq) >> >> The whole reason that a sequence supports testing is exactly for this >> scenario. This is not an afterthought -- it's a fundamental design >> decision of the language. > > That might have made sense when Python and string, list, tuple were the > only sequence types around. > > Nowadays, Python has all kinds of spiffy types like numpy arrays, > interators, generators, etc., for which "empty sequence is false" just > doesn't make sense. If Python had been designed with these types in > mind, I'm not sure "empty list is false" would have been part of the > language, let alone recommend practice. Bruno's already mentioned that iterators and generators aren't sequences. Numpy arrays act like the other sequence types: >>> a = numpy.array([]) >>> a array([], dtype=int64) >>> len(a) 0 >>> bool(a) False (0-dimensional numpy arrays are pathological anyways) -- |>|\/|< /--\ |David M. Cooke |cookedm(at)physics(dot)mcmaster(dot)ca -- http://mail.python.org/mailman/listinfo/python-list
Re: Coding style
[EMAIL PROTECTED] (David M. Cooke) writes: > > Bruno's already mentioned that iterators and generators aren't > sequences. Numpy arrays act like the other sequence types: > >>>> a = numpy.array([]) >>>> a > array([], dtype=int64) >>>> len(a) > 0 >>>> bool(a) > False > > (0-dimensional numpy arrays are pathological anyways) *cough* as a Numpy developer I should know better. Numpy arrays that have more than one element don't work in a boolean context: >>> a = numpy.array([1,2]) >>> bool(a) Traceback (most recent call last): File "", line 1, in ? ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all() The reason for this is that it really was a common source of errors, because of the rich comparision semantics used. If a and b are numpy arrays, 'a == b' is an array of booleans. Numpy arrays of one element act like scalars in boolean contexts: >>> a = numpy.array([0]) >>> bool(a) False >>> a = numpy.array([1]) >>> bool(a) True (this is partly because we define a comphensive hierarchy of scalar types to match those available in C). -- |>|\/|< /--\ |David M. Cooke |cookedm(at)physics(dot)mcmaster(dot)ca -- http://mail.python.org/mailman/listinfo/python-list
Re: setting a breakpoint in the module
"Jason Jiang" <[EMAIL PROTECTED]> writes: > Hi, > > I have two modules: a.py and b.py. In a.py, I have a function called > aFunc(). I'm calling aFunc() from b.py (of course I import module a first). > The question is how to directly set a breakpoint in aFunc(). > > The way I'm doing now is to set a breakpoint in b.py at the line to call > aFunc(), 'c' to it, then 's' to step in, then set the breakpoint inside > aFunc() by 'b lineNumber'. It's too cumbersome. You can also add in your source import pdb; pdb.set_trace() at the point you want the debugger to stop. Useful if you want to break after some failing condition, for instance. -- |>|\/|< /--\ |David M. Cooke |cookedm(at)physics(dot)mcmaster(dot)ca -- http://mail.python.org/mailman/listinfo/python-list
Re: good library for pdf
"Murali" <[EMAIL PROTECTED]> writes: > Pulling out pages from existing PDF files can be done with Open Source > stuff. The simplest would be pdftk (PDF Toolkit). The most fancy will > be using latex and the pdfpages package together with pdflatex. > > - Murali There's also pyPDF, at http://pybrary.net/pyPdf/. I haven't tried it, but it looks interesting. -- |>|\/|< /------\ |David M. Cooke |cookedm(at)physics(dot)mcmaster(dot)ca -- http://mail.python.org/mailman/listinfo/python-list
Re: Scientific Computing with NumPy
"linda.s" <[EMAIL PROTECTED]> writes: > where to download numpy for Python 2.3 in Mac? > Thanks! > Linda I don't know if anybody's specifically compiled for 2.3; I think most of the developers on mac are using 2.4 :-) But (assuming you have the developer tools installed) it's really to compile: python setup.py build && python setup.py install. Do you need Tiger (10.4) or Panther (10.3) compatibility? -- |>|\/|< /------\ |David M. Cooke |cookedm(at)physics(dot)mcmaster(dot)ca -- http://mail.python.org/mailman/listinfo/python-list
Re: Python advocacy in scientific computation
Robert Kern <[EMAIL PROTECTED]> writes: > sturlamolden wrote: > >> 5. Versioning control? For each program there is only one developer and >> a single or a handful users. > > I used to think like that up until two seconds before I entered this gem: > > $ rm `find . -name "*.pyc"` > > Okay, I didn't type it exactly like that; I was missing one character. I'll > let > you guess which. I did that once. I ended up having to update decompyle to run with Python 2.4 :-) Lost comments and stuff, but the code came out great. -- |>|\/|< /------\ |David M. Cooke |cookedm(at)physics(dot)mcmaster(dot)ca -- http://mail.python.org/mailman/listinfo/python-list
Re: Wrapping c functions
Andrew Dalke <[EMAIL PROTECTED]> writes:
> Glenn Pierce wrote:
>> if (!PyArg_ParseTuple(args, "isi", &format, filename, &flags))
>> return NULL;
>
> Shouldn't that be &filename ? See
> http://docs.python.org/ext/parseTuple.html
> for examples.
>
>
>> dib = FreeImage_Load(format, filename, flags);
>
>> Also I have little Idea what to return from the function. FIBITMAP * is
>> an opaque pointer
>> that I pass to other FreeImage functions, I pretty certain
>> Py_BuildValue("o", dib) is wrong.
>
> If it's truly opaque and you trust your use of the code you can
> cast it to an integer, use the integer in the Python code, and
> at the Python/C interface cast the integer back to a pointer.
> Of course if it no longer exists you'll get a segfault.
>
> If you want more type safety you can use the SWIG approach and
> encode the pointers as a string, with type information and
> pointer included.
Better yet, use a CObject. That way, a destructor can be added so as
to not leak memory. Type info could be included in the desc field.
return PyCObject_FromVoidPtr(dib, NULL)
(the NULL can be replaced with a routine that will free the image.)
--
|>|\/|<
/--\
|David M. Cooke
|cookedm(at)physics(dot)mcmaster(dot)ca
--
http://mail.python.org/mailman/listinfo/python-list
Re: editor for shelve files
"Amir Michail" <[EMAIL PROTECTED]> writes: > Hi, > > Is there a program for editing shelve databases? > > Amir I doubt it. A shelf is basically just a file-based dictionary, where the keys must be strings, while the values can be arbitrary objects. An editor could handle the keys, but most likely not the values. If you have an editor for arbitrary objects, you could probably make an editor for shelfs easily enough :-) Do you have a specific use in mind? That would be easier to handle than the general case. -- |>|\/|< /------\ |David M. Cooke |cookedm(at)physics(dot)mcmaster(dot)ca -- http://mail.python.org/mailman/listinfo/python-list
Re: computer algebra packages
Fernando Perez <[EMAIL PROTECTED]> writes: > Rahul wrote: > >> Hi. >> The reason is simple enough. I plan to do some academic research >> related to computer algebra for which i need some package which i can >> call as a library. Since i am not going to use the package >> myself..(rather my program will)..it will be helpful to have a python >> package since i wanted to write the thing in python. if none is >> available then probably i will need to work on an interface to some >> package written in some other language or work in that language itself. > > I've heard of people writing a Python MathLink interface to Mathematica, which > essentially turns Mathematica into a Python module. But I don't have any > references handy, sorry, and as far as I remember it was done as a private > contract. But it's doable. It should also be doable with Maple, using the OpenMaple API. I've looked at it, and it should be possible. I haven't had the time to actually do anything, though :-) -- |>|\/|< /--\ |David M. Cooke |cookedm(at)physics(dot)mcmaster(dot)ca -- http://mail.python.org/mailman/listinfo/python-list
Re: Annoying behaviour of the != operator
Greg Ewing <[EMAIL PROTECTED]> writes:
> Rocco Moretti wrote:
>
> > This gives the wacky world where
>> "[(1,2), (3,4)].sort()" works, whereas "[1+2j, 3+4j].sort()" doesn't.
>
> To solve that, I would suggest a fourth category of "arbitrary
> ordering", but that's probably Py3k material.
We've got that: use hash().
[1+2j, 3+4j].sort(key=hash)
Using the key= arg in sort means you can do other stuff easily of course:
by real part:
import operator
[1+2j, 3+4j].sort(key=operator.attrgetter('real'))
by size:
[1+2j, 3+4j].sort(key=abs)
and since .sort() is stable, for those numbers where the key is the
same, the order will stay the same.
--
|>|\/|<
/--\
|David M. Cooke
|cookedm(at)physics(dot)mcmaster(dot)ca
--
http://mail.python.org/mailman/listinfo/python-list
Re: without shell
Donn Cave <[EMAIL PROTECTED]> writes:
> In article <[EMAIL PROTECTED]>,
> Grant Edwards <[EMAIL PROTECTED]> wrote:
>
>> On 2005-06-10, Mage <[EMAIL PROTECTED]> wrote:
>>
>> >>py> file_list = os.popen("ls").read()
>> >>
>> >>Stores the output of ls into file_list.
>> >>
>> > These commands invoke shell indeed.
>>
>> Under Unix, popen will not invoke a shell if it's passed a
>> sequence rather than a single string.
>
> I suspect you're thinking of the popen2 functions.
> On UNIX, os.popen is posix.popen, is a simple wrapper
> around the C library popen. It always invokes the
> shell.
>
> The no-shell alternatives are spawnv (instead of
> system) and the popen2 family (given a sequence
> of strings.)
Don't forget the one module to rule them all, subprocess:
file_list = subprocess.Popen(['ls'], stdout=subprocess.PIPE).communicate()[0]
which by default won't use the shell (unless you pass shell=True to it).
--
|>|\/|<
/--\
|David M. Cooke
|cookedm(at)physics(dot)mcmaster(dot)ca
--
http://mail.python.org/mailman/listinfo/python-list
Re: Annoying behaviour of the != operator
Robert Kern <[EMAIL PROTECTED]> writes: > greg wrote: >> David M. Cooke wrote: >> >>>>To solve that, I would suggest a fourth category of "arbitrary >>>>ordering", but that's probably Py3k material. >>> >>>We've got that: use hash(). >>>[1+2j, 3+4j].sort(key=hash) >> What about objects that are not hashable? >> The purpose of arbitrary ordering would be to provide >> an ordering for all objects, whatever they might be. > > How about id(), then? > > And so the circle is completed... Or something like def uniquish_id(o): try: return hash(o) except TypeError: return id(o) hash() should be the same across interpreter invocations, whereas id() won't. -- |>|\/|< /--\ |David M. Cooke |cookedm(at)physics(dot)mcmaster(dot)ca -- http://mail.python.org/mailman/listinfo/python-list
Re: ANN: NumPy 0.9.8 released
"Travis E. Oliphant" <[EMAIL PROTECTED]> writes: > NumPy 0.9.8 has been released. It can be downloaded from > > http://numeric.scipy.org > > The release notes are attached. > > Best regards, > > -Travis Oliphant > NumPy 0.9.8 is a bug-fix and optimization release with a > few new features. The C-API was changed so that extensions compiled > against NumPy 0.9.6 will need re-compilation to avoid errors. > > The C-API should be stabilizing. The next release will be 1.0 which > will come out in a series of release-candidates during Summer 2006. > > There were many users and developers who contributed to the fixes for > this release. They deserve much praise and thanks. For details see > the Trac pages where bugs are reported and fixed. > > http://projects.scipy.org/scipy/numpy/ > > > * numpy should install now with easy_install from setuptools Note that you'll need to use the latest setuptools (0.6b1). The hacks I added to get easy_install and numpy.distutils to get along are hard enough without trying to be backward compatible :-( -- |>|\/|< /--\ |David M. Cooke |cookedm(at)physics(dot)mcmaster(dot)ca -- http://mail.python.org/mailman/listinfo/python-list
Re: Concurrency Email List
On 5/16/2009 5:26 PM, Aahz wrote: > On Sat, May 16, 2009, Pete wrote: > >> [email protected] is a new email list >> for discussion of concurrency issues in python. > > Is there some reason you chose not to create a list on > python.org? I'm not joining the list because Google > requires that you create a login. i too would join if it was hosted at python.org, and will not if it's hosted at google for the same reason. -- david -- http://mail.python.org/mailman/listinfo/python-list
Re: "pow" (power) function
"Russ" <[EMAIL PROTECTED]> writes: > Ben Cartwright wrote: >> Russ wrote: > >> > Does "pow(x,2)" simply square x, or does it first compute logarithms >> > (as would be necessary if the exponent were not an integer)? >> >> >> The former, using binary exponentiation (quite fast), assuming x is an >> int or long. >> >> If x is a float, Python coerces the 2 to 2.0, and CPython's float_pow() >> function is called. This function calls libm's pow(), which in turn >> uses logarithms. > > I just did a little time test (which I should have done *before* my > original post!), and 2.0**2 seems to be about twice as fast as > pow(2.0,2). That seems consistent with your claim above. > > I'm a bit surprised that pow() would use logarithms even if the > exponent is an integer. I suppose that just checking for an integer > exponent could blow away the gain that would be achieved by avoiding > logarithms. On the other hand, I would think that using logarithms > could introduce a tiny error (e.g., pow(2.0,2) = 3.96 <- made > up result) that wouldn't occur with multiplication. It depends on the libm implementation of pow() whether logarithms are used for integer exponents. I'm looking at glibc's (the libc used on Linux) implementation for Intel processors, and it does optimize integers. That routine is written in assembly language, btw. >> > Does "x**0.5" use the same algorithm as "sqrt(x)", or does it use some >> > other (perhaps less efficient) algorithm based on logarithms? >> >> The latter, and that algorithm is libm's pow(). Except for a few >> special cases that Python handles, all floating point exponentation is >> left to libm. Checking to see if the exponent is 0.5 is not one of >> those special cases. > > I just did another little time test comparing 2.0**0.5 with sqrt(2.0). > Surprisingly, 2.0**0.5 seems to take around a third less time. > > None of these differences are really significant unless one is doing > super-heavy-duty number crunching, of course, but I was just curious. > Thanks for the information. And if you are, you'd likely be doing it on more than one number, in which case you'd probably want to use numpy. We've optimized x**n so that it does handle n=0.5 and integers specially; it makes more sense to do this for an array of numbers where you can do the special manipulation of the exponent, and then apply that to all the numbers in the array at once. -- |>|\/|< /--\ |David M. Cooke |cookedm(at)physics(dot)mcmaster(dot)ca -- http://mail.python.org/mailman/listinfo/python-list
Re: (OT) Recommend FTP Client
On 11/12/2009 11:26 AM, Dave Angel wrote: > Try http://fireftp.mozdev.org/ in the past i found this to be buggy. i'd recommend something different. what is your OS? -- david -- http://mail.python.org/mailman/listinfo/python-list
Re: Compiler malware rebutted
On 11/13/2009 3:26 PM, Aahz wrote: > Ken Thompson's classic paper on bootstrapped malware > finally gets a rebuttal: > > http://lwn.net/Articles/360040/ thanks for pointing this out. -- david -- http://mail.python.org/mailman/listinfo/python-list
