Unicode charmap decoders slow

[Tony Nelson]

Is there a faster way to decode from charmaps to utf-8 than unicode()?

I'm writing a small card-file program.  As a test, I use a 53 MB MBox 
file, in mac-roman encoding.  My program reads and parses the file into 
messages in about 3..5 seconds, but takes about 13.5 seconds to iterate 
over the cards and convert them to utf-8:

for i in xrange(len(cards)):
u = unicode(cards[i], encoding)
cards[i] = u.encode('utf-_8')

The time is nearly all in the unicode() call.  It's not so much how much 
time it takes, but that it takes 4 times as long as the real work, just 
to do table lookups.

Looking at the source (which, if I have it right, is 
PyUnicode_DecodeCharmap() in unicodeobject.c), I think it is doing a 
dictionary lookup for each character.  I would have thought that it 
would make and cache a LUT the size of the charmap (and hook the 
relevent dictionary stuff to delete the cached LUT if the dictionary is 
changed).

I thought of using U"".translate(), but the unicode version is defined 
to be slow.  Is there some similar approach?  I'm almost (but not quite) 
ready to try it in Pyrex.

I'm new to Python.  I didn't google anything relevent on python.org or 
in groups.

TonyN.:'[EMAIL PROTECTED]
  '  
-- 
http://mail.python.org/mailman/listinfo/python-list

gtk.TextView.move_mark_onscreen() broken?

[Tony Nelson]

Is gtk.TextView.move_mark_onscreen() broken?  Perhaps only in Python's 
gtk module, in Python 2.3, gtk 2.4.14?  I'm asking here because I'm 
using gtk from Python and don't want to write a C program to verify my 
issue.  I've also tried gtk.TextView.scroll_to_mark() and 
gtk.TextView.place_cursor_onscreen(), and none of them want to do 
anything.  The rest of my program works, so I'm not a complete gtk bazo.

TonyN.:'[EMAIL PROTECTED]
  '  
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Unicode charmap decoders slow

[Tony Nelson]

In article <[EMAIL PROTECTED]>,
 "Martin v. Löwis" <[EMAIL PROTECTED]> wrote:

> Tony Nelson wrote:
> > Is there a faster way to decode from charmaps to utf-8 than unicode()?
> 
> You could try the iconv codec, if your system supports iconv:
> 
> http://cvs.sourceforge.net/viewcvs.py/python-codecs/practicecodecs/iconv/

I had seen iconv.  Even if my system supports it and it is faster than 
Python's charmap decoder, it might not be available on other systems.  
Requiring something unusual in order to do a trivial LUT task isn't an 
acceptable solution.  If I write a charmap decoder as an extension 
module in Pyrex I can include it with the program.  I would prefer a 
solution that doesn't even need that, preferably in pure Python.  Since 
Python does all the hard wark so fast it certainly could do it, and it 
can almost do it with "".translate().

TonyN.:'[EMAIL PROTECTED]
  '  <http://www.georgeanelson.com/>
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Unicode charmap decoders slow

[Tony Nelson]

In article <[EMAIL PROTECTED]>,
 "Martin v. Löwis" <[EMAIL PROTECTED]> wrote:

> Tony Nelson wrote:
> > I had seen iconv.  Even if my system supports it and it is faster than 
> > Python's charmap decoder, it might not be available on other systems.  
> > Requiring something unusual in order to do a trivial LUT task isn't an 
> > acceptable solution.  If I write a charmap decoder as an extension 
> > module in Pyrex I can include it with the program.  I would prefer a 
> > solution that doesn't even need that, preferably in pure Python.  Since 
> > Python does all the hard wark so fast it certainly could do it, and it 
> > can almost do it with "".translate().
> 
> Well, did you try a pure-Python version yourself?
> 
> table = [chr(i).decode("mac-roman","replace") for i in range(256)]
> 
> def decode_mac_roman(s):
>  result = [table[ord(c)] for c in s]
>  return u"".join(result)
> 
> How much faster than the standard codec is that?

It's .18x faster.

TonyN.:'[EMAIL PROTECTED]
  '  <http://www.georgeanelson.com/>
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Exception raising, and performance implications.

[Tony Nelson]

In article <[EMAIL PROTECTED]>,
 "leo" <[EMAIL PROTECTED]> wrote:

> Hello all -
> 
> I was wondering about the performance implications of explicitly
> raising exceptions to get information about the current frame.
> Something like what the inspect module does, with:

Python uses exceptions internally, using StopIteration to terminate the 
iterator in a for: loop.

> ---
> def currentframe():
> """Return the frame object for the caller's stack frame."""
> try:
> raise 'catch me'
> except:
> return sys.exc_traceback.tb_frame.f_back
> ---

This also does a traceback; you might want to measure the cost of that.

> I come from a java background, where Exceptions are a big Avoid Me, but
> are the performance implications the same in Python? We're expecting a
> big load on our app (100,000 users/hour) , so we'd like to be as tuned
> as possible.

Switching to Python, eh?  Remember to measure, measure, measure!

TonyN.:'[EMAIL PROTECTED]
  '  
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Controlling who can run an executable

[Tony Nelson]

In article <[EMAIL PROTECTED]>,
 "Cigar" <[EMAIL PROTECTED]> wrote:

> I am developing a program for a client.  She runs a shop where her
> clients bring in items for sale or short term buyback.  Development of
> the program has been going great but she's mentioned that there is a
> 'feature' coming up in the next couple of weeks that she'd like me to
> implement that has me a bit worried.
> 
> My client has told me a story of how she hired someone from a competing
> store and that person had brought a copy of the program her competition
> was using to track clients and transactions.  He couldn't demonstrate
> the program for one reason or another because it was protected in a way
> that neither could circumvent. (She didn't remember how it was
> protected, she had hired this person a long time ago.)
> 
> Now that I'm three months into the development of this program, my
> client tells me she would like to protect her investment by preventing
> her employees from doing the same to her.  (Going to the competition
> and using her program.)
 ...

Call the competition and ask them what they used.  Point out that it 
worked.  If they won't tell you, just look at their software until you 
find out.

TonyN.:'[EMAIL PROTECTED]
  '  
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: C Wrapper Function, crashing Python?

[Tony Nelson]

In article <[EMAIL PROTECTED]>,
 "Java and Swing" <[EMAIL PROTECTED]> wrote:

> one more update...
> 
> if I remove PyMem_Free and free(...) ...so no memory clean up...I can
> still only call doStuff 4 times, the 5th attemp crashes Python.
> 
> Java and Swing wrote:
> > update:
> > if I use C's free(result), free(a) free(b) instead of PyMem_Free...I
> > only get one successfuly use/call of doStuff.
> >
> > i.e.
> > // this works
> > doStuff(...)
> >
> > // python crashes here
> > doStuff(...)
> >
> > Java and Swing wrote:
> > > static PyObject *wrap_doStuff(PyObject *self, PyObject *args) {
> > >   // this will store the result in a Python object
> > >   PyObject *finalResult;
> > >
> > >   // get arguments from Python
> > >   char *result = 0;
> > >   char *in= 0;
> > >   char *aString = 0;
> > >   char *bString = 0;
> > >   MY_NUM *a;
> > >   MY_NUM *b;
> > >   int ok = PyArg_ParseTuple(args, "sss", &in, &aString, &bString);
> > >   if (!ok) return 0;
> > >
> > >   // do work to get a and b
> > >   // count - returns an int;  GetVal - returns a char *
> > >   a = GetVal(aString, count(aString, ","));
> > >   b = GetVal(bString, count(bString, ","));
> > >
> > >   // make function call, which returns a char *
> > >   result = doStuff(in, a, b);
> > >
> > >   // save result in Python string
> > >   finalResult = PyString_FromString(result);
> > >
> > >   // free memory
> > >   PyMem_Free(result);
> > >   PyMem_Free(a);
> > >   PyMem_Free(b);
> > >
> > >   // return the result as a Python string
> > >   return finalResult;
> > > }
> > >
> > > ...from python I can call this function 4 times...works fine.  WHen I
> > > call it for the fifth time python.exe crashes.  im thinking some memory
> > > problem in the wrapper function perhaps...but I am not sure.  The
> > > actually C function, doStuff can be called 5, 6,7...N times without a
> > > problem
> > > so i know its gotta be my wrapper.
> > > 
> > > Any ideas?  Thanks!
> 

I think your wrapper should look something like:

static PyObject *wrap_doStuff(PyObject *self, PyObject *args)
{
// this will store the result in a Python object
PyObject *finalResult;

// get arguments from Python
char *result = 0;
char *in= 0;
char *aString = 0;
char *bString = 0;
MY_NUM *a;
MY_NUM *b;

int ok = PyArg_ParseTuple(args, "sss", &in, &aString, &bString);
if (!ok) return 0;

// do work to get a and b
// count - returns an int;  GetVal - returns a char *
a = GetVal(aString, count(aString, ","));
b = GetVal(bString, count(bString, ","));

// make function call, which returns a char *
result = doStuff(in, a, b);

// save result in Python string
finalResult = PyString_FromString(result);

// free memory
free(result);
free(a);
free(b);

// return the result as a Python string
return finalResult;
}

You must match malloc() with free(), and PyMem_Malloc() with 
PyMem_Free().  Malloc() and free() usually crash /after/ the call that 
did the damage.  You may wish to avail yourself of your platform's 
malloc debugging facilities.

Note that I don't do this stuff in C, I do it in pyrex, and I'm new to 
it anyway, so there may still be something wrong.  Unless you are 
determined to learn how to do this in C, I think you should switch to 
pyrex.

TonyN.:'[EMAIL PROTECTED]
  '  
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: how to make this code faster

[Tony Nelson]

In article <[EMAIL PROTECTED]>,
 "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> wrote:

> def f(x,y):
> return math.sin(x*y) + 8 * x
> I have code like this:
> 
> def main():
> n = 2000
> a = zeros((n,n), Float)
> xcoor = arange(0,1,1/float(n))
> ycoor = arange(0,1,1/float(n))
> 
> 
> for i in range(n):
> for j in range(n):
> a[i,j] = f(xcoor[i], ycoor[j])  # f(x,y) = sin(x*y) + 8*x
> 
> print a[1000,1000]
> pass
> 
> if __name__ == '__main__':
> main()

I would guess that you are spending most of your time calculating 
sin(x*y).  To find out, just replace f(x,y) with 1, which will produce 
wrong results really fast, and see what that does to your execution time.

TonyN.:'[EMAIL PROTECTED]
  '  
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Can module access global from __main__?

[Tony Nelson]

In article <[EMAIL PROTECTED]>,
 "Neal  Norwitz" <[EMAIL PROTECTED]> wrote:

> Steve Holden wrote:
> > Neal Becker wrote:
> > >
> > > Still curious about the answer.  If I know that I am imported from 
> > > __main__,
> > > then I can do access X as sys.modules[__main__].X.  In general, I don't
> > > know how to determine who is importing me.
> > >
> > I don't think you can without huge amounts of introspection - it's even
> > worse than the "what's the name of this object" question that seems to
> > come up regularly.
> 
> import sys
> 
> frame = sys._getframe()
> caller = frame.f_back
> print 'Called from', caller.f_code.co_filename, caller.f_lineno
> # for more info, look into the traceback module
> 
> > A module can be imported from multiple modules, and
> > you only get to execute code on the first import.
> > Even then (on the first import) I am not sure how you could introspect
> > to find the answer you want.
> 
> You can install your own __import__() hook to catch all imports.
> 
> Just because you can do something, it doesn't follow that you should do
> it, especially in this case.  Unless you really, really need these
> tricks, they shouldn't be used.

Neal, I have a similar question (in that maybe I shouldn't do 
something), and you seem to know your way around modules, so I'll just 
butt in.  If this isn't OK I'll post as a new thread.

I have written a python module that uses some C functions.  I wrote the 
module in two parts, one python, one pyrex (C).  They need to share some 
globals.  (I use pyrex to handle ref counting.  I think I'm glad I did.)

At first they just sort of mutually imported each other, and it worked 
until I put tests in the python one and set it up to run them when it 
was __main__.  This reminded me that there are also other ways modules 
can be imported under different names, so I tried a different approach.

Now I'm just shoving references to the python module's globals into the 
pyrex module (first defining them in the pyrex module).  The pyrex 
module doesn't import the python module anymore.  This also works, even 
when the python module has a different name ("__main__").  I just feel 
dirty about it.

How does one normally make a module that uses some functions in C?  Will 
that approach work with pyrex?  With globals?

TonyN.:'[EMAIL PROTECTED]
  '  
-- 
http://mail.python.org/mailman/listinfo/python-list

[ANN] Speed up Charmap codecs with fastcharmap module

[Tony Nelson]

Fastcharmap is a python extension module that speeds up Charmap codecs 
by about 5 times.

 

Usage:

import fastcharmap
fastcharmap.hook('codec_name')

Fastcharmap will then speed up calls that use that codec, such as 
unicode(str, 'codec_name') and str.encode('codec_name'), and won't 
interfere with Charmap codecs that haven't been hooked.

Documentation is in PyDoc form:

import fastcharmap
help(fastcharmap)

Fastcharmap is available as a standard Python source tarball, binary 
tarball, and RPMs.  It isn't packaged for MSWindows yet (maybe soon), or 
for MOSX.  It is written in Python and Pyrex 0.9.3, but builds from .c 
source when Pyrex is not available.  A C compiler is required for source 
installs.  I have only used it with Python 2.3.4 on FC3 Linux, but it 
should work on Python 2.4 and on other platforms.

As fastcharmap is an extension module, it might not be available on a 
particular computer.  I handle that this way in a program of mine:

try:
import fastcharmap
except ImportError:
print "fastcharmap not available"
else:
fastcharmap.hook('mac_roman')

This is done on document open, which on Gnome / GTK is chatty anyway.

I wrote fastcharmap when I found that decoding a large amount of text 
was taking 3 times as long as loading the document from a file.  Python 
should be fast!  The application is a simple card-file program that can 
also open mbox files as cards.  I am using a 50 MB test file from a Mac 
that loads in about 4 seconds on my computer (wow!), and was decoding in 
about 13 seconds.  Now it takes 2 seconds to decode or encode.

Python developers are working on faster Charmap codecs for a future 
version of Python.  Fastcharmap may be useful until then, and shouldn't 
cause any problems when the new codecs are available.

As this is my first Python module, I'd like some experienced module 
authors and packagers to comment on it, before I make it into cheese.

TonyN.:'[EMAIL PROTECTED]
  '  
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How do you draw this layout with wxpython?

[Tony Nelson]

In article <[EMAIL PROTECTED]>,
 "Young H. Rhiu" <[EMAIL PROTECTED]> wrote:

> See: http://hilug.org/img/app_layout.GIF
> 
> I'm implementing an album-like application with wxpython but I'm new to
> wxPython though I know how to program with python. The problem is that
> it's not easy for me to deal with drawing layout stuff with wxpython.
> What layout I'm thinking of looks like the gif image above.
> 
> The application is about saving some comments for each pictures on the
> filesystem. There are two windows. The above one is a window for
> loading some images and the below one is a TextCtrl for writing
> comments and if img1, for instance, is clicked, the first notebook(?)
> folder should be active.
> 
> Below is the program I tried but I don't know how to attach a notebook
> window under the image window.
> Any help is greatly appreciated.
> 
> Thanks in Advance, Rhiu
> 
> My Code:
> from wxPython.wx import *
> 
> class smallAppFrame(wxFrame):
> def __init__(self, parent, id, title):
> wxFrame.__init__(self, parent = None, id = -1,
>  title = "MySmallApp", pos = wxPoint(200, 200),
>  size = wxSize(460, 200), style =
> wxDEFAULT_FRAME_STYLE)
> 
> self.bitmap01 = wxStaticBitmap(self, -1,
> wxEmptyBitmap(100,100))
> self.bitmap02 = wxStaticBitmap(self, -1,
> wxEmptyBitmap(100,100))
> self.bitmap03 = wxStaticBitmap(self, -1,
> wxEmptyBitmap(100,100))
> self.bitmap04 = wxStaticBitmap(self, -1,
> wxEmptyBitmap(100,100))
> 
> box = wxBoxSizer(wxHORIZONTAL)
> 
> box.Add((10,10))
> box.Add(self.bitmap01, 0, wxALIGN_CENTER)
> box.Add((10,10))
> box.Add(self.bitmap02, 0, wxALIGN_CENTER)
> box.Add((10,10))
> box.Add(self.bitmap03, 0, wxALIGN_CENTER)
> box.Add((10,10))
> box.Add(self.bitmap04, 0, wxALIGN_CENTER)
> box.Add((10,10))
> 
> self.SetAutoLayout(True)
> self.SetSizer(box)
> 
> class MySmallApp(wxApp):
> def OnInit(self):
> frame = smallAppFrame(None, -1, "MySmallApp")
> frame.Show(true)
> self.SetTopWindow(frame)
> return true
> 
> app = MySmallApp(0)
> app.MainLoop()

I've never used (or learned) wxWidgets, and I'm even new to GTK, but I 
think you just need a deeper structure.  SmallAppFrame() would contain 
two subwindows, probably using something like wxBoxSizer(wxVERTICAL).  
The top subwindow would contain what you've done now, and the bottom one 
would have the rest of your stuff.

TonyN.:'[EMAIL PROTECTED]
  '  
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Stripping ASCII codes when parsing

[Tony Nelson]

In article <[EMAIL PROTECTED]>,
 David Pratt <[EMAIL PROTECTED]> wrote:

> I am working with a text format that advises to strip any ascii control 
> characters (0 - 30) as part of parsing data and also the ascii pipe 
> character (124) from the data. I think many of these characters are 
> from a different time. Since I have never seen most of these characters 
> in text I am not sure how these first 30 control characters are all 
> represented (other than say tab (\t), newline(\n), line return(\r) ) so 
> what should I do to remove these characters if they are ever 
> encountered. Many thanks.

Most of those characters are hard to see.

Represent arbitrary characters in a string in hex: "\x00\x01\x02" or 
with chr(n).

If you just want to remove some characters, look into "".translate().  

nullxlate = "".join([chr(n) for n in xrange(256)])
delchars = nullxlate[:31] + chr(124)
outputstr = inputstr.translate(nullxlate, delchars)

TonyN.:'[EMAIL PROTECTED]
  '  
-- 
http://mail.python.org/mailman/listinfo/python-list

Mutual module imports

[Tony Nelson]

How does one normally make a Python extension module that has some parts 
in Python and some functions in C share globals between the Python and C 
functions?  Will that approach work with Pyrex?

I have written a Python module that uses some C functions.  I wrote the 
module in two parts, one Python, one Pyrex (C).  They need to share some 
globals.  (I use pyrex to handle ref counting.  I think I'm glad I did.)

At first they just sort of mutually imported each other, and it worked 
until I put tests in the Python one and set it up to run them when it is 
named "__main__".  What happened reminded me that there are also other 
ways modules can be imported under different names, so I tried a 
different approach.

Now the Python module imports the Pyrex module and just shoves 
references to its globals into the Pyrex module (the Pyrex module 
defines them as None).  The Pyrex module doesn't import the Python 
module anymore.  This also works, even when the Python module has a 
different name (e.g. "__main__").  I just feel dirty about it.

TonyN.:'[EMAIL PROTECTED]
  '  
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Stripping ASCII codes when parsing

[Tony Nelson]

In article <[EMAIL PROTECTED]>,
 David Pratt <[EMAIL PROTECTED]> wrote:

> This is very nice :-)  Thank you Tony.  I think this will be the way to  
> go.  My concern ATM is where it will be best to unicode. The data after  
> this will go into dict and a few processes and into database. Because  
> input source if not explicit encoding, I will have to assume ISO-8859-1  
> I believe but could well be cp1252 for most part ( because it says no  
> ASCII (0-30) but alright ASCII chars 128-254) and because most are  
> Windows users.  Am thinking to unicode after stripping these characters  
> and validating text, then unicoding (utf-8) so it is unicode in dict.  
> Then when I perform these other processes it should be uniform and then  
> it will go into database as unicode.  I think this should be ok.

Definitely "".translate() then unicode().  See the docs for 
"".translate().  As far as charset, well, if you can't know in advance 
you'll want to have some way to configure it for when it's wrong.  Also, 
maybe 255 is not allowed and should be checked for?

TonyN.:'[EMAIL PROTECTED]
  '  
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How best to reference parameters.

[Tony Nelson]

In article <[EMAIL PROTECTED]>,
 "David Poundall" <[EMAIL PROTECTED]> wrote:

> I am writing a scada package that has a significant amount of user
> defined parameters stored in text files that I wish to cleanly access
> in code.  By way of an example, a few lines from the configuration file
> would typically be ...
> 
> [Plant Outputs]
> Y0P1  Pump 1 Pressure
> Y1P2  Pump 2 Fluid Transfer Pump
> Y2P3  Pump 3 Vac Pump
> Y3P4  Pump 4 Vac Pump
> Y4P5  Pump 5 / Pump 1B
> Y5P6  Pump 6 / Pump 2B
> Y6M
> Y7D
> Y10   E
> Y11   F
> 
> I can read these values in as dictionary items and refernce them in
> code like this...
> 
> Y['P4'] = 1   # Which will ultimately switch my pump on
> Y['P3'] = 0   # Which will ultimately switch my pump off
> 
> but I would much rather reference the plant outputs like this ...
> 
> Y.P4 = 1
> Y.P3 = 0
 ...

d = {'a':1, 'b':2, 'c':3}

class foo:
def __init__(self, d):
self.__dict__.update(d)

f = foo(d)

print f.a, f.b, f.c

(retyped from memory)

TonyN.:'[EMAIL PROTECTED]
  '  
-- 
http://mail.python.org/mailman/listinfo/python-list

Any Pythonic GTK Undo library?

[Tony Nelson]

I'm looking for a "pythonic" GTK Undo library/class.  It would have a 
framework for Undo/Redo, and would provide Undo/Redo for TextView, 
Entry, and containers and other classes.  In a "batteries included" 
fashion, just instantiating a "UndoableTextView" or "UndoableEntry" or 
"UndoableContainer" would provide Undo and Redo in the right-click menu; 
simply connecting such an object to an "UndoableUIManager" would take 
care of the stock items in the menus and toolbar; and there would be a 
simple connection to some sort of "UndoableDocument" interface or mix-in 
for more global sequencing of Undo/Redo.  Does something like this exist 
for Python or GTK?  Googling didn't turn up anything useful.

I'm disappointed that GTK doesn't do this already.  Making my own seems 
doable, but a fair amount of work.  If there isn't some such thing 
already, is there interest in using one that I make?

I know about GUndo, which doesn't implement any actual undo operations; 
the actual operations would need to be coded for TextView and Entry and 
anything else.

TonyN.:'[EMAIL PROTECTED]
  '  
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Any Pythonic GTK Undo library?

[Tony Nelson]

In article <[EMAIL PROTECTED]>,
 Dave Cook <[EMAIL PROTECTED]> wrote:

> On 2005-10-29, Tony Nelson <[EMAIL PROTECTED]> wrote:
> 
> > I'm looking for a "pythonic" GTK Undo library/class.  It would have a 
> 
> You might ask the authors of Kiwi if they plan to add undo/redo.  Or help
> them add it if you can.
> 
> http://www.async.com.br/projects/kiwi/
> 

Well, after I implement it myself, if I do, I could give them the code 
to port to kiwi.


> It would be great to have this feature in the Gtk C API, though.  I do see
> some relevant bugzilla entries:
> 
> http://bugzilla.gnome.org/show_bug.cgi?id=316551
> 
> You might want to make a new request for a general undo/redo interface.

Well, after I implement it myself, if I do, I could give them the code 
to translate to C.

TonyN.:'[EMAIL PROTECTED]
  '  <http://www.georgeanelson.com/>
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Scanning a file

[Tony Nelson]

In article <[EMAIL PROTECTED]>,
 [EMAIL PROTECTED] wrote:

> Steve Holden wrote:
> > Indeed, but reading one byte at a time is about the slowest way to
> > process a file, in Python or any other language, because it fails to
> > amortize the overhead cost of function calls over many characters.
> >
> > Buffering wasn't invented because early programmers had nothing better
> > to occupy their minds, remember :-)
> 
> Buffer, and then read one byte at a time from the buffer.

Have you mesured it?

#!/usr/bin/python
'''Time some file scanning.
'''

import sys, time

f = open(sys.argv[1])
t = time.time()
while True:
b = f.read(256*1024)
if not b:
break
print 'initial read', time.time() - t
f.close()

f = open(sys.argv[1])
t = time.time()
while True:
b = f.read(256*1024)
if not b:
break
print 'second read', time.time() - t
f.close()

if 1:
f = open(sys.argv[1])
t = time.time()
while True:
b = f.read(256*1024)
if not b:
break
for c in b:
pass
print 'third chars', time.time() - t
f.close()

f = open(sys.argv[1])
t = time.time()
n = 0
srch = '\x00\x00\x01\x00'
laplen = len(srch)-1
lap = ''
while True:
b = f.read(256*1024)
if not b:
break
n += (lap+b[:laplen]).count(srch)
n += b.count(srch)
lap = b[-laplen:]
print 'fourth scan', time.time() - t, n
f.close()


On my (old) system, with a 512 MB file so it won't all buffer, the 
second time I get:

initial read 14.513395071
second read 14.8771388531
third chars 178.250257969
fourth scan 26.1602909565 1

TonyN.:'[EMAIL PROTECTED]
  '  
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: NTFS reparse points

[Tony Nelson]

In article <[EMAIL PROTECTED]>,
 Stanislaw Findeisen <[EMAIL PROTECTED]> wrote:
 ...
> However I can't see FILE_ATTRIBUTE_REPARSE_POINT turned on in any file / 
> directory shortcuts I create. In fact the only attribute set in 
> shortcuts created using Windows Explorer is FILE_ATTRIBUTE_ARCHIVE. (I 
> am using GetFileAttributes() to examine this.)
 ...

Shortcuts are files with a .lnk extention.  Reparse points are NTFS 
directory data.

TonyN.:'[EMAIL PROTECTED]
  '  
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: when and how do you use Self?

[Tony Nelson]

In article <[EMAIL PROTECTED]>,
 "Tieche Bruce A MSgt USMTM/AFD" <[EMAIL PROTECTED]> 
 wrote:

> I am new to python,

> Could someone explain (in English) how and when to use self?

> I have been reading, and haven't found a good example/explanation

 is a good explanation of just about all of 
Python.  You should read it.  It explains when to use "self".

TonyN.:'[EMAIL PROTECTED]
  '  
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: ? MDI depreciated

[Tony Nelson]

In article <[EMAIL PROTECTED]>,
 "LenS" <[EMAIL PROTECTED]> wrote:

> Hate to ask this dum question (since I've been hiding under a rock).
> But if the MDI UI model is/was depreciated.  What is the new UI model.
> 
> Would love some links that explain in gerneral and specific terms.

In article <[EMAIL PROTECTED]>,
 "Brendan" <[EMAIL PROTECTED]> wrote:

> This is probably a question better suited for a wxPython or MSDN
> newsgroup.  What OS are you referring to?  What GUI toolkit are you
> using?
> 
> Microsoft's office on Windows has moved to a model where every document
> has its own toolbar, menubar, and taskbar entry.  Windows developers
> tend to mimic MS Office, so many are also moving to this model. Mac
> apps have never had MDI.

MS also uses a "Tabbed" version of MDI where only one document at a time 
is visible.  Sometimes this is /implemented/ using the MDI APIs (and 
they also have some MDI apps that don't use the APIs; go figure).  Gnome 
uses Tabbed windows as well; see Gedit, which opens documents in tabs, 
though they can be dragged out into their own windows.  In GTK, at 
least, the Tabbed interface is easily done as a Notebook.

MacOS apps used MDI from the beginning of Multifinder; it just worked 
better than on MSWindows because instead of a grey background you got to 
see the rest of the desktop and the other apps.  On MacOS, MDI was 
referred to as "Layers".  If on MSWindows MDI windows were always 
maximized, had no grey background, and hid the MDI Frame when not in 
front, they would be almost exactly what MacOS did.

MOSX, being a version of NextOS and NextStep, has the more "advanced" 
no-layer, no-MDI UI, and Apple recommends that each app should have only 
one window.

TonyN.:'[EMAIL PROTECTED]
  '  
-- 
http://mail.python.org/mailman/listinfo/python-list

Validate string as UTF-8?

[Tony Nelson]

I'd like to have a fast way to validate large amounts of string data as 
being UTF-8.

I don't see a fast way to do it in Python, though:

unicode(s,'utf-8').encode('utf-8)

seems to notice at least some of the time (the unicode() part works but 
the encode() part bombs).  I don't consider a RE based solution to be 
fast.  GLib provides a routine to do this, and I am using GTK so it's 
included in there somewhere, but I don't see a way to call GLib 
routines.  I don't want to write another extension module.

Is there a (fast) Python function to validate UTF-8 data?

Is there some other fast way to validate UTF-8 data?

Is there a general way to call GLib functions?

TonyN.:'[EMAIL PROTECTED]
  '  
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Validate string as UTF-8?

[Tony Nelson]

In article <[EMAIL PROTECTED]>,
 david mugnai <[EMAIL PROTECTED]> wrote:

> On Sun, 06 Nov 2005 18:58:50 +0000, Tony Nelson wrote:
> 
> [snip]
> 
> > Is there a general way to call GLib functions?
> 
> ctypes?
> http://starship.python.net/crew/theller/ctypes/

Umm.  Might be easier to write an extension module.

TonyN.:'[EMAIL PROTECTED]
  '  <http://www.georgeanelson.com/>
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Validate string as UTF-8?

[Tony Nelson]

In article <[EMAIL PROTECTED]>,
 "Fredrik Lundh" <[EMAIL PROTECTED]> wrote:

> Tony Nelson wrote:
> 
> > I'd like to have a fast way to validate large amounts of string data as
> > being UTF-8.
> 
> define "validate".

All data conforms to the UTF-8 encoding format.  I can stand if someone 
has made data that impersonates UTF-8 that isn't really Unicode.


> > I don't see a fast way to do it in Python, though:
> >
> > unicode(s,'utf-8').encode('utf-8)
> 
> if "validate" means "make sure the byte stream doesn't use invalid
> sequences", a plain
> 
> unicode(s, "utf-8")
> 
> should be sufficient.

You are correct.  I misunderstood what was happening in my code.  I 
apologise for wasting bandwidth and your time (and I wasted my own time 
as well).

Indeed, unicode(s, 'utf-8') will catch the problem and is fast enough 
for my purpose, adding about 25% to the time to load a file.

TonyN.:'[EMAIL PROTECTED]
  '  <http://www.georgeanelson.com/>
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: modifying small chunks from long string

[Tony Nelson]

In article <[EMAIL PROTECTED]>,
 "MackS" <[EMAIL PROTECTED]> wrote:

> Hello everyone
> 
> I am faced with the following problem. For the first time I've asked
> myself "might this actually be easier to code in C rather than in
> python?", and I am not looking at device drivers. : )
> 
> This program is meant to process relatively long strings (10-20 MB) by
> selectively modifying small chunks one at a time. Eg, it locates
> approx. 1000-2000 characters and modifies them. Currently I was doing
> this using a string object but it is getting very slow: although I only
> modify a tiny bit of the string at a time, a new entire string gets
> created whenever I "merge" it with the rest. Eg,
> 
> shortstr = longstr[beg:end]
> 
> # edit shortstr...
> 
> longstr = longstr[:beg] + shortstr + longstr[end:] # new huge string is
> created!!
> 
> Can I get over this performance problem without reimplementing the
> whole thing using a barebones list object? I though I was being "smart"
> by avoiding editing the long list, but then it struck me that I am
> creating a second object of the same size when I put the modified
> shorter string in place...

A couple of minutes experimenting with array.array at the python command 
line indicates that it will work fine for you.  Quite snappy on a 16 MB 
array, including a slice assignment of 1 KB near the beginning.  
Array.array is probably better than lists for speed, and uses less 
memory.  It is the way to go if you are going to be randomly editing all 
over the place but don't need to convert to string often.

MutableString warns that it is very slow.  It seems to work by having a 
string data item that it keeps replacing.  I didn't try it.


> shortstr = longstr[beg:end]
> 
> # edit shortstr...
> 
> longstr = longstr[:beg] + shortstr + longstr[end:] # new huge string is
> created!!

Replace this with slice assignment:

longarray = array.array('c',longstr) # once only at beginning!

shortstring = longarray[beg:end].tostring() # or just edit an array

# edit shortstring (or shortarray)

longarray[beg:end] = array.array('c',shortstr)

longstring = longarray.tostring() # if needed

TonyN.:'[EMAIL PROTECTED]
  '  
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python Library Reference - question

[Tony Nelson]

In article <[EMAIL PROTECTED]>,
 [EMAIL PROTECTED] wrote:

> The "Python LIbrary Reference" at
> http://docs.python.org/lib/contents.html seems to be an important
> document. I have two questions
> 
> Q1. How do you search inside "Python LibraryReference" ? Does it exist
> in pdf or chm form?
 ...

I use Google:

site:docs.python.org/lib foo bar

Note that Google is pretty flexible about what is a "site".  
"docs.python.org/lib", "docs.python.org", and "python.org" are all 
"sites", so you can zoom in and out as needed.

The other suggestions look good, too.

TonyN.:'[EMAIL PROTECTED]
  '  
-- 
http://mail.python.org/mailman/listinfo/python-list

os.path.expanduser('~/foo') and MSWindows "My Documents"

[Tony Nelson]

On *nix, ~/foo refers to a file in a user's home directory.  On 
MSWindows, users normally look at "My Documents" in their home 
directory.  ISTM that a file that my program would put in ~/. on Linux 
should be put in "~/My Documents/" (modulo os.path.normpath()) on 
MSWindows, where a user would expect it.  How do you guys cope with this 
issue?

TonyN.:'[EMAIL PROTECTED]
  '  
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Hot to split string literals that will across two or more lines ?

[Tony Nelson]

In article <[EMAIL PROTECTED]>,
 "Sam Pointon" <[EMAIL PROTECTED]> wrote:

> > print "a string which is very loo" \
> > + "ong."
> 
> Minor pedantry, but the plus sign is redundant. Python automatically
> concatenates string literals on the same logical line separated by only
> whitespace.
> 

While we're at it, I use bracketing instead of line continuation:

print ( "a long string, longer than this "
"and some more of the string" )

TonyN.:'[EMAIL PROTECTED]
  '  
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: os.path.expanduser('~/foo') and MSWindows "My Documents"

[Tony Nelson]

In article <[EMAIL PROTECTED]>,
 [EMAIL PROTECTED] wrote:

> Try this:
> 
> 
> from win32com.shell import shell, shellcon
> HOMEDIR = shell.SHGetSpecialFolderPath(0, shellcon.CSIDL_PERSONAL)
> myfile_location = os.path.join(HOMEDIR, myfile_name)
> 
> Define a HOMEDIR for your various platforms (use sys.platform to figure
> out what choice to make) and the rest of your code should not need to
> care where "home" is...

Sounds good, thanks.  I'll read up on it all tomorrow.

TonyN.:'[EMAIL PROTECTED]
  '  
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: os.path.expanduser('~/foo') and MSWindows "My Documents"

[Tony Nelson]

In article <[EMAIL PROTECTED]>,
 [EMAIL PROTECTED] wrote:

> python
> 
> >>> import os
> >>> help(os.path.expanduser)
> >>> import platform
> >>> platform.system()
> 'Windows'
> >>> platform.release()
> 'XP'
> >>>

platform looks good.

TonyN.:'[EMAIL PROTECTED]
  '  
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Using gettext to provide different language-version of a script

[Tony Nelson]

In article <[EMAIL PROTECTED]>,
 "Thomas W" <[EMAIL PROTECTED]> wrote:

> I'm trying to wrap my head around the docs at python.org related to the
> gettext-module, but I'm having some problem getting it to work. Is
> there any really simple, step-by-step on how to use this module
> available?
> 
> This is my script so far :
> 
> import gettext
> gettext.install('test2', '.', unicode=1)
> lang1 = gettext.translation('test2', languages=['no'])
> print  _('writing a log to file')

I do:

# assume the script is named "myscript.py":
us = os.path.splitext(os.path.basename(sys.argv[0]))[0]
usdir = os.path.dirname(sys.argv[0])

import gettext
gettext.install(us, usdir)


> in the folder where the test2.py-script lives I've created a
> folder-structure like
> 
> ./locales/NO/LC_MESSAGES/messages.mo

Mine looks like:

./en_PL/LC_MESSAGES/myscript.mo


> the messages.mo-file I've created using the scripts in the
> \Tools\i18l\-folder by running :
> 
> python pygettext.py test2.py
> 
> and renaming the generated messages.pot-file to messages.po, and
> editing it to look like :
> 
> # SOME DESCRIPTIVE TITLE.
> # Copyright (C) YEAR ORGANIZATION
> # FIRST AUTHOR <[EMAIL PROTECTED]>, YEAR.
> #
> msgid ""
> msgstr ""
> "Project-Id-Version: PACKAGE VERSION\n"
> "POT-Creation-Date: 2005-11-22 13:02+W. Europe Standard Time\n"
> "PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
> "Last-Translator: FULL NAME <[EMAIL PROTECTED]>\n"
> "Language-Team: LANGUAGE <[EMAIL PROTECTED]>\n"
> "MIME-Version: 1.0\n"
> "Content-Type: text/plain; charset=CHARSET\n"
> "Content-Transfer-Encoding: ENCODING\n"
> "Generated-By: pygettext.py 1.5\n"
 ...

You need to set the "Content-Type: charset" and 
"Content-Transfer-Encoding:".  I use:

"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"

TonyN.:'[EMAIL PROTECTED]
  '  
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Mixed types and variants

[Tony Nelson]

In article <[EMAIL PROTECTED]>,
 [EMAIL PROTECTED] wrote:

 ...
> - Maybe someone here can suggest some other variant type, or some other
> solution.

Pyrex?  Pyrex is mostly like Python with the possibility of C types.  It 
handles mixed types just like Python, and the C code it produces is sort 
of readable.

TonyN.:'[EMAIL PROTECTED]
  '  
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: ncurses' Dark Devilry

[Tony Nelson]

In article <[EMAIL PROTECTED]>,
 Jeremy Moles <[EMAIL PROTECTED]> wrote:

> I'm working on a project using ncurses w/ Python. As an aside, I
> implemented addchstr in the cursesmodule.c file in Python SVN, if anyone
> wants me to try and get that made permanent.
> 
> AT ANY RATE...
> 
> I was wondering--and this is more a general curses question rather than
> a Python one, but I know there are some old-timers here who have made
> curses obey before--is there a way to "repaint" a portion of screen
> without stealing the "cursor?" That is:
> 
> I have a focus "wheel" of sorts that allows the user to do input on
> various wigets and windows and whatnot. However, if I want to quickly
> call addstr somewhere else in the application I have to:
> 
>   1. Store the YX coords of the cursor currently
>   2. Use the cursor in the "current" action
>   3. Restore the old cursor location
> 
> I know there are ways around this as I have seen curses apps that, for
> example, have a clock that updates every second without stealing
> "focus."
> 
> I tried implementing/using addchstr (mentioned above) to no success.
> 
> Any ideas? Is this just the plain wrong place to ask this? :)

I've only tried to read the Python Library Curses docs, but I thought 
that the Window object method addstr() would do what you want.

TonyN.:'[EMAIL PROTECTED]
  '  
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: unicode speed

[Tony Nelson]

In article <[EMAIL PROTECTED]>,
 David Siroky <[EMAIL PROTECTED]> wrote:

> Hi!
> 
> I need to enlighten myself in Python unicode speed and implementation.
> 
> My platform is AMD [EMAIL PROTECTED] (x86-32), Debian, Python 2.4.
> 
> First a simple example (and time results):
> 
> x = "a"*5000
> real0m0.195s
> user0m0.144s
> sys 0m0.046s
> 
> x = u"a"*5000
> real0m2.477s
> user0m2.119s
> sys 0m0.225s
> 
> So my first question is why creation of a unicode string lasts more then 10x
> longer than non-unicode string?

Your first example uses about 50 MB.  Your second uses about 200 MB, (or 
100 MB if your Python is compiled oddly).  Check the size of Unicode 
chars by:

>>> import sys
>>> hex(sys.maxunicode)

If it says '0x10' each unichar uses 4 bytes; if it says '0x', 
each unichar uses 2 bytes.


> Another situation: speed problem with long strings
> 
> I have a simple function for removing diacritics from a string:
> 
> #!/usr/bin/python2.4
> # -*- coding: UTF-8 -*-
> 
> import unicodedata
> 
> def no_diacritics(line):
> if type(line) != unicode:
> line = unicode(line, 'utf-8')
> 
> line = unicodedata.normalize('NFKD', line)
> 
> output = ''
> for c in line:
> if not unicodedata.combining(c):
> output += c
> return output
> 
> Now the calling sequence (and time results):
> 
> for i in xrange(1):
> x = u"a"*5
> y = no_diacritics(x)
> 
> real0m17.021s
> user0m11.139s
> sys 0m5.116s
> 
> for i in xrange(5):
> x = u"a"*1
> y = no_diacritics(x)
> 
> real0m0.548s
> user0m0.502s
> sys 0m0.004s
> 
> In both cases the total amount of data is equal but when I use shorter strings
> it is much faster. Maybe it has nothing to do with Python unicode but I would
> like to know the reason.

It has to do with how strings (either kind) are implemented.  Strings 
are "immutable", so string concatination is done by making a new string 
that has the concatenated value, ans assigning it to the left-hand-side.  
Often, it is faster (but more memory intensive) to append to a list and 
then at the end do a u''.join(mylist).  See GvR's essay on optimization 
at .

Alternatively, you could use array.array from the Python Library (it's 
easy) to get something "just as good as" mutable strings.

TonyN.:'[EMAIL PROTECTED]
  '  
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: wxPython : getting started

[Tony Nelson]

In article <[EMAIL PROTECTED]>,
 David Sulc <[EMAIL PROTECTED]> wrote:

> Hi !
> 
> I've looked all over (internet, books, etc.) and I haven't found a very 
> good ressource to get started with wxPython (yes, I've been through 
> their tutorial).
> 
> What I would basically like to do for starters is to be able to define 
> the main panel being displayed. For example :
> 1. wxFrame contains a wxPanel  (call it mainPanel).
> 2. mainPanel contains another panel (childPanelA)
> 3. another panel has been defined (childPanelB) but is not displayed 
> (the user can only see childPanelA inside mainPanel)
> 4. by clicking on a menu entry (for example), the displayed panel is now 
> childPanelA (which is inside mainPanel)
> 
> So how do I do this ? I realize it's a very basic question, but it's 
> been driving me mad...
 ...

I don't know or use wxWidgets, and I've just learned GTK, but I think 
one good answer is the same as with GTK:  use a wxNotebook.  You may be 
able to hide the tabs, or you may just decide they're a good thing to 
have.

To do what you asked, see the wxWindow method Show(), which you would 
call for each or A and B etc. in response to the command.

TonyN.:'[EMAIL PROTECTED]
  '  
-- 
http://mail.python.org/mailman/listinfo/python-list

Pydoc: restrict base class doc?

[Tony Nelson]

I'd like to prevent Pydoc from adding base class documentation for some 
of my classes.  Specifically, I have a couple of classes that derive 
from GTK widgets, and dumping all that documentation in doesn't have 
much benefit.  Is there some thing I can do in my source, or to Pydoc, 
to tell it to skip some base classes?

TonyN.:'[EMAIL PROTECTED]
  '  
-- 
http://mail.python.org/mailman/listinfo/python-list

How to keep Pydoc from listing too much?

[Tony Nelson]

How can I tell Pydoc not to list information for some of the base 
classes?  For example, when a class inherits from gtk.Widget, lots of 
GTK stuff gets added that doesn't really need to be there.  Is there 
some option to Pydoc to tell it to skip some classes?  Is there 
something I can put in my source that is a hint to Pydoc?

TonyN.:'[EMAIL PROTECTED]
  '  
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: ANN: Dao Language v.0.9.6-beta is release!

[Tony Nelson]

In article <[EMAIL PROTECTED]>,
 Dave Hansen <[EMAIL PROTECTED]> wrote:

> On 2 Dec 2005 10:08:21 -0800 in comp.lang.python, [EMAIL PROTECTED]
> wrote:
> 
> >Here it is again...  Python bypassed/discounted because, of all things,
> >scoping by indentation!?!?
> >
> >This used to surprise me.  Until I hear more and more otherwise
> >reasonable programmers list this as their number one reason for
> >shunning Python.
> >
> >I gauge design defects by how much after market
> >discussion/documentation a feature generates/requires.   Scoping by
> >indentation is a whopper of a defect.
> 
> FWIW, indentation scoping one one of the features that _attracted_ me
> to Python.

Me too.  Or rather, Python code is much more readable because every 
nincompoop has to use indentation correctly whether they want to or not, 
and I like readable code.

TonyN.:'[EMAIL PROTECTED]
  '  
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: OO in Python? ^^

[Tony Nelson]

In article <[EMAIL PROTECTED]>,
 Matthias Kaeppler <[EMAIL PROTECTED]> wrote:
 ...
> obj = Base() # I want a base class reference which is polymorphic

obj now refers to an instance of Base.

> if ():
> obj =  D1()

obj now refers to an instance of D1().  The Base instance is 
unreferenced.

> else:
> obj = D2()

obj now refers to an instance of D2().  The Base instance is 
unreferenced.

Note that there is no code path that results in obj still referring to 
an instance of Base.  Unless making a Base had side effects, there is no 
use in the first line.


> I could as well leave the whole inheritance stuff out and the program 
> would still work (?).

That program might.


> Please give me hope that Python is still worth learning :-/

Python has inheritance and polymorphism, implemented via dictionaries.  
Python's various types of namespace are implemented with dictionaries.

Type this in to the Python interpreter:

class Base:
def foo(self):
print 'in Base.foo'

class D1(Base):
def foo(self):
print 'in D1.foo'
Base.foo(self)

class D2(Base):
def foo(self):
print 'in D2.foo'
Base.foo(self)

def makeObj():
return needD1 and D1() or D2()

needD1 = True
makeObj().foo()

needD1 = False
makeObj().foo()

TonyN.:'[EMAIL PROTECTED]
  '  
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Wingide is a beautiful application

[Tony Nelson]

In article <[EMAIL PROTECTED]>,
 "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> wrote:
 ...
> I get the feeling that a ot of people working with heavy IDEs don't
> realize how capable vim/emacs are, so I'll give a brief rundown of what
> my Vim environment does for me.  (I do Python web development)--if you
> don't like the Vi keybindings, the Cream package is Vim that behaves
> like a regular modeless editor but with all of vim's power (and a nice
> embedded Python interpreter for writing extensions):
> 
> 1. Python syntax checking: as I'm typing along, if I input a syntax
> error then the line is immediately highlighted in red.  Useful for
> catching brainos like:
> if a=1:
> (which will highlight in red when I hit enter, point out that I need ==
> instead of =).

What do you use to do this?  Cream doesn't seem to do this oob.


> 2. Normal tag-jump stuff: Ctrl-click on a function/method call (or
> class or whatever) will jump to the function/method/class definition
> (Ctrl-T works as well if you don't like clicking).  It keeps a stack of
> visited files so you can drill down through your call stack and then
> pop back up to where you came from.

Do you set up ctags for this?  I get error messages "E433: No tags file" 
and "E426: tag not found: xxx" when I Ctrl-click on a method call.


> 3. Python class browsing stuff: A Class menu shows the parent and child
> classes of the one you're currently in, and all the methods of the
> current class; selecting any of the above jumps to the appropriate file
> and line.

Is this the Tag List?


> 4. Interactive documentation stuff: When I type an open-paren, it looks
> to see what the prior keyword is and displays help for it in the status
> line (preferring Python documentation, then docstrings, then comments
> before the function/method/class definition).  Even if there's no
> help/comments, it'll show the arguments that the function takes.  So
> if, say, I type:
> 
> cmp(
> 
> then the status line displays:
> 
> cmp(x, y) Compare the two objects X and Y and return an integer
> according to ...
> 
> If I hit F1 it'll show the full help text.  Often the arguments are
> enough, and I find the status-line display a lot less intrusive than
> many on-the-fly help systems I've seen.

This stuff doesn't happen either.  How is it set up?


> 5. A client menu selects which client I want to work in (so, say, I get
> a bug report for Client A, I select them from the menu).  The Class
> menu and other functions respect this (if I'm in the generic Company
> class, the Class menu will list Client A's Company subclass before the
> subclasses of other companies; if I jump to the Company definition,
> it'll go to Company A's client-specific version).  It also restarts
> development httpd servers on the current machine running with conf
> files appropriate to that client.
 ...

Where is this "client menu"?  How is it set up?

TonyN.:'[EMAIL PROTECTED]
  '  
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Wingide is a beautiful application

[Tony Nelson]

In article <[EMAIL PROTECTED]>,
 "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> wrote:

> Tony Nelson wrote:
> > > 1. Python syntax checking: as I'm typing along, if I input a syntax
> > > error then the line is immediately highlighted in red.
> >
> > What do you use to do this?  Cream doesn't seem to do this oob.
> 
> Nope.  I'll try to package up my vimrc and get it uploaded somewhere
> next week (busy with holiday stuff).

OK, thanks.

> The meat of it is:
> 
> import vim
> def cur_x():
> return vim.current.window.cursor[1]
> def cur_y():
> return vim.current.window.cursor[0]
> def find_current_block():
> block = [vim.current.line]
> current = cur_y()-1
> while len(block[0])==0 or block[0][0] in " \t#":
> current = current - 1
> if current < 0:
> break
> block = [vim.current.buffer[current] ]+ block
> return block
> def check_current_block():
> import code
> vim.command("syn clear Error")
> block = find_current_block()
> length = len(block)
> try:
> code.compile_command("\n".join(block))
> print ""
> return 0
> except:
> (type, value, tb) = sys.exc_info()
> line_no = cur_y()-1+value.lineno-length
> badline = vim.current.buffer[line_no]
> badline = badline.replace('"', '\\"')
> print "Error at line %d: " %(line_no+1), badline
> return 1

OK, I can tell that this is Python code, not VI script stuff.  I'll need 
to see how your vimrc sets this up.  It looks like learning to use VI 
(even with Cream) would take a few weeks of hard work.


> > > 2. Normal tag-jump stuff: Ctrl-click on a function/method call (or
> > > class or whatever) will jump to the function/method/class definition
> > > (Ctrl-T works as well if you don't like clicking).  It keeps a stack of
> > > visited files so you can drill down through your call stack and then
> > > pop back up to where you came from.
> >
> > Do you set up ctags for this?  I get error messages "E433: No tags file"
> > and "E426: tag not found: xxx" when I Ctrl-click on a method call.
> 
> Umm, either click the "build tags in current directory" button on the
> toolbar, or "find . -name '*.py' | ctags -L -" in the top directory of
> your python file.

OK.


> > > 3. Python class browsing stuff: A Class menu shows the parent and child
> > > classes of the one you're currently in, and all the methods of the
> > > current class; selecting any of the above jumps to the appropriate file
> > > and line.
> >
> > Is this the Tag List?
> 
> No, this is more complex, I'll post it w/ the rest of my stuff.
 ...

Thanks.

TonyN.:'[EMAIL PROTECTED]
  '  <http://www.georgeanelson.com/>
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Wingide is a beautiful application

[Tony Nelson]

In article <[EMAIL PROTECTED]>,
 "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> wrote:

> Tony Nelson wrote:
> > OK, I can tell that this is Python code, not VI script stuff.  I'll need
> > to see how your vimrc sets this up.
> 
> vim has a Python interpreter embedded in it (assuming it's a reasonably
> complete build--it's possible to leave the interpreter, or even parts
> of the vim scripting stuff, out).

OK, mine has Python in it.


> I just put .py files in my .vim
> directory, import them, and then use ":py myfile.doStuff()" or map keys
> to such commands.

So, you bind check_current_block() to the Enter key?


> The "vim" Python module (included in vim, just "import vim" from your
> Python scripts) lets you run vim commands, access
> windows/buffers/variables/etc.
 ...

OK.  I see Python in the help.

TonyN.:'[EMAIL PROTECTED]
  '  <http://www.georgeanelson.com/>
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Wingide is a beautiful application

[Tony Nelson]

In article <[EMAIL PROTECTED]>,
 "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> wrote:

> Tony Nelson wrote:
> > So, you bind check_current_block() to the Enter key?
> 
> Yeah.  The binding's not quite just "check_current_block()"
> because you need a bit of magic to keep autoindent working.  I'll post
> it with my conf files next week, essentially use = with an empty
> vim function wrapper around the python function (instead of ) and
> input a character before leaving insert mode that's deleted when you
> return (to preserve indent).
> 
> My goal is to make my conf files into a decent drop-in so you just put
> them in your .vim directory and go, and post them next week.

OK, thank you.

TonyN.:'[EMAIL PROTECTED]
  '  <http://www.georgeanelson.com/>
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Wingide is a beautiful application

[Tony Nelson]

In article <[EMAIL PROTECTED]>,
 "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> wrote:

> Tony Nelson wrote:
> >  "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> wrote:
> >
> > > My goal is to make my conf files into a decent drop-in so you just put
> > > them in your .vim directory and go, and post them next week.
> >
> > OK, thank you.
> >
> 
> FYI, I am still working on this but some changes in vim 7 are requiring
> more work than I expected.

I suspected that it might be taking longer than you thought.  No hurry 
from my end.


> On the bright side, there are some nice
> features that will allow for cool stuff like debugger info (after
> running in a debugger, you'll be able to mouse over variables and get
> tooltip popups with their values, etc) when it's done.  I'm talking
> with Bram Moolenaar (vim author) on the vim-dev mailing list to try to
> get things squared away.

Thank you for your efforts.

TonyN.:'[EMAIL PROTECTED]
  '  <http://www.georgeanelson.com/>
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: What is the best way to print the usage string ?

[Tony Nelson]

In article <[EMAIL PROTECTED]>,
 "Simon Forman" <[EMAIL PROTECTED]> wrote:
 ...
> Python also concatenates adjacent strings, but the "real" newlines
> between your strings will need to be escaped (otherwise, because the
> newlines are statement separators, you will have one print statement
> followed by string literals with the wrong indentation.)
> 
> print "Usage: blah blah blah\n" \
>   "Some more lines in the usage text\n" \
>   "Some more lines here too."
> 
> (Note that the final string literal newline is not needed since print
> will add one of it's own.)

One can also use parentheses:

print ( "Usage: blah blah blah\n"
"Some more lines in the usage text\n"
"Some more lines here too." )

The newlines in the string are still needed, but the ones escaping the 
EOLs are not.

TonyN.:'[EMAIL PROTECTED]
  '  
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: E' possibile integrare ironpython con visual studio 2005?

[Tony Nelson]

In article <[EMAIL PROTECTED]>,
 [EMAIL PROTECTED] (Alex Martelli) wrote:

> LaGuna <[EMAIL PROTECTED]> wrote:
> 
> > Se si come?
> > 
> > Ciao by Enzo
> 
> Questo newsgroup preferisce l'inglese -- per favore, chiedi su
> it.comp.lang.python invece che qui.
> 
> This newsgroup prefers English -- please ask on it.comp.lang.python
> rather than here.

Still, the question is obvious enough.

Microsoft's "Visual Studio 2005 SDK" now contains IronPython 
integration.  I haven't used it, as I have only the free Visual C# 2005 
Express Edition.

TonyN.:'[EMAIL PROTECTED]
  '  
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Py3K idea: why not drop the colon?

[Tony Nelson]

In article <[EMAIL PROTECTED]>,
 Bjoern Schliessmann <[EMAIL PROTECTED]> 
 wrote:

> Michael Hobbs wrote:
> 
> > That is, assume that the expression ends at the colon, not at the
> > newline. That would make this type of statement possible:
> > if color == red or
> >   color == blue or
> >   color == green:
> > return 'primary'
> > Right now, such a statement would have to be spelled thus:
> > if color == red or \
> >   color == blue or \
> >   color == green:
> > return 'primary'
> > or
> > if (color == red or
> >   color == blue or
> >   color == green):
> > return 'primary'
> 
> What about
> 
> if color == red or blue or green:
> return 'primary'

What about

if color in [red, blue, green]:
return 'primary'

TonyN.:'[EMAIL PROTECTED]
  '  
-- 
http://mail.python.org/mailman/listinfo/python-list

Trace KeyboardInterrupt exception?

[Tony Nelson]

I'm trying to find out what is eating some KeyboardInterrupt exceptions 
in a fairly large program (yum).  My KeyboardInterrupt handler is called 
for some Ctl-C presses, but for others nothing seems to happen.  
Grepping the source (what of it I've found, looking at import 
statements) doesn't turn up anything likely.

My thinking is that either some "except:" clause is eating them, or some 
place I haven't looked is eating them, or possibly C code is eating 
them.  For the first two, at least, I'd like to use a debugger to trace 
KeyboardInterrupt exceptions, make sure that they're happening, and see 
what is handling them.  I don't see a way to trace or break on a 
specific exception type in Idle.  Can someone give me a hint on how to 
do this?  I'm willing to consider other debuggers, including gdb (DDD).

TonyN.:'[EMAIL PROTECTED]
  '  
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Trace KeyboardInterrupt exception?

[Tony Nelson]

In article <[EMAIL PROTECTED]>,
 [EMAIL PROTECTED] wrote:

> Tony Nelson wrote:
> > I'm trying to find out what is eating some KeyboardInterrupt exceptions
> > in a fairly large program (yum).  My KeyboardInterrupt handler is called
> > for some Ctl-C presses, but for others nothing seems to happen.
> 
> >  ... I'd like to use a debugger to trace
> > KeyboardInterrupt exceptions, make sure that they're happening, and see
> > what is handling them.
> 
> I don't know how to do that in Idle.  You can replace the default
> Ctrl-C interrupt handler with your own and use that to inspect the
> current stack.

Thanky you, that helps.  Interestingly, some Ctl-Cs don't get caught.  
Presumably they're happening in a subprocess.

Now to see if I can get into that situation again where Ctl-C is 
ignored.  I need to know what's eating the exceptions.  I don't think 
it's a subprocess in the case I'm concerned with.  I don't think yum is 
threaded, but apparantly importing the tread module anywhere should keep 
KeyboardInterrupt on the main thread anyway (?).

It would be nice if I could inspect the stack and find the exception 
handlers.  I'm using trace handlers, but their output seems somewhat 
spotty and inconsistent, or maybe just confusing.

TonyN.:'[EMAIL PROTECTED]
  '  <http://www.georgeanelson.com/>
-- 
http://mail.python.org/mailman/listinfo/python-list

socket and Ctl-C workaround?

[Tony Nelson]

I've been trying to figure out why Ctl-C sometimes doesn't interrupt 
yum.  It appears to be unresolved Python bug 926423, unresolved proposed 
patch 1102879, don't know if anything ever came of it.  Note that I 
cannot ask all yum users to apply the patch.  I'm not sure I should be 
getting rid of the timeout, either.

When a Ctl-C is lost, sometimes it is silently eaten, and other times it 
becomes socket.error(11,...) (EWOULDBLOCK).  As (AFAICT) the socket in 
question is a blocking socket, I could use that nonsense exception to 
detect at least some of the lost Ctl-Cs (eww).  The socket does have a 
timeout.

Note that I am using FC5 Linux, Python 2.4.2.

TonyN.:'[EMAIL PROTECTED]
  '  
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Trace KeyboardInterrupt exception?

[Tony Nelson]

In article <[EMAIL PROTECTED]>,
 "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> wrote:

> if you want to interrupt the code to find out where it is,
> you can instead connect to it in gdb and get the python traceback of
> each thread.
> if you're interested I'll post the necesary gdb-macro for that (didn't
> put it on the net yet)

I think I've found the problem, using Python's Bugzilla.  This appears 
to be unresolved Python bug 926423, unresolved proposed patch 1102879, 
don't know if anything ever came of it.  See other thread.

TonyN.:'[EMAIL PROTECTED]
  '  
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: super(...).__init__() vs Base.__init__(self)

[Tony Nelson]

In article <[EMAIL PROTECTED]>,
 Jan Niklas Fingerle <[EMAIL PROTECTED]> wrote:

> ...Super is a good tool to use, when dealing with
> diamond shape inheritance. In any other case I would use the direct
> calls to the base classes. In fact, i've yet to find a non-textbook-case
> where I really need diamond shape inheritance. ...

As long as you don't use multiple inheritance with new-style classes, 
you'll be fine.

TonyN.:'[EMAIL PROTECTED]
  '  
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP 353: Using ssize_t as the index type

[Tony Nelson]

In article <[EMAIL PROTECTED]>,
 "Martin v. Löwis" <[EMAIL PROTECTED]> wrote:
 ...
> Discussion
> ==
> 
> Why not size_t
> --
> 
> An initial attempt to implement this feature tried to use
> size_t. It quickly turned out that this cannot work: Python
> uses negative indices in many places (to indicate counting
> from the end). Even in places where size_t would be usable,
> to many reformulations of code where necessary, e.g. in ...
  ^-
|
Minor typo:  "too"

TonyN.:'[EMAIL PROTECTED]
  '  
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: time.sleep(1) sometimes runs for 200 seconds under windows

[Tony Nelson]

In article <[EMAIL PROTECTED]>,
 Claudio Grondi <[EMAIL PROTECTED]> wrote:

> Claudio Grondi wrote:
> > Claudio Grondi wrote:
> > 
> >> Paul Probert wrote:
> >>
> >>> Peter Hansen wrote:
> >>>
>  Are you saying that you believe the time.sleep(1) call is actually 
>  blocking for 200 seconds?
> >>
> >> With such rare occurrence it is very hard to tell what is going on. 
> >> Usually I put such strange things on a list of curiosities I don't 
> >> want to know the reason of, because it is in my eyes not worth the 
> >> effort. Maybe it is even a common problem not yet detected by me, 
> >> because I have never run this kind of tests for such a long time.
> >> Starting today, I can tell you statistically not earlier than in one 
> >> week, if I have the same problem on my machines (currently I am 
> >> running only one or two at the same time).
> > 
> > 
> > Here the intermediate results on my Windows XP machine connected to the 
> > Internet via very fast digital phone line connection (network 
> > card/digital-converter box/phone-line):
> > 
> > dt= 1.125  time= 2006_02_24_11h_36m_15s
> > dt= 9.20200014114  time= 2006_02_24_12h_46m_49s
> > dt= 1.1876376  time= 2006_02_24_14h_43m_32s
> > 
> > The code used:
> > """
> > import time
> > while True:
> >   oldtime=time.time()
> >   time.sleep(1.0)
> >   newtime=time.time()
> >   dt=newtime-oldtime
> >   if dt > 1.1:
> > print 'dt=',dt,' time=',time.strftime('%Y_%m_%d_%Hh_%Mm_%Ss')
> > """
> > running in a command line console parallel to usual daily business on 
> > the computer.
> > 
> > The yesterday night run (5 hours) gave max. 1.125 sec., so I am 
> > surprized to see the 9 seconds already after only two hours today.
> > 
> > Claudio
> 
> The 9.2 seconds difference was also because of time synchronization 
> Windows XP does using time.windows.com server - it reported to be done 
> last time 2006-02-24 at 12:46 o'clock i.e. exactly at the same time 
> where the 9.2 delay occurred.

ISTM that using time-of-day is the wrong thing for measuring elapsed 
time, but I don't find anything in the time module that would be better.  
/proc/uptime looks like it would work on linux, in a roundabout way:

>>> up = open('/proc/uptime')
>>> up.seek(0) ; up.read()

will print uptime and idle time, but on linux only.  At least it seems 
to be fast enough for the purpose at hand (about 18 microseconds on my 
old box, according to timeit).

Is there a better timer available in python?  I see that timeit module 
uses time.time() (or time.clock() on MSWindows), so I am not hopeful.

TonyN.:'[EMAIL PROTECTED]
  '  
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Microsoft's Dynamic Languages Runtime (DLR)

[Tony Nelson]

In article <[EMAIL PROTECTED]>,
 Kaz Kylheku <[EMAIL PROTECTED]> wrote:

> On May 2, 5:19 pm, sturlamolden <[EMAIL PROTECTED]> wrote:
> > On May 3, 2:15 am, Kaz Kylheku <[EMAIL PROTECTED]> wrote:
> >
> > > Kindly refrain from creating any more off-topic, cross-posted threads.
> > > Thanks.
> >
> > The only off-topic posting in this thread is your own (and now this
> > one).
> 
> You are making a very clumsy entrance into these newsgroups.
 ...

Go away.

TonyN.:'[EMAIL PROTECTED]
  '  
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Possible to set cpython heap size?

[Tony Nelson]

In article <[EMAIL PROTECTED]>,
 "Andy Watson" <[EMAIL PROTECTED]> wrote:

 ...
> If I could have a heap that is larger and does not need to be
> dynamically extended, then the Python GC could work more efficiently.
 ...

GC!  If you're allocating lots of objects and holding on to them, GC 
will run frequently, but won't find anything to free.  Maybe you want to 
turn off GC, at least some of the time?  See the GC module, esp. 
set_threshold().

Note that the cyclic GC is only really a sort of safety net for 
reference loops, as normally objects are free'd when their last 
reference is lost.

TonyN.:'[EMAIL PROTECTED]
  '  
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: newbie needs help building Python 2.5 for Fedora Core 6

[Tony Nelson]

In article <[EMAIL PROTECTED]>,
 "bobmon" <[EMAIL PROTECTED]> wrote:

> Hello, and please be gentle...
> 
> I'm trying to build Python 2.5 on my Fedora Core 6 installation.  I
> think I've resolved most of my problems, but "make test" reports an
> error for test_socket.py, shown below.
> 
> I suppose my FC6 installation is missing something, but I have no idea
> what.  Any ideas, directions, pointers would be most appreciated.
> 
> 
> 
> ==
>  ERROR: testSockName (__main__.GeneralModuleTests)
>  
> --
>  Traceback (most recent call last):
>File "./Lib/test/test_socket.py", line 456, in testSockName
>  my_ip_addr = socket.gethostbyname(socket.gethostname())
>  gaierror: (-2, 'Name or service not known')
> 
>  
> --
>  Ran 66 tests in 35.478s
> 
>  FAILED (errors=1)
>  Traceback (most recent call last):
>File "./Lib/test/test_socket.py", line 962, in 
>  test_main()
>File "./Lib/test/test_socket.py", line 958, in test_main
>  test_support.run_unittest(*tests)
>File "/home/Installer/Python/Python-2.5/Lib/test/test_support.py",
> line 441, in run_uni
>  ttest
>  run_suite(suite, testclass)
>File "/home/Installer/Python/Python-2.5/Lib/test/test_support.py",
> line 426, in run_sui
>  te
>  raise TestFailed(err)
>  test.test_support.TestFailed: Traceback (most recent call last):
>File "./Lib/test/test_socket.py", line 456, in testSockName
>  my_ip_addr = socket.gethostbyname(socket.gethostname())
>  gaierror: (-2, 'Name or service not known')

OK, so this fails:

my_ip_addr = socket.gethostbyname(socket.gethostname())

Try it from the python command line.  This is what happens when I try it 
on FC6 w/ Python 2.5 (retyped, tho):

>>> import socket
>>> socket.gethostname()
'localhost.localdomain'
>>> socket.gethostbyname(_)
'127.0.0.1'
>>>

TonyN.:'[EMAIL PROTECTED]
  '  
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: python tutorial on a single html page?

[Tony Nelson]

In article <[EMAIL PROTECTED]>,
 BartlebyScrivener <[EMAIL PROTECTED]> wrote:

> Is the main Python tutorial posted on single searchable page
> somewhere?  As opposed to browsing the index and clicking NEXT etc.

For completeness (though a bit late), I'll mention that Google can 
search a group of web pages with:

site:docs.python.org/tut/ search terms

TonyN.:'[EMAIL PROTECTED]
  '  
-- 
http://mail.python.org/mailman/listinfo/python-list