Re: Manipulate Large Binary Files

[Paul Rubin]

Derek Martin <[EMAIL PROTECTED]> writes:
> > Both are clocking in at the same time (1m 5sec for 2.6Gb), are there
> > any ways I can optimize either solution?  

Getting 40+ MB/sec through a file system is pretty impressive. 
Sounds like a RAID?

> That said, due to normal I/O generally involving double-buffering, you
> might be able to speed things up noticably by using Memory-Mapped I/O
> (MMIO).  It depends on whether or not the implementation of the Python
> things you're using already use MMIO under the hood, and whether or
> not MMIO happens to be broken in your OS. :)

Python has the mmap module and I use it sometimes, but it's not
necessarily the right thing for something like this.  Each page you
try to read from results in own delay while the resulting page fault
is serviced, so any overlapped i/o you get comes from the OS being
nice enough to do some predictive readahead for you on sequential
access if it does that.  By coincidence there are a couple other
threads mentioning AIO which is a somewhat more powerful mechanism.

-- 
http://mail.python.org/mailman/listinfo/python-list

Python for low-level Windows programming

[haxier]

Hi all

I've some experience with Python in desktop apps, but now I'm looking
to code a tool like Kee-Pass[1] which must have access to some low-
level primitives in a windows environment: hooks when windows are
displayed, automatic form fill, and so on in a variety of scenarios
(desktop apps, web-based apps, citrix...)

Is this possible without too much pain? I know I can code it with C#
or C++ but tha'ts a road to avoid, if possible.

Thanks

[1] http://sourceforge.net/projects/keepass
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python for low-level Windows programming

[Tim Golden]

haxier wrote:
> I've some experience with Python in desktop apps, but now I'm looking
> to code a tool like Kee-Pass[1] which must have access to some low-
> level primitives in a windows environment: hooks when windows are
> displayed, automatic form fill, and so on in a variety of scenarios
> (desktop apps, web-based apps, citrix...)
> 
> Is this possible without too much pain? I know I can code it with C#
> or C++ but tha'ts a road to avoid, if possible.

Well of course I don't know exactly what you'd need, but the
answer's certainly Yes :) In short, even native Python comes
with the ctypes module which will let you call -- with only
a little working out the structures -- into any Windows DLL
or COM interface (the latter via the comtypes extension).

But in any case the pywin32 packages and various other
open source projects have already done a lot of the work
for you. Plus it's easy enough to write your own extension
which does the dirty work in C and exposes it to Python as
functions, objects or whatever suits your purposes.

Good luck!

TJG
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: function call - default value & collecting arguments

[Primoz Skale]

<[EMAIL PROTECTED]> wrote in message 
news:[EMAIL PROTECTED]
> On 2 avr, 22:32, "Primoz Skale" <[EMAIL PROTECTED]> wrote:
>> >> I also understand (fairly) how to collect arguments. For example, 
>> >> let's
>> >> define another function:
>>
>> >> def f(*a):
>> >>print a
>>
>> > This means that f takes any number of optional positional arguments.
>> > If nothing is passed, within f, 'a' will be an empty tuple. Note that
>> > this is *not* the usual way to define a function taking multiple
>> > (mandatory) arguments.
>>
>> M. Lutz in "Learning Python" had defined it this way. What is the *usual*
>> way in this case?
>
> You mean : "what's the usual way to define a function taking multiple
> *mandatory* arguments" ? I'd think it's explained in your book ???
>
> def f(a, b, c):
>  print a, b, c
>
> But this is such a cs101 point that we're surely misunderstanding each
> other here.
>

Yes, this was misunderstanding. I thought you meant defining with the *a, 
and not with a,b,c, etcThanks anyway :)

>>
>> > or (slightly more involved, and certainly overkill):
>>
>> > def with_default_args(default):
>> >def decorator(func):
>> >def wrapper(*args):
>> >if not args:
>> >args = default
>> >return func(*args)
>> >return wrapper
>> >return decorator
>>
>> > @with_default_args((0,))
>> > def f(*a):
>> >print a[0]
>>
>> Now, this is interesting. Thanks! :)
>
> Dont take this as a "recommanded" solution to your problem - it was
> (mostly) to be exhaustive (and a bit on the "showing off" side too to
> be honest).
>

No of course not - but it is interesting...

P. 


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Prototype OO

[sam]

[EMAIL PROTECTED] napisał(a):


> So, while I often use Python's lambdas, the imposed limitations is ok
> to me since I wouldn't use it for anything more complex. 

> Also - as a side note - while the syntax is a bit different, the
> resulting object is an ordinary function.

And people start asking why this is that or the other way in Python, and you 
can't give a good answer to newcomers, especially if Python was chosen as a 
first learning language. You can't tell "because Python's parsing mechanism 
don't allow statements in expressions...".







>> But somebody may prefix his names with class names and cause nameconflict,
> 
> Here the problem is more philosophical than anything else. 

Almost the whole discussion is philosofical, but clear philosophy is a good 
thing.





> It's enough. FWIW, I'm not sure I had a use-case for this feature more
> than a couple time in 7+ years.

> Simple, indeed. But why "not perfect" ? What else would you want ?

I would expect you to told me at the beginning that this is enough to solve the 
problem and that somebody can still cause name conflict, but Python was built 
for wise people.




-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Prototype OO

[sam]

Gabriel Genellina napisał(a):

>> Yes. Funciton is always a piece of code (program) that does something. 
>> There is
>> no need for different syntax.
> 
> Guido has regretted lambda for a long time; it was scheduled for 
> deletion on Python 3000 [2] but finally will stay [3].

Thanks for that info and links.



> Class methods and instance methods are not just standard functions; 
> instance methods were plain functions before 2.2 and the Class object 
> was in charge of doing the "self magic". Now the descriptor protocol 
> provides far more possibilities.

Actually I don't know what is "descriptor protocol", so maybe I should have 
finished discussing. I will aslo search for "self magic" -- some pieces of old 
code, or something.



> I didn't say that (note that you trimmed most attribution lines) but I 
> like to have "short anonymous functions" altough the syntax might be 
> different. Perhaps in Python 4000.

And I say "syntax should be the same". These are only opinions, so forgive me 
for wasting your time.



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Prototype OO

[Diez B. Roggisch]

> And I say "syntax should be the same". These are only opinions, so 
> forgive me for wasting your time.

You mean like in JS?

function foo(args) {}

foo = function(args) {}

Somehow the JS-designers also made a compromise to allow to create 
unnamed and named functions. Could it be that it make *sense* to 
sacrifice simplicity and orthogonality here, so that different usage 
scenarios get to be written concisely?

Start coding in python. And enjoy it. Is it perfect? Heck no! But it 
sure is fun enough to deal with the occasional wart.

Diez
-- 
http://mail.python.org/mailman/listinfo/python-list

unicode in exception traceback

[WaterWalk]

Hello. I just found on Windows when an exception is raised and
traceback info is printed on STDERR, all the characters printed are
just plain ASCII. Take the unicode character u'\u4e00' for example. If
I write:

print u'\u4e00'

If the system locale is "PRC China", then this statement will print
this character as a single Chinese character.

But if i write: assert u'\u4e00' == 1

An AssertionError will be raised and traceback info will be put to
STDERR, while this time, u'\u4e00' will simply be printed just as
u'\u4e00', several ASCII characters instead of one single Chinese
character. I use the coding directive commen(# -*- coding: utf-8 -*-)t
on the first line of Python source file and also save it in utf-8
format, but the problem remains.

What's worse, if i directly write Chinese characters in a unicode
string, when the traceback info is printed, they'll appear in a non-
readable way, that is, they show themselves as something else. It's
like printing something DBCS characters when the locale is incorrect.

I think this problem isn't unique. When using some other East-Asia
characters, the same problem may recur.

Is there any workaround to it?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python in High School

[Bruno Desthuilliers]

Jan Claeys a écrit :
(snip)
> I learned about pointers while learning Pascal (and later embedded 
> assembler) using Borland's tools.
> 
> Later I learned C (and even later C++), and I've always been wondering why 
> those languages were making simple things so complicated...
> 

Similar pattern here : I had difficulties grasping pointers in C, then 
learned them in Pascal and got enlightned. Then I was able to use them 
correctly in C.


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Prototype OO

[Bruno Desthuilliers]

sam a écrit :
> [EMAIL PROTECTED] napisał(a):
> 
> 
>> So, while I often use Python's lambdas, the imposed limitations is ok
>> to me since I wouldn't use it for anything more complex. 
> 
>> Also - as a side note - while the syntax is a bit different, the
>> resulting object is an ordinary function.
> 
> And people start asking why this is that or the other way in Python, and 
> you can't give a good answer to newcomers, especially if Python was 
> chosen as a first learning language. You can't tell "because Python's 
> parsing mechanism don't allow statements in expressions...".


If the people asking is able to understand this explanation, I'll 
happily give it. Else, I'd answer "for reasons that are perhaps too 
complex to explain to you right now, but I'll try if you insist" !-)

>>> But somebody may prefix his names with class names and cause 
>>> nameconflict,
>>
>> Here the problem is more philosophical than anything else. 
> 
> Almost the whole discussion is philosofical, but clear philosophy is a 
> good thing.
>   
> 
> 
>> It's enough. FWIW, I'm not sure I had a use-case for this feature more
>> than a couple time in 7+ years.
> 
>> Simple, indeed. But why "not perfect" ? What else would you want ?
> 
> I would expect you to told me at the beginning

which "beginning" ?

> that this is enough to 
> solve the problem

what did I do ?

> and that somebody can still cause name conflict, 

Why should I tell you something that's so obvious you observed it by 
yourself ?

> but 
> Python was built for wise people.

for normally intelligent people. I don't play russian roulette, and 
don't claim being any "wiser" for this.

Ok, I'm going to be a bit harsh, but this time I'll assume it.

Sam, you started this thread by asking about prototype vs class based 
pros and cons - with a (somewhat cargo-cult IMHO) a priori in favor of 
prototype. Then you started arguing about limitations and complexity 
induced by the class-based approach, without being able to back any of 
your claims wrt/ Python's object model. Then you started arguing about 
minor syntactic points that, whether you like them or not (and
you're of course entitled to not like them) have nothing to do with 
Python's object model nor prototypes vs classes. Now you're complaining 
that "we didn't tell you x and z from the start" - like we were supposed 
to do your education. Are we supposed to teach you reading and writing 
too ?  What's your problem anyway ? If you're a trolling - which I start 
to suspect -, then your not too bad at it. Else, it may be time to grow up.
-- 
http://mail.python.org/mailman/listinfo/python-list

Examples using msilib to build windows installers

[imageguy]

I have been using InnoSetup to distribute my wxpython app and ir works
great, howver, I would like to offer a *.msi installer to customers as
an option and this isn't available using Innosetup.

It would appear to me that the msilib library included with standard
python 2.5 would allow be to do this. I found the source code that
builds the python distrubition installer packages, but I was wondering
if there were other examples that I can learn from.

TIA.
-- 
http://mail.python.org/mailman/listinfo/python-list

State of ctypes Support on HP-UX?

[Phil Thompson]

Could somebody confirm how well ctypes is supported on HP-UX (for both PA-RISC 
and Itanium) for both Python v2.4 and v2.5?

I don't have access to an HP system and Google doesn't come up with a 
definitive answer (which may just mean it works fine, but prior experience 
with HP means I'd like more specific assurances).

Thanks,
Phil
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: who said python can't be obsfucated!?

[Bruno Desthuilliers]

[EMAIL PROTECTED] a écrit :
> def s(c):return[]if c==[]else s([_ for _ in c[1:]if _ +s([_ for _ in c[1:]if _>=c[0]])
> 
> Anyone else got some wonders...?

Nothing as bad, but:

sig=lambda m:'@'.join(['.'.join([w[::-1] for w in p.split('.')]) for p 
in m.split('@')])

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: What motivates all unpaid volunteers at Pycon?

[Bruno Desthuilliers]

[EMAIL PROTECTED] a écrit :
> On Apr 1, 11:45 am, Ed Leafe <[EMAIL PROTECTED]> wrote:
> Assuming that people get nothing back by participating in a
>> community, yes, it would be curious. My experience, though, is that I
>> get a lot more out of it than I could ever contribute. IOW, it's a
>> great example of synergy.
> 
> What do the people get back who did all the hard work at registration
> desk and
> preparing conference attendee bags? ...who did all hotel preparations?

I can't speak for them, but I'd think they feel like they *already* got 
at least a couple things 'back': a language they like, and a helpful 
community.

Just my 2 cents...
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Prototype OO

[olive]

On 3 avr, 10:32, sam <[EMAIL PROTECTED]> wrote:
> Bruno Desthuilliers napisa³(a):
>
> > Ok, I'm going to be a bit harsh, but this time I'll assume it.
> > Sam, you started this thread by asking about prototype vs class based
> > minor syntactic points that, whether you like them or not (and
>
> I think I will get back to this discussion after learning "descriptor 
> protocol"
> and maybe I will not talk about syntax then, and maybe it won't get off topic.

may I recommend this http://www.cafepy.com/article/ (the first 2)?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Prototype OO

[Bruno Desthuilliers]

sam a écrit :
> Bruno Desthuilliers napisał(a):
> 
>> Ok, I'm going to be a bit harsh, but this time I'll assume it.
> 
>> Sam, you started this thread by asking about prototype vs class based 
> 
>> minor syntactic points that, whether you like them or not (and
> 
> I think I will get back to this discussion after learning "descriptor 
> protocol" 

Add lookup (name resolution) rules, metaclasses, 'magic' methods etc to 
the list then, so you'll get a general view of Python's object model.

> and maybe I will not talk about syntax then, and maybe it 
> won't get off topic.
> 
> As you can see I'm novice in Python, but I can show python-experts some 
> newbie opinions. If somebody is an expert then he can't take a fresh 
> look at his many years work.

This is indeed true. But be assured their are quite a few newbies 
(whether  nwebie to Python or newbie to programming) posting here to 
comment on what they perceive as warts - sometimes making good points, 
sometimes just not knowing enough about the language's inners and/or 
philosophy to understand why what they think is a wart is considered a 
feature by more "advanced" users. As some other contributor (hi Diez !) 
to this thread already mentionned, "Is [Python] perfect? Heck no! But it 
sure is fun enough to deal with the occasional wart."

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: splitting an XML file on the basis on basis of XML tags

[Chris]

On Apr 3, 8:51 am, Steve Holden <[EMAIL PROTECTED]> wrote:
> bijeshn wrote:
> > On Apr 2, 5:37 pm, Chris <[EMAIL PROTECTED]> wrote:
> >> [EMAIL PROTECTED] wrote:
> >>> Hi all,
> >>>          i have an XML file with the following structure::
> >>> 
> >>> -|
> >>>      |
> >>>      |
> >>> .           |
> >>> .           |         > constitutes one record.
> >>> .           |
> >>> .           |
> >>> .           |
> >>>     |
> >>>     |
> >>> |
> >>> 
> >>> .
> >>> .
> >>> .    ---|
> >>> .                           |
> >>> .                           |
> >>> .                           |--> there are n
> >>> records in between
> >>> .                           |
> >>> .                           |
> >>> .                           |
> >>> .   |
> >>> .
> >>> .
> >>> 
> >>> -|
> >>>      |
> >>>      |
> >>> .           |
> >>> .           |         > constitutes one record.
> >>> .           |
> >>> .           |
> >>> .           |
> >>>     |
> >>>     |
> >>> |
> >>> 
> >>>        Here  is the main root tag of the XML, and ...
> >>> constitutes one record. What I would like to do is
> >>> to extract everything (xml tags and data) between nth  tag and (n
> >>> +k)th  tag. The extracted data is to be
> >>> written down to a separate file.
> >>> Thanks...
> >> You could create a generator expression out of it:
>
> >> txt = """
> >>     1
> >>     2
> >>     3
> >>     4
> >>     5
> >>     
> >>     """
> >> l = len(txt.split('r2>'))-1
> >> a = ('%sr2>'%i for j,i in enumerate(txt.split('r2>')) if 0 < j < l
> >> and i.replace('>','').replace('<','').strip())
>
> >> Now you have a generator you can iterate through with a.next() or
> >> alternatively you could just create a list out of it by replacing the
> >> outer parens with square brackets.- Hide quoted text -
>
> >> - Show quoted text -
>
> > Hmmm... will look into it.. Thanks
>
> > the XML file is almost a TB in size...
>
> Good grief. When will people stop abusing XML this way?
>
> > so SAX will have to be the parser i'm thinking of doing something
> > to split the file using SAX
> > ... Any suggestions on those lines..? If there are any other parsers
> > suitable, please suggest...
>
> You could try pulldom, but the documentation is disgraceful.
>
> ElementTree.iterparse *might* help.
>
> regards
>   Steve
>
> --
> Steve Holden        +1 571 484 6266   +1 800 494 3119
> Holden Web LLC              http://www.holdenweb.com/

I abuse it because I can (and because I don't generally work with XML
files larger than 20-30meg) :)
And the OP never said the XML file for 1TB in size, which makes things
different.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Rationale for read-only property of co_code

[João Neves]

On Apr 3, 4:43 am, Scott David Daniels <[EMAIL PROTECTED]> wrote:
> Nope:  If you change the code in-place, the whole stack's references
> to where they were running would need to get updated to corresponding
> locations in the new code.  _That_ is a lot of work.

Ah, there it is. Now I get it, it makes perfect sense.
Looks like I'll have to stick to the usual mechanisms!
Thanks everyone!

---
João Neves
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: splitting an XML file on the basis on basis of XML tags

[Marco Mariani]

Steve Holden wrote:

>> the XML file is almost a TB in size...
>>
> Good grief. When will people stop abusing XML this way?

Not before somebody writes a clever xmlfs for the linux kernel :-/

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: unicode in exception traceback

[Peter Otten]

WaterWalk wrote:

> Hello. I just found on Windows when an exception is raised and
> traceback info is printed on STDERR, all the characters printed are
> just plain ASCII. Take the unicode character u'\u4e00' for example. If
> I write:
> 
> print u'\u4e00'
> 
> If the system locale is "PRC China", then this statement will print
> this character as a single Chinese character.
> 
> But if i write: assert u'\u4e00' == 1
> 
> An AssertionError will be raised and traceback info will be put to
> STDERR, while this time, u'\u4e00' will simply be printed just as
> u'\u4e00', several ASCII characters instead of one single Chinese
> character. I use the coding directive commen(# -*- coding: utf-8 -*-)t
> on the first line of Python source file and also save it in utf-8
> format, but the problem remains.
> 
> What's worse, if i directly write Chinese characters in a unicode
> string, when the traceback info is printed, they'll appear in a non-
> readable way, that is, they show themselves as something else. It's
> like printing something DBCS characters when the locale is incorrect.
> 
> I think this problem isn't unique. When using some other East-Asia
> characters, the same problem may recur.
> 
> Is there any workaround to it?

Pass a byte string but make some effort to use the right encoding:

>>> assert False, u"\u4e00".encode(sys.stdout.encoding or "ascii", 
>>> "xmlcharrefreplace")
Traceback (most recent call last):
  File "", line 1, in 
AssertionError: 一

You might be able to do this in the except hook:

$ cat unicode_exception_message.py
import sys

def eh(etype, exc, tb, original_excepthook=sys.excepthook):
message = exc.args[0]
if isinstance(message, unicode):
exc.args = (message.encode(sys.stderr.encoding or "ascii", 
"xmlcharrefreplace"),) + exc.args[1:]
return original_excepthook(etype, exc, tb)

sys.excepthook = eh

assert False, u"\u4e00"

$ python unicode_exception_message.py
Traceback (most recent call last):
  File "unicode_exception_message.py", line 11, in 
assert False, u"\u4e00"
AssertionError: 一

If python cannot figure out the encoding this falls back to ascii with 
xml charrefs:

$ python unicode_exception_message.py 2>tmp.txt
$ cat tmp.txt
Traceback (most recent call last):
  File "unicode_exception_message.py", line 11, in 
assert False, u"\u4e00"
AssertionError: 一

Note that I've not done any tests; e.g. if there are exceptions with 
immutable .args the except hook itself will fail.

Peter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Rationale for read-only property of co_code

[Peter Otten]

João Neves wrote:

> Let me give a very basic example. Say we have these two functions:

I suppose you mean

>>> def inc(x): return x + 1
...
>>> def dec(x): return x - 1
...
>>> inc(1), dec(1)
(2, 0)


> Examining the compiled bytecodes for these two functions:
> 
> >>> inc.func_code.co_code
> '|\x00\x00d\x01\x00\x17}\x00\x00d\x00\x00S'
> 
> >>> dec.func_code.co_code
> '|\x00\x00d\x01\x00\x18}\x00\x00d\x00\x00S'
> 
> Now suppose that I wanted to mess with inc, and have it behave like
> dec.

>>> inc.func_code = dec.func_code
>>> inc(1), dec(1)
(0, 0)

There you are, and there wasn't even a slight chance that you combined the
byte code with an incompatible function signature ;)

Peter
-- 
http://mail.python.org/mailman/listinfo/python-list

Get all strings matching given RegExp

[Alex9968]

Can I get sequence of all strings that can match a given regular 
expression? For example, for expression '(a|b)|(x|y)' it would be ['ax', 
'ay', 'bx', 'by']

It would be useful for example to pass these strings to a search engine 
not supporting RegExp (therefore adding such support to it). A program 
can also let user specify sequence of strings using RegExp (filenames to 
process, etc.). If there are other types of expressions for these 
purposes, please let me know.

I know that for some expressions there would be infinite amount of 
matching strings, but these aren't the cases I'm considering. It'd still 
be possible if string length is limited (there might be large but finite 
number of matching strings).

Thanks
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: splitting an XML file on the basis on basis of XML tags

[Marco Mariani]

Marco Mariani wrote:

>>> the XML file is almost a TB in size...
>>>
>> Good grief. When will people stop abusing XML this way?
> 
> Not before somebody writes a clever xmlfs for the linux kernel :-/

Ok.

I meant it as a joke, but somebody has been there and done that.

Twice.


http://xmlfs.modry.cz/user_documentation/

http://www.haifa.ibm.com/projects/storage/xmlfs/index.html
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Prototype OO

[sam]

Bruno Desthuilliers napisał(a):

> Ok, I'm going to be a bit harsh, but this time I'll assume it.

> Sam, you started this thread by asking about prototype vs class based 

> minor syntactic points that, whether you like them or not (and

I think I will get back to this discussion after learning "descriptor protocol" 
and maybe I will not talk about syntax then, and maybe it won't get off topic.

As you can see I'm novice in Python, but I can show python-experts some newbie 
opinions. If somebody is an expert then he can't take a fresh look at his many 
years work.

Anyway -- thank you for spending time and giving explanations.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: State of ctypes Support on HP-UX?

[Thomas Heller]

Phil Thompson schrieb:
> Could somebody confirm how well ctypes is supported on HP-UX (for both 
> PA-RISC 
> and Itanium) for both Python v2.4 and v2.5?
> 
> I don't have access to an HP system and Google doesn't come up with a 
> definitive answer (which may just mean it works fine, but prior experience 
> with HP means I'd like more specific assurances).
> 
> Thanks,
> Phil

I cannot answer your question, but if you want to try it out
yourself there is the HP testdrive program: http://www.testdrive.hp.com/

Thomas

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Prototype OO

[Marc 'BlackJack' Rintsch]

On Thu, 03 Apr 2008 10:07:38 +0200, sam wrote:

> [EMAIL PROTECTED] napisał(a):
> 
> 
>> So, while I often use Python's lambdas, the imposed limitations is ok
>> to me since I wouldn't use it for anything more complex. 
> 
>> Also - as a side note - while the syntax is a bit different, the
>> resulting object is an ordinary function.
> 
> And people start asking why this is that or the other way in Python, and you 
> can't give a good answer to newcomers, especially if Python was chosen as a 
> first learning language. You can't tell "because Python's parsing mechanism 
> don't allow statements in expressions...".

But one can tell "because the way Python's syntax is based on indentation,
no one has come up with a syntax for anonymous functions without the
limitations of the current ``lambda``, that doesn't need 'curly braces' and
is still as readable as Python is now."

Ciao,
Marc 'BlackJack' Rintsch
-- 
http://mail.python.org/mailman/listinfo/python-list

Efficient way of testing for substring being one of a set?

[tinnews]

What's the neatest and/or most efficient way of testing if one of a
set of strings (contained in a dictionary, list or similar) is a
sub-string of a given string?

I.e. I have a string delivered into my program and I want to see if
any of a set of strings is a substring of the string I have been
given.  It's quite OK to stop at the first one found.  Ideally the
strings being searched through will be the keys of a dictionary but
this isn't a necessity, they can just be in a list if it could be done
more efficiently using a list.


Is this the best one can do (ignoring the likelihood that I've got
some syntax wrong) :-

# l is the list
# str is the incoming string
answer = ""
for x in l:
if str.find(x) < 0:
continue
answer = x


-- 
Chris Green
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Efficient way of testing for substring being one of a set?

[Paul Hankin]

On Apr 3, 12:37 pm, [EMAIL PROTECTED] wrote:
> What's the neatest and/or most efficient way of testing if one of a
> set of strings (contained in a dictionary, list or similar) is a
> sub-string of a given string?
>
> I.e. I have a string delivered into my program and I want to see if
> any of a set of strings is a substring of the string I have been
> given.  It's quite OK to stop at the first one found.  Ideally the
> strings being searched through will be the keys of a dictionary but
> this isn't a necessity, they can just be in a list if it could be done
> more efficiently using a list.
>
> Is this the best one can do (ignoring the likelihood that I've got
> some syntax wrong) :-
>
> # l is the list
> # str is the incoming string
> answer = ""
> for x in l:
> if str.find(x) < 0:
> continue
> answer = x

I'd not use 'l' (confused with '1') or 'str' (a standard module) as
variable names. Your code checks every string in the list even when
it's found one... you can reverse the test and break when the first
one is found. Using 'in' rather than testing the return value of find
is nicer as a substring test. Finally, using the 'else' clause lets
you make it clear that answer is set to the empty string when no match
is found.

for answer in l:
if str in answer: break
else:
answer = ''

--
Paul Hankin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Get all strings matching given RegExp

[Marc Christiansen]

Alex9968 <[EMAIL PROTECTED]> wrote:
> Can I get sequence of all strings that can match a given regular 
> expression? For example, for expression '(a|b)|(x|y)' it would be ['ax', 
> 'ay', 'bx', 'by']
> 
> It would be useful for example to pass these strings to a search engine 
> not supporting RegExp (therefore adding such support to it). A program 
> can also let user specify sequence of strings using RegExp (filenames to 
> process, etc.). If there are other types of expressions for these 
> purposes, please let me know.
> 
> I know that for some expressions there would be infinite amount of 
> matching strings, but these aren't the cases I'm considering. It'd still 
> be possible if string length is limited (there might be large but finite 
> number of matching strings).

This will give you all (byte-)strings upto a given length which match a
given regular expression. But beware, it can be slow ;)

import re

all_chars = [chr(i) for i in xrange(256)]

def gen_strings(length, alphabet=all_chars):
if length == 1:
for c in alphabet:
yield c
else:
for i in alphabet:
yield c
for s in gen_strings(length - 1, alphabet):
yield c + s

def regex_matches(regex, max_length, alphabet=all_chars):
r = re.compile('(' + regex + r')\Z')
return (s for s in gen_strings(max_length, alphabet) if r.match(s))

Marc
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Efficient way of testing for substring being one of a set?

[tinnews]

Paul Hankin <[EMAIL PROTECTED]> wrote:
> On Apr 3, 12:37 pm, [EMAIL PROTECTED] wrote:
> > What's the neatest and/or most efficient way of testing if one of a
> > set of strings (contained in a dictionary, list or similar) is a
> > sub-string of a given string?
> >
> > I.e. I have a string delivered into my program and I want to see if
> > any of a set of strings is a substring of the string I have been
> > given.  It's quite OK to stop at the first one found.  Ideally the
> > strings being searched through will be the keys of a dictionary but
> > this isn't a necessity, they can just be in a list if it could be done
> > more efficiently using a list.
> >
> > Is this the best one can do (ignoring the likelihood that I've got
> > some syntax wrong) :-
> >
> > # l is the list
> > # str is the incoming string
> > answer = ""
> > for x in l:
> > if str.find(x) < 0:
> > continue
> > answer = x
> 
> I'd not use 'l' (confused with '1') or 'str' (a standard module) as
> variable names.

Neither would I, it was just thrown together to show how I was
thinking.

> Your code checks every string in the list even when
> it's found one... you can reverse the test and break when the first
> one is found. Using 'in' rather than testing the return value of find
> is nicer as a substring test. Finally, using the 'else' clause lets
> you make it clear that answer is set to the empty string when no match
> is found.
> 
> for answer in l:
> if str in answer: break
> else:
> answer = ''
> 
OK, that does improve things somewhat, thanks.

-- 
Chris Green
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Efficient way of testing for substring being one of a set?

[tinnews]

Jeff <[EMAIL PROTECTED]> wrote:
> def foo(sample, strings):
> for s in strings:
> if sample in s:
> return True
> return False
> 
> This was an order of magnitude faster for me than using str.find or
> str.index.  That was finding rare words in the entire word-list (w/
> duplicates) of War and Peace.

However it's the wrong way around, in my case 'sample' is the longer
string and I want to know if s is in it.  It's simple enough to do it
the other way around though:-

def foo(sample, strings):
for s in strings:
if s in sample:
return True
return False

Using in rather than find() and making it a function would seem to be
the way to go, thanks.

-- 
Chris Green
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Efficient way of testing for substring being one of a set?

[George Sakkis]

On Apr 3, 8:03 am, Jeff <[EMAIL PROTECTED]> wrote:
> def foo(sample, strings):
> for s in strings:
> if sample in s:
> return True
> return False
>
> This was an order of magnitude faster for me than using str.find or
> str.index.  That was finding rare words in the entire word-list (w/
> duplicates) of War and Peace.

If you test against the same substrings over and over again, an
alternative would be to build a regular expression:

import re
search = re.compile('|'.join(re.escape(x)
 for x in substrings)).search
p = search(somestring)
if p is not None:
  print 'Found', p.group()


George
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python queue madness

[nnp]

Hrm, it sounds likely that I am using something mutable and that is messing
things up. I'll look into it.

As for providing sample code to recreate the problem, I would find it
difficult I think to provide a simple example that accurately reflects what
is truly going on so there wouldn't be much point.

Cheers,
nnp

On Thu, Apr 3, 2008 at 5:39 AM, Gabriel Genellina <[EMAIL PROTECTED]>
wrote:

> En Wed, 02 Apr 2008 10:52:08 -0300, nnp <[EMAIL PROTECTED]> escribió:
>
> > Basically I have a system where component 1, 2 and 3 communicate with
> > each
> > other using two Python Queues, we'll call them R and W. Here is what is
> > happening
> >
> > 1 writes data to W and reads from R
> > 2 reads data from W and writes data it receives from 3 to R (but not
> > data it
> > receives from 1)
> > 3 writes to W
> >
> > The problem is that data being written by 1 to W is appearing back on R.
> > I
> > have verified that 1 never writes to R and that 2 never writes data it
> > receives from 1 to R, by overwriting the put() and put_nowait() methods
> > of
> > R.
> >
> > Is there any other way for data to get onto a queue or are there any
> > known
> > bugs with Python's Queue module that could lead to this kind of
> > behaviour?
>
> Yes, your own code :)
> Perhaps you put mutable objects into the queue, like a list? and later
> modify the list in another place?
>
> --
> Gabriel Genellina
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>



-- 
http://www.smashthestack.org
http://www.unprotectedhex.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Get all strings matching given RegExp

[Jeff]

I don't think there is any built in way.  Regular expressions are
compiled into an expanded pattern internally, but I don't think that
it is anything that would be useful for you to directly access.

If you are interested in a lot of work, you could do something with
PLY and write an re parser that would expand it into a series of
possible textual matches :)
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Efficient way of testing for substring being one of a set?

[Jeff]

def foo(sample, strings):
for s in strings:
if sample in s:
return True
return False

This was an order of magnitude faster for me than using str.find or
str.index.  That was finding rare words in the entire word-list (w/
duplicates) of War and Peace.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Prototype OO

[Gabriel Genellina]

En Thu, 03 Apr 2008 05:20:48 -0300, sam <[EMAIL PROTECTED]> escribió:

> Gabriel Genellina napisał(a):
>
>> Class methods and instance methods are not just standard functions;
>> instance methods were plain functions before 2.2 and the Class object
>> was in charge of doing the "self magic". Now the descriptor protocol
>> provides far more possibilities.
>
> Actually I don't know what is "descriptor protocol", so maybe I should  
> have
> finished discussing. I will aslo search for "self magic" -- some pieces  
> of old
> code, or something.

I meant the way instance methods acquire their "self" argument; the  
transformation from someobject.foo(arg1,arg2) into  
(ClassOfSomeobject.foo)(someobject, arg1, arg2) that classic classes do  
themselves and could not be changed nor extended. The descriptor protocol  
provides a way for attributes to modify the way they're handled; this is  
how instance methods get their self argument, and allows for the existence  
of class methods, static methods, and properties, among other things.
The descriptor protocol is very important for Python and understanding it  
is a good idea, but not before you are rather fluent with the language or  
it will be fairly incomprehensible.

-- 
Gabriel Genellina

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange MySQL Problem...

[Victor Subervi]

Comments in line...
On Thu, Apr 3, 2008 at 12:35 AM, John Nagle <[EMAIL PROTECTED]> wrote:

> Steve Holden wrote:
>


> Define "no longer works".
>

Sorry. Throws HTTP 200 error.

On Thu, Apr 3, 2008 at 12:35 AM, John Nagle <[EMAIL PROTECTED]> wrote:

> John Nagle wrote:
> > "works fine"? Please check again...
> > The same remarks I've posted earlier apply here.
>

Must have missed those. Yes, the code works fine. Repeatedly.

>
>
>In addition, you're not committing the database update.
> You need to do
>
>connection.commit()


Oh, thanks :)

>
>
> after updating.
>
>In general, server side programs should have a try-block
> wrapped around most of the program, with some code to display or
> log errors in some useful way.


Curious. Why? During testing, I understand, but after testing, why?

On Thu, Apr 3, 2008 at 12:35 AM, John Nagle <[EMAIL PROTECTED]> wrote:

> Gabriel Genellina wrote:
>

> > print 'Content-Type: image/jpeg\r\n'
> > print '\nHi!\n'
>
> Don't you see a conflict among those two lines?
> (You're a liar: first you promise to send a nice picture and then you only
> send a letter!)
>

LOL! Okay, make me honest ;) I want to post both text and images. What use?


>
>
> > cursor.execute("CREATE TABLE justatest (name TEXT, ablob BLOB)")
>
> Note that you *always* attempt to create a table.
>

Yeah, I am dropping it, too. This was just some code I found online and
tweaked.


>
>
> > sql = "INSERT INTO justatest VALUES (%s, %s)"
> > cursor.execute(sql, ('ramis', _mysql.escape_string(f)) )
>
> You're using bound parameters now, a good thing. There is no need to
> escape strings passed as parameters - in fact, it is wrong, as the adapter
> will escape the text again.
> In this case, the column is a BLOB, and you have to use the Binary
> function: MySQLdb.Binary(f)
>

Nope. Threw me another HTTP 200.


> > Now, if I take out this part, which I can´t see does anything at all in
> > the
> > code, it no longer works:
>
> I don't think "it no longer works" because you removed anything, but
> because this script can only be run at most once. Probably you're getting
> an error in the CREATE TABLE statement.
>

Wrong. I check the mysql and drop the table if the code throws me an HTTP
200 error. I have run this code many times now. That is what makes the whole
thing so ridiculously strange to me. That code snippet, that the script
insists on, is a relic from the code I tweaked. It was there to make a
binary, that is all. So why do I still need the darn thing??

>
> You have to look at the server error logs, else you're blind fighting.
>

I am blind fighting. No longer run my own server, no longer have root
access.


> See
> other suggestions in my previous post.
>

Sorry. Can you repost it?
TIA,
Victor
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Efficient way of testing for substring being one of a set?

[Ant]

On Apr 3, 12:37 pm, [EMAIL PROTECTED] wrote:
> What's the neatest and/or most efficient way of testing if one of a

A different approach:

>>> words = ["he", "sh", "bla"]
>>> name = "blah"
>>> True in (word in name for word in words)
True

>>> name = "bling"
>>> True in (word in name for word in words)
False

Perhaps not as obvious or readable as Jeff's example, but is
essentially doing the same thing using generator syntax.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: who said python can't be obsfucated!?

[Marco Mariani]

Bruno Desthuilliers wrote:

> sig=lambda m:'@'.join(['.'.join([w[::-1] for w in p.split('.')]) for p 
> in m.split('@')])


Pff... you call that a quicksort?


 From http://www.p-nand-q.com/python/obfuscated_python.html


import sys

funcs = range(10)

def A(_,o):
 _[3]=_[5]()

def B(_,o):
 o[_[2]]=_[9]()

def C(_,o):
 _[3]=_[7]()

def D(_,o):
 o[_[1]]=_[14]()

def E(_,o):
 _[1]=_[4]()

def F(_,o):
 _[2]=_[6]()

def G(_,o,O):
 if _[O[0]]():return O[-1](_,o) or 1

def H(o, start, stop):
 _=[o[stop],[lambda x,y:x+y,lambda x,y:x-y,lambda x,
 y:y|1,0,0][1](start,funcs[4](range(funcs[3](),
 len(o[:],stop,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]

 for i in range(4,19):
 _[i]=lambda _=_,o=o,s="reduce([lambda x,y:x+y,lambda "\
   "x,y:x-y,lambda x,y:y|1,0,0][0],[_[1],funcs[4]("\
   "range(eval(\"funcs[3]()\"),_[10]()))])$funcs[4"\
   "](range(eval(\"funcs[3]()\"),_[10]()))$[lambda"\
   " x,y:x+y,lambda x,y:x-y,lambda x,y:y|1,0,0][1]"\
   "(_[2],funcs[4](range(funcs[3](),_[10]($fun"\
   "cs[4](range(funcs[3](),_[10]()))$range(_[10]()"\
   "*_[10]())$o[:][_[1]]$len(o[:])$not _[3]$_[1]=="\
   "_[2]$o[:][_[1]]>_[0]$o[:][_[2]]$o[_[2]]<_[0]$_"\
   "[2]==_[1]$_[11]() and not E(_,0) and not G(_,o"\
   ",[12,A]) and not G(_,o,[13,B])$_[11]() and not"\
   " F(_,_) and not G(_,o,[16,C]) and not G(_,o,[1"\
   "5,D])".split('$')[:][i-4]:eval("eval('eval(s)')")

 while _[11]():
 while _[17](): pass
 while _[18](): pass
 o[_[2]] = _[0]
 return _[2]

def quicksort(list,start,stop):
 exec('funcs[3] = lambda:reduce([lambda x,y:x+y,lambda x,y'\
  ':x-y,lambda x,y:y|1,0,0][1],[[lambda x,y:x+y,lambda'\
  ' x,y:x-y,lambda x,y:y|1,0,0][2](200,200)]*2)\nfuncs'\
  '[4] = lambda x:reduce(lambda x,y:y%2,range(eval("re'\
  'duce([lambda x,y:x+y,lambda x,y:x-y,lambda x,y:y|1,'\
  '0,0][2],[len(o[:]),len(o[:])])"),eval("reduce([lamb'\
  'da x,y:x+y,lambda x,y:x-y,lambda x,y:y|1,0,0][2],[l'\
  'en(o[:]),len(o[:])])")+((len(o)and 3)or 3)))\nif st'\
  'art < stop:\n\tsplit = H(list, start, stop)\n\tquic'\
  'ksort(list, start, [lambda x,y:x+y,lambda x,y:x-y,l'\
  'ambda x,y: y|1,0,0][1](split,funcs[4](funcs[3]('\
  '\n\tquicksort(list, reduce([lambda x,y:x+y,lambda x'\
  ',y:x-y,lambda x,y:y|1,0,0][0],[split,funcs[4](funcs'\
  '[3]())]), stop)\n')

# test code: 2000 elements to sort
list = []
import whrandom,time
for i in range(2000):
 list.append(whrandom.randint(1,100))
start = time.clock()
quicksort(list,0,len(list)-1)
print "Sorting took %.2f" % (time.clock() - start)

# just a test loop to see if everything *is* sorted
element = -1
for i in list:
 if i >= element:
 element = i
 else:
 print "FUNK DAT: %20s" % str(i)
 break

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: who said python can't be obsfucated!?

[Marco Mariani]

Bruno Desthuilliers wrote:

> sig=lambda m:'@'.join(['.'.join([w[::-1] for w in p.split('.')]) for p 
> in m.split('@')])


Pff... you call that a quicksort?


 From http://www.p-nand-q.com/python/obfuscated_python.html


import sys

funcs = range(10)

def A(_,o):
 _[3]=_[5]()

def B(_,o):
 o[_[2]]=_[9]()

def C(_,o):
 _[3]=_[7]()

def D(_,o):
 o[_[1]]=_[14]()

def E(_,o):
 _[1]=_[4]()

def F(_,o):
 _[2]=_[6]()

def G(_,o,O):
 if _[O[0]]():return O[-1](_,o) or 1

def H(o, start, stop):
 _=[o[stop],[lambda x,y:x+y,lambda x,y:x-y,lambda x,
 y:y|1,0,0][1](start,funcs[4](range(funcs[3](),
 len(o[:],stop,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]

 for i in range(4,19):
 _[i]=lambda _=_,o=o,s="reduce([lambda x,y:x+y,lambda "\
   "x,y:x-y,lambda x,y:y|1,0,0][0],[_[1],funcs[4]("\
   "range(eval(\"funcs[3]()\"),_[10]()))])$funcs[4"\
   "](range(eval(\"funcs[3]()\"),_[10]()))$[lambda"\
   " x,y:x+y,lambda x,y:x-y,lambda x,y:y|1,0,0][1]"\
   "(_[2],funcs[4](range(funcs[3](),_[10]($fun"\
   "cs[4](range(funcs[3](),_[10]()))$range(_[10]()"\
   "*_[10]())$o[:][_[1]]$len(o[:])$not _[3]$_[1]=="\
   "_[2]$o[:][_[1]]>_[0]$o[:][_[2]]$o[_[2]]<_[0]$_"\
   "[2]==_[1]$_[11]() and not E(_,0) and not G(_,o"\
   ",[12,A]) and not G(_,o,[13,B])$_[11]() and not"\
   " F(_,_) and not G(_,o,[16,C]) and not G(_,o,[1"\
   "5,D])".split('$')[:][i-4]:eval("eval('eval(s)')")

 while _[11]():
 while _[17](): pass
 while _[18](): pass
 o[_[2]] = _[0]
 return _[2]

def quicksort(list,start,stop):
 exec('funcs[3] = lambda:reduce([lambda x,y:x+y,lambda x,y'\
  ':x-y,lambda x,y:y|1,0,0][1],[[lambda x,y:x+y,lambda'\
  ' x,y:x-y,lambda x,y:y|1,0,0][2](200,200)]*2)\nfuncs'\
  '[4] = lambda x:reduce(lambda x,y:y%2,range(eval("re'\
  'duce([lambda x,y:x+y,lambda x,y:x-y,lambda x,y:y|1,'\
  '0,0][2],[len(o[:]),len(o[:])])"),eval("reduce([lamb'\
  'da x,y:x+y,lambda x,y:x-y,lambda x,y:y|1,0,0][2],[l'\
  'en(o[:]),len(o[:])])")+((len(o)and 3)or 3)))\nif st'\
  'art < stop:\n\tsplit = H(list, start, stop)\n\tquic'\
  'ksort(list, start, [lambda x,y:x+y,lambda x,y:x-y,l'\
  'ambda x,y: y|1,0,0][1](split,funcs[4](funcs[3]('\
  '\n\tquicksort(list, reduce([lambda x,y:x+y,lambda x'\
  ',y:x-y,lambda x,y:y|1,0,0][0],[split,funcs[4](funcs'\
  '[3]())]), stop)\n')

# test code: 2000 elements to sort
list = []
import whrandom,time
for i in range(2000):
 list.append(whrandom.randint(1,100))
start = time.clock()
quicksort(list,0,len(list)-1)
print "Sorting took %.2f" % (time.clock() - start)

# just a test loop to see if everything *is* sorted
element = -1
for i in list:
 if i >= element:
 element = i
 else:
 print "FUNK DAT: %20s" % str(i)
 break

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Efficient way of testing for substring being one of a set?

[Jeff]

On Apr 3, 8:19 am, George Sakkis <[EMAIL PROTECTED]> wrote:
> On Apr 3, 8:03 am, Jeff <[EMAIL PROTECTED]> wrote:
>
> > def foo(sample, strings):
> >         for s in strings:
> >                 if sample in s:
> >                         return True
> >         return False
>
> > This was an order of magnitude faster for me than using str.find or
> > str.index.  That was finding rare words in the entire word-list (w/
> > duplicates) of War and Peace.
>
> If you test against the same substrings over and over again, an
> alternative would be to build a regular expression:
>
> import re
> search = re.compile('|'.join(re.escape(x)
>                              for x in substrings)).search
> p = search(somestring)
> if p is not None:
>   print 'Found', p.group()
>
> George

That would be an enormous regular expression and eat a lot of memory.
But over an enormous number of substrings, it would be O(log n),
rather than O(n).
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Efficient way of testing for substring being one of a set?

[Jeff]

On Apr 3, 8:44 am, Ant <[EMAIL PROTECTED]> wrote:
> On Apr 3, 12:37 pm, [EMAIL PROTECTED] wrote:
>
> > What's the neatest and/or most efficient way of testing if one of a
>
> A different approach:
>
> >>> words = ["he", "sh", "bla"]
> >>> name = "blah"
> >>> True in (word in name for word in words)
>
> True
>
> >>> name = "bling"
> >>> True in (word in name for word in words)
>
> False
>
> Perhaps not as obvious or readable as Jeff's example, but is
> essentially doing the same thing using generator syntax.

That's pretty :)
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: object-relational mappers

[Paul Boddie]

On 2 Apr, 15:50, Aaron Watters <[EMAIL PROTECTED]> wrote:
>

[Quoting hdante]

> >  Seriously, you'll forget there's a relational database below. (there
> > are even intefaces for "relational lists", "trees", etc.)
>
> My experience with this sort of thing is that it is a bit
> like morphine.  It can feel really good, and in emergencies
> it can save you a lot of pain.  But if you use it too often
> and too seriously you end up with really big problems.

That's two candidates for quote of the week in the same thread!

I agree with those who question why you'd want to treat a relational
database like a big dictionary, and although the interface between
queries, results and program data structures can often seem quite
awkward, I've come to realise that most object-relational mappers are
solving the wrong problems: they pretend that the database is somehow
the wrong representation whilst being a fast enough black box for
holding persistent data (although I doubt that many people push the
boundaries enough to see that it's not possible to ignore all details
of such a database whilst preserving performance), or they pretend
that languages like SQL (which can be cumbersome, admittedly) are
inconvenient for querying whilst replicating a less concise mechanism
for querying using client language mechanisms.

I'm more encouraged by the idea of "query templating", which might
sound like a recipe for all sorts of problems, but if done right could
provide more effective ways of working with relational databases than
pretending that different things in the database are somehow "objects"
in the client language sense.

Paul
-- 
http://mail.python.org/mailman/listinfo/python-list

A question about creating Maildir mailboxes with mailbox.Maildir

[tinnews]

I have mail delivered into Maildirs which live in a directory
hierarchy as follows:-

/home/isbd/Mail/Li/
/home/isbd/Mail/In/

Note that neither /home/isbd/Mail/Li nor /home/isbd/Mail/In are
Maildir mailboxes.  How do I create a new Maildir mailbox in either Li
or In with mailbox.Maildir.add_folder()?

If I have created an instance of Maildir with:-

md = mailbox.Maildir("/home/isbd/Mail/Li")

that implies that /home/isbd/Mail/Li is a Maildir, which it isn't, and
anyway the above fails because it can't see the new, cur and tmp
directories in /home/isbd/Mail/Li.


It seems that the Python Maildir implementation assumes a Courier
style (probably IMAP) Maildir hierarchy with an INBOX and using dots
rather than real directories to create sub-directories.

I don't have an INBOX or any sort of 'home' maildir, it's most
definitely not a requirement of maildir.

-- 
Chris Green
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Efficient way of testing for substring being one of a set?

[[EMAIL PROTECTED]]

On Apr 3, 1:37 pm, [EMAIL PROTECTED] wrote:
> What's the neatest and/or most efficient way of testing if one of a
> set of strings (contained in a dictionary, list or similar) is a
> sub-string of a given string?
> [...]

You could use the Aho-Corasick algorithm .
I don't know if there's a Python implementation yet.

Dennis Benzinger
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Efficient way of testing for substring being one of a set?

[bearophileHUGS]

Dennis Benzinger:
> You could use the Aho-Corasick algorithm  Aho-Corasick_algorithm>.
> I don't know if there's a Python implementation yet.

http://hkn.eecs.berkeley.edu/~dyoo/python/ahocorasick/

Bye,
bearophile
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: object-relational mappers

[Tim Golden]

Paul Boddie wrote:
> ... I've come to realise that most object-relational mappers are
> solving the wrong problems: they pretend that the database is somehow
> the wrong representation whilst being a fast enough black box for
> holding persistent data (although I doubt that many people push the
> boundaries enough to see that it's not possible to ignore all details
> of such a database whilst preserving performance), or they pretend
> that languages like SQL (which can be cumbersome, admittedly) are
> inconvenient for querying whilst replicating a less concise mechanism
> for querying using client language mechanisms.


Well at the risk of oversimplifying (!) I find there are two kinds
of programmers using databases: those, like me, for whom the database
is the application and who can happily envisage any number of
interfaces, human and otherwise, to the data; and those, like 70% of the
people I've ever interviewed for a job as a SQL developer, for whom
the interface is the application and who simply throw things at whatever
database they're presented with.

The former will more likely tend to reach first for SQL to retrieve
their data efficiently before passing it on to the front end for
presentation or manipulation. The latter (and I've seen this far
too often in interviews) will basically do "SELECT * FROM x WHERE y"
to pull everything back into their VB.Net app where they feel more
at home. Or, in the case of Python, reach for an ORM.

I've recently used Elixir and found it very useful for a small-scale
database with no more than a dozen tables, well-structured and
easily understood. I'd certainly use it again for anything like that
to save me writing what would amount to boilerplate SQL. But I'd
hate to imagine it in the context of my day job: a messy, organic
and sprawling SQL Server database with over 1,000 tables, let alone
views, procedures and so on.


TJG
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Parsing HTML?

[Paul Boddie]

On 3 Apr, 06:59, Benjamin <[EMAIL PROTECTED]> wrote:
> I'm trying to parse an HTML file.  I want to retrieve all of the text
> inside a certain tag that I find with XPath.  The DOM seems to make
> this available with the innerHTML element, but I haven't found a way
> to do it in Python.

With libxml2dom you'd do the following:

 1. Parse the file using libxml2dom.parse with html set to a true
value.
 2. Use the xpath method on the document to select the desired
element.
 3. Use the toString method on the element to get the text of the
element (including start and end tags), or the textContent
property
to get the text between the tags.

See the Package Index page for more details:

  http://www.python.org/pypi/libxml2dom

Paul
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: non-terminating regex match

[Gabriel Genellina]

En Wed, 02 Apr 2008 13:01:59 -0300, Maurizio Vitale  
<[EMAIL PROTECTED]> escribió:

> The intention is to match C++ identifiers, with or without namespace
> qualification, with or without arguments (e.g. variables, functions and
> macros).
> The following should be accepted:
> main
> main(int,char**)
> ::main
> std::cout
> ::std::cout
> NDEBUG

What about foo(int(*f)(int)) ?
You can't match a function definition with a regular expression alone, due  
to the nested (). The r.e. can recognize an identifier; you can later see  
whether there is a ( and scan up to the matching ).

-- 
Gabriel Genellina

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: State of ctypes Support on HP-UX?

[Phil Thompson]

On Thursday 03 April 2008, Thomas Heller wrote:
> Phil Thompson schrieb:
> > Could somebody confirm how well ctypes is supported on HP-UX (for both
> > PA-RISC and Itanium) for both Python v2.4 and v2.5?
> >
> > I don't have access to an HP system and Google doesn't come up with a
> > definitive answer (which may just mean it works fine, but prior
> > experience with HP means I'd like more specific assurances).
> >
> > Thanks,
> > Phil
>
> I cannot answer your question, but if you want to try it out
> yourself there is the HP testdrive program: http://www.testdrive.hp.com/
>
> Thomas

Thanks for the pointer. Unfortunately the answer is that there is no support 
(at least for ctypes v1.0.2).

Phil
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: who said python can't be obsfucated!?

[Bruno Desthuilliers]

Marco Mariani a écrit :
> Bruno Desthuilliers wrote:
> 
>> sig=lambda m:'@'.join(['.'.join([w[::-1] for w in p.split('.')]) for p 
>> in m.split('@')])
> 
> 
> Pff... you call that a quicksort?
> 

Nope, only somewhat obfuscated Python. And it seems it's at least 
obfuscated enough for you to believe it could be a quicksort 
implementation !-)

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Efficient way of testing for substring being one of a set?

[George Sakkis]

On Apr 3, 9:00 am, Jeff <[EMAIL PROTECTED]> wrote:
> On Apr 3, 8:44 am, Ant <[EMAIL PROTECTED]> wrote:
>
>
>
> > On Apr 3, 12:37 pm, [EMAIL PROTECTED] wrote:
>
> > > What's the neatest and/or most efficient way of testing if one of a
>
> > A different approach:
>
> > >>> words = ["he", "sh", "bla"]
> > >>> name = "blah"
> > >>> True in (word in name for word in words)
>
> > True
>
> > >>> name = "bling"
> > >>> True in (word in name for word in words)
>
> > False
>
> > Perhaps not as obvious or readable as Jeff's example, but is
> > essentially doing the same thing using generator syntax.
>
> That's pretty :)

It's even prettier in 2.5:

any(word in name for word in words)

George
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: who said python can't be obsfucated!?

[Marco Mariani]

Bruno Desthuilliers wrote:

>> Pff... you call that a quicksort?
>>
> Nope, only somewhat obfuscated Python. And it seems it's at least 
> obfuscated enough for you to believe it could be a quicksort 
> implementation !-)

You're right, but I'm past the quine age and don't bother parsing 
obfuscated code (*) anymore.




(*) outside my day job, that is.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: object-relational mappers

[Marco Mariani]

Tim Golden wrote:

> I've recently used Elixir and found it very useful for a small-scale
> database with no more than a dozen tables, well-structured and
> easily understood. I'd certainly use it again for anything like that
> to save me writing what would amount to boilerplate SQL. But I'd
> hate to imagine it in the context of my day job: a messy, organic
> and sprawling SQL Server database with over 1,000 tables, let alone
> views, procedures and so on.

That's the scenario where the rest of SQLAlchemy (beyond Elixir, that 
is, and with reflection turned to 11) can do mucho bueno.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: object-relational mappers

[Tim Golden]

Marco Mariani wrote:
> Tim Golden wrote:
> 
>> I've recently used Elixir and found it very useful for a small-scale
>> database with no more than a dozen tables, well-structured and
>> easily understood. I'd certainly use it again for anything like that
>> to save me writing what would amount to boilerplate SQL. But I'd
>> hate to imagine it in the context of my day job: a messy, organic
>> and sprawling SQL Server database with over 1,000 tables, let alone
>> views, procedures and so on.
> 
> That's the scenario where the rest of SQLAlchemy (beyond Elixir, that 
> is, and with reflection turned to 11) can do mucho bueno.

Well, true (and I've done good things with it) but, ultimately
if I need to write SQL I'll write SQL: that's what I'm paid for.
And no matter how good sa's generative queries are -- and they
are good -- I've been writing complex SQL queries for 15 years
and learning a more Pythonesque equivalent doesn't really seem
to offer me anything.

Not to take away from the achievements of SqlAlchemy: I'm just
not really the target market, I think.

TJG
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: object-relational mappers

[Luis M . González]

I have come to the same conclusion.
ORMs make easy things easier, but difficult things impossible...

The best approach I've seen so far is webpy's (if we are talking of
web apps).
It isn't an ORM, it is just a way to make the database api easier to
use.
Queries don't return objects, they return something similar to
dictionaries, which can be used with dot notation ( for example,
result.name is equal to result['name'] ).

A simple select query would be db.select('customers') or
db.select('customers', name='John').
But you can also resort to plain sql as follows: db.query('select *
from customers where name = "John"').

Simple, effective and doesn't get in your way.

Luis
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: object-relational mappers

[Jarek Zgoda]

Bruno Desthuilliers napisał(a):

> Now my own experience is that whenever I tried this approach for
> anything non-trivial, I ended up building an "ad-hoc,
> informally-specified bug-ridden slow implementation of half of "
> SQLAlchemy. Which BTW is not strictly an ORM, but primarily an attempt
> at a better integration of SQL into Python. So while it may feel like
> learning the inner complexities of SQLALchemy (or Django's ORM which is
> not that bad either) is "wasting brain cells", MVHO is that it's worth
> the time spent. But YMMV of course - IOW, do what works best for you.

I like OR mappers, they save me lot of work. The problem is, all of them
are very resource hungry, processing resultset of 300k objects one by
one can effectively kill most of commodity systems. This is where raw
SQL comes in handy.

-- 
Jarek Zgoda
Skype: jzgoda | GTalk: [EMAIL PROTECTED] | voice: +48228430101

"We read Knuth so you don't have to." (Tim Peters)
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Get all strings matching given RegExp

[Gabriel Genellina]

En Thu, 03 Apr 2008 06:50:52 -0300, Alex9968 <[EMAIL PROTECTED]>  
escribió:

> Can I get sequence of all strings that can match a given regular
> expression? For example, for expression '(a|b)|(x|y)' it would be ['ax',
> 'ay', 'bx', 'by']

See this thread:
http://groups.google.com/group/comp.lang.python/browse_thread/thread/bc6e305844aeee82/

-- 
Gabriel Genellina

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: object-relational mappers

[Bruno Desthuilliers]

Luis M. González a écrit :
> I have come to the same conclusion.
> ORMs make easy things easier, but difficult things impossible...

Not my experience with SQLAlchemy. Ok, I still not had an occasion to 
test it against stored procedures, but when it comes to complex queries, 
it did the trick so far - and (warning: front-end developper 
considerations ahead) happened to be much more usable than raw strings 
to dynamically *build* the queries.

> The best approach I've seen so far is webpy's (if we are talking of
> web apps).
> It isn't an ORM, it is just a way to make the database api easier to
> use.
> Queries don't return objects, they return something similar to
> dictionaries, which can be used with dot notation ( for example,
> result.name is equal to result['name'] ).
> 
> A simple select query would be db.select('customers') or
> db.select('customers', name='John').
> But you can also resort to plain sql as follows: db.query('select *
> from customers where name = "John"').
> 
> Simple, effective and doesn't get in your way.

Seems nice too in another way. Is that part independant of the rest of 
the framework ? If so, I'll have to give it a try at least for admin 
scripts.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How easy is it to install python as non-root user?

[Bruno Desthuilliers]

[EMAIL PROTECTED] a écrit :
> Does python install fairly easily for a non-root user?
> 
> I have an ssh login account onto a Linux system that currently
> provides Python 2.4.3 and I'd really like to use some of the
> improvements in Python 2.5.x.
> 
> So, if I download the Python-2.5.2.tgz file is it just the standard:-
> 
> ./configure --prefix=$HOME
> make
> make install
> 
IIRC there's something like make altinstall - but you'd better read the 
doc (INSTALL.txt anyone ?)

-- 
http://mail.python.org/mailman/listinfo/python-list

How easy is it to install python as non-root user?

[tinnews]

Does python install fairly easily for a non-root user?

I have an ssh login account onto a Linux system that currently
provides Python 2.4.3 and I'd really like to use some of the
improvements in Python 2.5.x.

So, if I download the Python-2.5.2.tgz file is it just the standard:-

./configure --prefix=$HOME
make
make install


-- 
Chris Green
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Rationale for read-only property of co_code

[Arnaud Delobelle]

On Apr 3, 11:10 am, João Neves <[EMAIL PROTECTED]> wrote:
> On Apr 3, 4:43 am, Scott David Daniels <[EMAIL PROTECTED]> wrote:
>
> > Nope:  If you change the code in-place, the whole stack's references
> > to where they were running would need to get updated to corresponding
> > locations in the new code.  _That_ is a lot of work.
>
> Ah, there it is. Now I get it, it makes perfect sense.
> Looks like I'll have to stick to the usual mechanisms!
> Thanks everyone!
>
> ---
> João Neves

FWIW, when I need to 'modify' a code object / function object I use
the following functions:

from new import code, function

code_args = (
'argcount', 'nlocals', 'stacksize', 'flags', 'code',
'consts', 'names', 'varnames', 'filename', 'name',
'firstlineno', 'lnotab', 'freevars', 'cellvars'
)

function_args = ('code', 'globals', 'name', 'defaults', 'closure')

def copy_code(code_obj, **kwargs):
"Return a copy of a code object, maybe changing some attributes"
for arg in code_args:
if not kwargs.has_key(arg):
kwargs[arg] = getattr(code_obj, 'co_%s' % arg)
return code(*map(kwargs.__getitem__, code_args))

def copy_function(func_obj, **kwargs):
"Return a copy of a function object, maybe changing some
attributes)"
for arg in function_args:
if not kwargs.has_key(arg):
kwargs[arg] = getattr(func_obj, 'func_%s' % arg)
return function(*map(kwargs.__getitem__, function_args))

# E.g. to change the code object of a function:

f = copy_function(f, code=new_code_object)

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: object-relational mappers

[Marco Mariani]

Bruno Desthuilliers wrote:

>> A simple select query would be db.select('customers') or
>> db.select('customers', name='John').
>> But you can also resort to plain sql as follows: db.query('select *
>> from customers where name = "John"').
>>
>> Simple, effective and doesn't get in your way.
> 
> Seems nice too in another way.

And no different than using SQLAlchemy's sa.select() or 
engine.execute(), after all.

 > Is that part independant of the rest of the framework ? If so, I'll 
have to give it a try at least for admin
> scripts.

My admin scripts go through SQLAlchemy as well, I just have some issues 
with postgres' COPY statement -- but I don't know if the DBAPI is 
supposed to handle that.
-- 
http://mail.python.org/mailman/listinfo/python-list

RE: manipulating hex values

[Stephen Cattaneo]

Thanks to everyone ( Grant, Cliff, and Gabriel) for responding and
helping me.

Cheers,

Steve

-Original Message-
From: Grant Edwards [mailto:[EMAIL PROTECTED] 
Sent: Tuesday, April 01, 2008 7:46 PM
To: python-list@python.org
Subject: Re: manipulating hex values

On 2008-04-01, Stephen Cattaneo <[EMAIL PROTECTED]> wrote:

>>> I am relatively new to socket programming.  I am attempting to
>>> use raw sockets to spoof my IP address.
>>
>> Don't bother to try...
>
> Is there a better solution to spoofing my IP. then using raw
> sockets 

You'll have to define "spoofing my IP", but I suspect that what
you're trying can't be done by using raw sockets.

> (I'm working on a machine with multiple interfaces and need to
> be able to some how specify which interface that traffic needs
> to be sent/recieved to/from)

That's what routing tables are for.

If you want to send packets using a particular IP, then
configure an interface so that it has that adrress, and then
bind your socket to that address.

> The source of my confusion is that I need to keep my bytes formated 
> correctly.   I am using the below  'raw socket example'
proof-of-concept 
> code as my example.

OK.

> (And yes, I have tried the proof-of-concept.  It works
> correctly.  It is not my code.)

I know.  It's my code. :) I wrote Python's raw socket support
code and the example code that is floating around the 'net.

> dstAddr = "\x01\x02\x03\x04\x05\x06" 
> dstAddr1 = "0x010203040506"
> dstAddr != dstAddr1

Right.

> Follow up question: What is the best to store my bytes up
> until sending the packets?

That depends on what you want to do with them.  Ultimately,
they need to be strings when they're sent out the socket (in
Python a "string" is really just an array of 8-bit bytes).  The
best way to store them is entirely dependent on how you want to
manipulate them before they're sent.

> Perhaps I should use lists of decimal numbers and then 
> before sending convert to hex. 

Again: you're not converting them to hex.  You're converting
them to a python "string" object which is really just an array
of bytes.  Stop saying "hex" when you're talking about a string
(array of bytes).  "hex" is a way to _represent_ a value
textually. It's simply a format used when print a value on
paper or a screen. The value itself isn't hex any more than a
particular instance of Canis lupus familiaris is "English"
because somebody spells it "dog" instead of "chien" or "Hund".

> I.E. dstAddr = [ 1, 2, 3, 4, 5, 6]
> dstAddr.prepareToSend()

I presume you know that list objects don't have a method called
prepareToSend().

> txFrame = struct.pack("!6s6sh",dstAddr,srcAddr,proto) + ethData
>
> Is there a better way to do this?

It's not at all clear what "this" is.  If you want to convert
from a list (or any sequence, really) of integer objects to a
string where each integer is converted into a single byte
within the string, then here are a few ways:

  import operator
  import struct

  intlist = [1,2,3,4,5,6]
  
  s1 = ''.join(chr(b) for b in intlist)
  s2 = reduce(operator.add,[chr(b) for b in intlist])
  s3 = struct.pack("6B",*tuple(intlist))

  print repr(s1)
  print repr(s2)
  print repr(s3)

  print s1==s2
  print s2==s3

When run you get this:

'\x01\x02\x03\x04\x05\x06'
'\x01\x02\x03\x04\x05\x06'
'\x01\x02\x03\x04\x05\x06'
True
True
  
I think the third alternative using struct.pack() is the most
readable, but others will no doubt disagree.

-- 
Grant Edwards   grante Yow!  I'm GLAD I
  at   remembered to XEROX
all
   visi.commy UNDERSHIRTS!!

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: unicode in exception traceback

[WaterWalk]

On Apr 3, 5:56 pm, Peter Otten <[EMAIL PROTECTED]> wrote:
> WaterWalk wrote:
> > Hello. I just found on Windows when an exception is raised and
> > traceback info is printed on STDERR, all the characters printed are
> > just plain ASCII. Take the unicode character u'\u4e00' for example. If
> > I write:
>
> > print u'\u4e00'
>
> > If the system locale is "PRC China", then this statement will print
> > this character as a single Chinese character.
>
> > But if i write: assert u'\u4e00' == 1
>
> > An AssertionError will be raised and traceback info will be put to
> > STDERR, while this time, u'\u4e00' will simply be printed just as
> > u'\u4e00', several ASCII characters instead of one single Chinese
> > character. I use the coding directive commen(# -*- coding: utf-8 -*-)t
> > on the first line of Python source file and also save it in utf-8
> > format, but the problem remains.
>
> > What's worse, if i directly write Chinese characters in a unicode
> > string, when the traceback info is printed, they'll appear in a non-
> > readable way, that is, they show themselves as something else. It's
> > like printing something DBCS characters when the locale is incorrect.
>
> > I think this problem isn't unique. When using some other East-Asia
> > characters, the same problem may recur.
>
> > Is there any workaround to it?
>
> Pass a byte string but make some effort to use the right encoding:
>
> >>> assert False, u"\u4e00".encode(sys.stdout.encoding or "ascii", 
> >>> "xmlcharrefreplace")
>
> Traceback (most recent call last):
>   File "", line 1, in 
> AssertionError: 一
>
> You might be able to do this in the except hook:
>
> $ cat unicode_exception_message.py
> import sys
>
> def eh(etype, exc, tb, original_excepthook=sys.excepthook):
> message = exc.args[0]
> if isinstance(message, unicode):
> exc.args = (message.encode(sys.stderr.encoding or "ascii", 
> "xmlcharrefreplace"),) + exc.args[1:]
> return original_excepthook(etype, exc, tb)
>
> sys.excepthook = eh
>
> assert False, u"\u4e00"
>
> $ python unicode_exception_message.py
> Traceback (most recent call last):
>   File "unicode_exception_message.py", line 11, in 
> assert False, u"\u4e00"
> AssertionError: 一
>
> If python cannot figure out the encoding this falls back to ascii with
> xml charrefs:
>
> $ python unicode_exception_message.py 2>tmp.txt
> $ cat tmp.txt
> Traceback (most recent call last):
>   File "unicode_exception_message.py", line 11, in 
> assert False, u"\u4e00"
> AssertionError: 一
>
> Note that I've not done any tests; e.g. if there are exceptions with
> immutable .args the except hook itself will fail.
>
> Peter

Thanks. My brief test indicates that it works. I'll try it in more
situations.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange MySQL Problem...

[Gabriel Genellina]

En Thu, 03 Apr 2008 09:43:57 -0300, Victor Subervi  
<[EMAIL PROTECTED]> escribió:

>> Steve Holden wrote:
>> Define "no longer works".
> Sorry. Throws HTTP 200 error.

HTTP 200 means OK.

>> > The same remarks I've posted earlier apply here.
> Must have missed those. Yes, the code works fine. Repeatedly.

http://groups.google.com/group/comp.lang.python/browse_thread/thread/b732eee4e91b1868/

>>In general, server side programs should have a try-block
>> wrapped around most of the program, with some code to display or
>> log errors in some useful way.
> Curious. Why? During testing, I understand, but after testing, why?

display or *log* errors...

>> Gabriel Genellina wrote:
>> > print 'Content-Type: image/jpeg\r\n'
>> > print '\nHi!\n'
>>
>> Don't you see a conflict among those two lines?
>> (You're a liar: first you promise to send a nice picture and then you  
>> only
>> send a letter!)
> LOL! Okay, make me honest ;) I want to post both text and images. What  
> use?

*post*? This is the server response. Return either text *or* an image (the  
text may be an html referencing the image)

>> > sql = "INSERT INTO justatest VALUES (%s, %s)"
>> > cursor.execute(sql, ('ramis', _mysql.escape_string(f)) )
>>
>> You're using bound parameters now, a good thing. There is no need to
>> escape strings passed as parameters - in fact, it is wrong, as the  
>> adapter
>> will escape the text again.
>> In this case, the column is a BLOB, and you have to use the Binary
>> function: MySQLdb.Binary(f)
>>
>
> Nope. Threw me another HTTP 200.

Again, HTTP 200 means OK. See section 10.2.1 at  
http://www.faqs.org/rfcs/rfc2616.html

> Wrong. I check the mysql and drop the table if the code throws me an HTTP
> 200 error. I have run this code many times now. That is what makes the  
> whole
> thing so ridiculously strange to me. That code snippet, that the script
> insists on, is a relic from the code I tweaked. It was there to make a
> binary, that is all. So why do I still need the darn thing??

Test it locally (just the database thing, no web), test the cgi script  
without database interaction, only then join the two.

>> You have to look at the server error logs, else you're blind fighting.
> I am blind fighting. No longer run my own server, no longer have root
> access.

But you can read the server logs at least?
Your code is throwing exceptions but you're not seeing them. Use the cgitb  
module http://docs.python.org/lib/module-cgitb.html

-- 
Gabriel Genellina

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: object-relational mappers

[Bruno Desthuilliers]

Jarek Zgoda a écrit :
> Bruno Desthuilliers napisał(a):
> 
>> Now my own experience is that whenever I tried this approach for
>> anything non-trivial, I ended up building an "ad-hoc,
>> informally-specified bug-ridden slow implementation of half of "
>> SQLAlchemy. Which BTW is not strictly an ORM, but primarily an attempt
>> at a better integration of SQL into Python. So while it may feel like
>> learning the inner complexities of SQLALchemy (or Django's ORM which is
>> not that bad either) is "wasting brain cells", MVHO is that it's worth
>> the time spent. But YMMV of course - IOW, do what works best for you.
> 
> I like OR mappers, they save me lot of work. The problem is, all of them
> are very resource hungry, processing resultset of 300k objects one by
> one can effectively kill most of commodity systems. This is where raw
> SQL comes in handy.

The problem here is not about how you build your query but about how you 
retrieve your data. FWIW, SQLAlchemy provides quite a lot of "lower 
level" SQL/Python integration that doesn't require the "object mapping" 
part. "raw SQL" is fine, until you have to dynamically build complex 
queries from user inputs and whatnot. This is where the "low-level" (ie: 
non-ORM) part of SQLAlchemy shines IMHO.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Nested try...except

[Duncan Booth]

Carl Banks <[EMAIL PROTECTED]> wrote:

> Perhaps the advent of with blocks will help reduce this error in the
> future.

Indeed, and to encourage its use I think this thread ought to include the 
'with statement' form of the function:

from __future__ import with_statement
from contextlib import closing

def count(self):
with closing(sqlite.connect(
self.filename,isolation_level=ISOLATION_LEVEL)) as db:
with closing(db.cursor()) as cur:
cur.execute("select count(*) from sessions")
return cur.fetchone()[0]
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How easy is it to install python as non-root user?

[Matt Nordhoff]

[EMAIL PROTECTED] wrote:
> Does python install fairly easily for a non-root user?
> 
> I have an ssh login account onto a Linux system that currently
> provides Python 2.4.3 and I'd really like to use some of the
> improvements in Python 2.5.x.
> 
> So, if I download the Python-2.5.2.tgz file is it just the standard:-
> 
> ./configure --prefix=$HOME
> make
> make install

It is easy, but on an oldish Debian box, I ran into problems where the
headers for some libraries weren't available, so my Python install
didn't support readline or bzip2 (or perhaps other things too).
-- 
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How easy is it to install python as non-root user?

[tinnews]

Bruno Desthuilliers <[EMAIL PROTECTED]> wrote:
> [EMAIL PROTECTED] a écrit :
> > Does python install fairly easily for a non-root user?
> > 
> > I have an ssh login account onto a Linux system that currently
> > provides Python 2.4.3 and I'd really like to use some of the
> > improvements in Python 2.5.x.
> > 
> > So, if I download the Python-2.5.2.tgz file is it just the standard:-
> > 
> > ./configure --prefix=$HOME
> > make
> > make install
> > 
> IIRC there's something like make altinstall - but you'd better read the 
> doc (INSTALL.txt anyone ?)
> 
Well it seems to work perfectly!  :-)

I just did the usual sequence as noted above and I seem to have a
working Python 2.5.2 of my very own.

I didn't need 'make altinstall' because I don't need to be able to see
the existing Python 2.4.3.

It takes a few Mb of disk space but since my free quote is 3Gb or
something that doesn't really worry me.

I'm impressed, well done Python developers.

-- 
Chris Green
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: object-relational mappers

[Diez B. Roggisch]

Bruno Desthuilliers schrieb:
> Jarek Zgoda a écrit :
>> Bruno Desthuilliers napisał(a):
>>
>>> Now my own experience is that whenever I tried this approach for
>>> anything non-trivial, I ended up building an "ad-hoc,
>>> informally-specified bug-ridden slow implementation of half of "
>>> SQLAlchemy. Which BTW is not strictly an ORM, but primarily an attempt
>>> at a better integration of SQL into Python. So while it may feel like
>>> learning the inner complexities of SQLALchemy (or Django's ORM which is
>>> not that bad either) is "wasting brain cells", MVHO is that it's worth
>>> the time spent. But YMMV of course - IOW, do what works best for you.
>>
>> I like OR mappers, they save me lot of work. The problem is, all of them
>> are very resource hungry, processing resultset of 300k objects one by
>> one can effectively kill most of commodity systems. This is where raw
>> SQL comes in handy.
> 
> The problem here is not about how you build your query but about how you 
> retrieve your data. FWIW, SQLAlchemy provides quite a lot of "lower 
> level" SQL/Python integration that doesn't require the "object mapping" 
> part. "raw SQL" is fine, until you have to dynamically build complex 
> queries from user inputs and whatnot. This is where the "low-level" (ie: 
> non-ORM) part of SQLAlchemy shines IMHO.

The same can be said for SQLObjects SQLBuilder. Even if I ended up 
generating SQL for some query that didn't touch the ORM-layer, it helps 
tremendously to write e.g subqueries and such using python-objects 
instead of manipulating strings. They help keeping track of already 
referenced tables, spit out properly escaped syntax and so forth.

Diez
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Efficient way of testing for substring being one of a set?

[Ant]

On Apr 3, 2:27 pm, George Sakkis <[EMAIL PROTECTED]> wrote:
...
> It's even prettier in 2.5:
>
> any(word in name for word in words)
>
> George

And arguably the most readable yet!
-- 
http://mail.python.org/mailman/listinfo/python-list

Looking for Agile Python Consulting

[Ed Leafe]

This isn't a specific job request, which is why I'm not placing it on  
the Python Job Board. Rather, I'm in the process of gathering  
information for an upcoming project, and I need to determine what  
resources, if any, are available in the Python community.

We're in the planning stages of a fairly long-term project (1-2  
years), and the management here has been very impressed by the work of  
a consulting group used on another project in the company. They use  
Agile programming techniques, and have a lot of experience managing  
these sorts of projects. Unlike other projects with consultants,  
theirs have been very successful, and thus management is leaning  
towards using them for this one.

The problem, though, is that these consultants are a Rails shop, and  
it would mean changing a large part of our coding efforts to Ruby. My  
very strong preference is to stay with Python, so I'm trying to locate  
Python-centric consultants who use a similar disciplined Agile  
approach so that I can propose an alternative to the Rails solution.  
I'm not terribly concerned with Django vs. TurboGears vs. any other  
framework; at this point I'm only concerned with Python vs. Ruby.

If you are part of such a consulting group, or know of one that fits  
these requirements, please reply to me off-list.

-- Ed Leafe





Confidentiality Notice: This e-mail message (including any attached or
embedded documents) is intended for the exclusive and confidential use of the
individual or entity to which this message is addressed, and unless otherwise
expressly indicated, is confidential and privileged information of Rackspace. 
Any dissemination, distribution or copying of the enclosed material is 
prohibited.
If you receive this transmission in error, please notify us immediately by 
e-mail
at [EMAIL PROTECTED], and delete the original message. 
Your cooperation is appreciated.

-- 
http://mail.python.org/mailman/listinfo/python-list

mailbox.Maildir(), confusing documentation

[tinnews]

Having got my Python 2.5.2 installed I'm trying some things out with
the mailbox.Maildir() class.

If I do the following:-

import maibox
mailbox.Maildir("/home/isbd/Mail/Li/pytest")

then the pytest Maildir mailbox is created - which is great but isn't
documented.  If the above creates the maildir then what is the
mailbox.Maildir.add_folder() method for?  I tried
mailbox.Maildir.add_folder() and it appeared to do nothing, it didn't
produce any errors either.

Anyway I'm happy that mailbox.Maildir() will create maildirs and it
means I can do basically what I want but the documentation could be a
bit more helpful.  Are there any HowTos or FAQs for this (quite new I
know) part of Python?

-- 
Chris Green
-- 
http://mail.python.org/mailman/listinfo/python-list

MLE in Python for distributions

[Mike P]

Hi All,

Can anyone point me towards some code for Maximum Likilehood for
distribution in python?

Thanks

Mike
-- 
http://mail.python.org/mailman/listinfo/python-list

ANN: Leo 4.4.8 rc1 released

[Edward K Ream]

Leo 4.4.8 rc1 is now available at:
http://sourceforge.net/project/showfiles.php?group_id=3458&package_id=29106

This version features a new ipython plugin that provides a two-way bridge
between Leo and IPython.  See 
http://webpages.charter.net/edreamleo/IPythonBridge.html

Leo is a text editor, data organizer, project manager and much more. See:
http://webpages.charter.net/edreamleo/intro.html

The highlights of Leo 4.4.8:

- Leo's source code is now managed by bzr: see link below.
- Leo's discussion is now hosted by Google Groups: see link below.
- Arguments to g.es and g.es_print can be translated using gettext.
- Completed ILeo: a bridge between IPython and Leo.
  See http://webpages.charter.net/edreamleo/IPythonBridge.html
- Minibuffer commands may have arguments.
- @menu trees can now refer to commands created by
  @command and @button nodes.
- Added support for common @commands nodes in settings files.

Links:
--
Leo:  http://webpages.charter.net/edreamleo/front.html
Forum:http://groups.google.com/group/leo-editor
Download: http://sourceforge.net/project/showfiles.php?group_id=3458
Bzr:  http://code.launchpad.net/leo-editor/
Quotes:   http://webpages.charter.net/edreamleo/testimonials.html

Edward K. Ream   email:  [EMAIL PROTECTED]
Leo: http://webpages.charter.net/edreamleo/front.html



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: object-relational mappers

[Luis M . González]

Yes, webpy's db api can be used in stand-alone scripts if you want.
See below:

import web
db = web.database(dbn='mysql', db='northwind', user='root')
x = db.select('employees')
...

Another good thing is that, since queries return Storage objects
(similar to dictionaries), they are much more flexible.
Suppose that you get the results of a form sent via a POST method, and
you want to insert this data into your database.
You would simple write:

i = web.input()
db.insert('orders', **i)

So everything related to CRUD operations are is easy to do, without
having to mess with objects.
I think this sticks strictly to the KISS principle, keeping it simple,
with less overhead, less layers of abstraction and therefore, less
bugs and complications.
And it matchs perfectly webpy's philosofy for creating web apps.

Luis


On 3 abr, 11:06, Bruno Desthuilliers  wrote:
> Seems nice too in another way. Is that part independant of the rest of
> the framework ? If so, I'll have to give it a try at least for admin
> scripts.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: object-relational mappers

[Luis M . González]

On 3 abr, 11:06, Bruno Desthuilliers  wrote:
> Luis M. González a écrit :
>
> > I have come to the same conclusion.
> > ORMs make easy things easier, but difficult things impossible...
>
> Not my experience with SQLAlchemy. Ok, I still not had an occasion to
> test it against stored procedures, but when it comes to complex queries,
> it did the trick so far - and (warning: front-end developper
> considerations ahead) happened to be much more usable than raw strings
> to dynamically *build* the queries.
>
> > The best approach I've seen so far is webpy's (if we are talking of
> > web apps).
> > It isn't an ORM, it is just a way to make the database api easier to
> > use.
> > Queries don't return objects, they return something similar to
> > dictionaries, which can be used with dot notation ( for example,
> > result.name is equal to result['name'] ).
>
> > A simple select query would be db.select('customers') or
> > db.select('customers', name='John').
> > But you can also resort to plain sql as follows: db.query('select *
> > from customers where name = "John"').
>
> > Simple, effective and doesn't get in your way.
>
> Seems nice too in another way. Is that part independant of the rest of
> the framework ? If so, I'll have to give it a try at least for admin
> scripts.

Yes, webpy's db api can be used in stand-alone scripts if you want.
See below:

import web
db = web.database(dbn='mysql', db='northwind', user='root')
x = db.select('employees')

for i in x:
print i.FirstName, i.LastName
...

Another good thing is that, since queries return Storage objects
(similar to dictionaries), they are much more flexible.
Suppose that you get the results of a form sent via a POST method, and
you want to insert this data into your database.
You would simple write:

i = web.input()
db.insert('orders', **i)


So everything related to CRUD operations are is easy to do, without
having to mess with objects.
I think this sticks strictly to the KISS principle, keeping it simple,
with less overhead, less layers of abstraction and therefore, less
bugs and complications.
And it matchs perfectly webpy's philosofy for creating web apps.

Luis
-- 
http://mail.python.org/mailman/listinfo/python-list

PyXMPP and GTalk

[Ivan Chernetsky]

Hello!

Here is the example from PyXMPP package:
http://pyxmpp.jajcus.net/trac/browser/trunk/examples/echobot.py

And here is output:

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Directed Graph Traversal

[bukzor]

On Apr 2, 8:33 pm, Scott David Daniels <[EMAIL PROTECTED]> wrote:
> bukzor wrote:
> > Can someone point me in the direction of a good solution of this? I'm
> > using it to construct a SQL query compiler, 
> > Given a directed graph and a list of points in the graph, what is the
> > minimal subgraph that contains them all? It is preferable that the
> > subgraph is a tree.
>
> I did something nice (but non-redistributable) on this once: here is the
> driving intuition:
>
> *  Start with every point a distinct color.
> *  Add all points adjacent in the digraph as the same color; merge
> colors as you join them.
> *  When you are down to to a single color, you have the minimal solution
> in the set you've chosen.
>
> I actually did a cheapest-first search; adding an edge each time.
> There is a post-join pruning step that was (as I recall) fairly simple.
>
> -Scott David Daniels
> [EMAIL PROTECTED]

That sounds like a kind of iterative deepening search, which is what
I'm planning to do. Once I have it written up, I'll post for your
pythonic enjoyment.

--Buck
-- 
http://mail.python.org/mailman/listinfo/python-list

Application of "with" statement in py3k. Property defining/redefining.

[Zaur Shibzoukhov]

http://szport.blogspot.com/2008/04/application-of-with-statement-in-py3k.html
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Why prefer != over <> for Python 3.0?

[D'Arcy J.M. Cain]

On Thu, 03 Apr 2008 05:12:28 GMT
Tim Roberts <[EMAIL PROTECTED]> wrote:
> Yes, indeed.  In response to a challenge posted on one of the x86 assembler
> newsgroups about two years ago, one intrepid Russian programmer produced a
> generic Sudoku solver in a 65-byte executable.  Yes, that's 65 BYTES -- not
> KB, not MB.

Wow!  I would have thought that the header on most executible file
formats was bigger than that these days.  Or was this done as a .COM
file or something like that?

-- 
D'Arcy J.M. Cain <[EMAIL PROTECTED]> |  Democracy is three wolves
http://www.druid.net/darcy/|  and a sheep voting on
+1 416 425 1212 (DoD#0082)(eNTP)   |  what's for dinner.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange MySQL Problem...

[Victor Subervi]

On Thu, Apr 3, 2008 at 9:34 AM, Gabriel Genellina <[EMAIL PROTECTED]>
wrote:

> En Thu, 03 Apr 2008 09:43:57 -0300, Victor Subervi
> <[EMAIL PROTECTED]> escribió:
>
> >> Steve Holden wrote:
> >> Define "no longer works".
> > Sorry. Throws HTTP 200 error.
>
> HTTP 200 means OK.


Yes. But there is an error somewhere, because it doesn´t run all the code.
For example, I have to manually drop the table.


>
>
> >> > The same remarks I've posted earlier apply here.
> > Must have missed those. Yes, the code works fine. Repeatedly.
>
>
> http://groups.google.com/group/comp.lang.python/browse_thread/thread/b732eee4e91b1868/


Thank you. I believe you mean by bound, something like this, right?
binary(picdata)
I am now doing that, if I am not mistaken.


> display or *log* errors...


Yeah. There are always errors...


> *post*? This is the server response. Return either text *or* an image (the
> text may be an html referencing the image)


Yes, post. But what do I write instead of this?
print 'Content-Type: image/jpeg\r\n'


> Test it locally (just the database thing, no web), test the cgi script
> without database interaction, only then join the two.


Okay, but how do I upload an image into mysql without a script? And once I
can do that, everything´s (almost) solved!


> But you can read the server logs at least?


I am new to Plesk. I discovered I can read logs. More following...


>
> Your code is throwing exceptions but you're not seeing them. Use the cgitb
> module http://docs.python.org/lib/module-cgitb.html


Now here you give me another laugh :) I added this line to the script
import cgitb; cgitb.enable()
and suddenly I can take out these junk lines that were throwing the HTTP 200
error:
names = 'aramis', 'athos', 'porthos'
data = {}
for name in names:
  datum = list(name)
  datum.sort()
  data[name] = cPickle.dumps(datum, 1)
Is this some kind of magic wand? :)

Okay, where I am at now...

I have a script called start.py which has this code in it:

import cgitb; cgitb.enable()
import MySQLdb
import struct
import _mysql, sys
Lots of imports, just to try things out. Most not needed...

  col_names = ['id', 'name_en', 'name_es', 'name_it', 'category',
'title_en', 'title_es', 'title_it', \
'description_en', 'description_es', 'description_it', 'price',
'bedrooms', 'bathrooms', 'pic1', 'sort_factor']
Just notice the pic1 there...
Then I make a nice list of colnames like MySQL wants...

  col_names_with_commas = ''
  for name in col_names:
col_names_with_commas += name + ', '
  count = len(col_names_with_commas)-2
  col_names_with_commas =
col_names_with_commas[0:len(col_names_with_commas)-2]
All that I have checked, it is good...
Here is the sticky point:

for id in ids:
  for d in id:
print '\n'
cursor.execute('select ' + col_names_with_commas + ' from products
where id = ' + str(d) + ';')
col_fields = cursor.fetchall()
i = 0
x = 0
for field in col_fields[0]:
  if x == 0:
# This is the ID field
i += 1
check = 'check' + str(i)
print '\n'
  if x == 15:
print '', field, '\n'
  else:
print '', field, '\n'
  x += 1
I have to do both of those for statements. The sticky point is in the last
if statement. 15 is the pic1 field. Should I use binary(field) instead? It
does not like that.

I send the whole thing over to another script which has this:

import cgitb; cgitb.enable()
import MySQLdb
import cgi
import struct
Same old same old...
Here´s the sticky part...

  if what_do == 'insert':
data = ['', '', '', '', '', '', '', '', '', '', '', 0.0, '', '', '',
500]
the_form(data, what_do)
  elif what_do == 'update':
cursor.execute('select * from products where id=' + update_these[0] +
';')
data = cursor.fetchall()
the_form(data[0], what_do)
And then try to render...


1ra Foto Grande:


I show you both parts, because the Internal Server Error comes from the
second page, but the logged error points to the first page. Here is the log
record...

[Thu Apr 03 09:53:33 2008] [error] [client 190.166.0.65] PythonHandler
mod_python.cgihandler: File "/var/www/vhosts/
livestocksling.com/httpdocs/test/python/Shop/iud.py", line 210, in
the_form\n print '"', binary(data[14]), '"', referer:
http://livestocksling.com/test/python/Shop/start.py
[Thu Apr 03 09:53:33 2008] [error] [client 190.166.0.65] PythonHandler
mod_python.cgihandler: NameError: global name 'binary' is not defined,
referer: http://livestocksling.com/test/python/Shop/start.py
TIA,
Victor
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python in High School

[Stef Mientki]

>> Well I doubt it's the visual environment that makes it more easy,
>> color, shape and position can give some extra information though.
>> I think apriori domain knowledge and flattness of information are of far
>> more importance.
>> The first issue is covered quit well by Robolab / Labview,
>> but the second issue certainly is not.
>> I'm right now working on a Labview like editor in Python,
>> which does obey the demand for flatness of information.
>> The first results can be seen 
>> here:http://oase.uci.kun.nl/~mientki/data_www/pylab_works/pw_animations_sc...
>>
>> 
>> cheers,
>> Stef Mientki
>>
>> 
>>> And you are going to teach them Java?  Oh, please don't.  Let the
>>> colleges torture them.  :=)
>>>   
>
> What do you mean by flatness of information?
>
>   
What I mean is something like; all the information at a certain 
abstraction level is visible on one screen or one piece of paper,
and not is available through multiple screen / multiple right-clicks 
etc. A wizard in general is an example of strong non-flatness of  
information  (try adding a mail-account in Thunderbird, this could 
easily be put on 1 page, which clearly would give a much better overview).

cheers,
Stef


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python-by-example - new online guide to Python Standard Library

[AK]

AK wrote:
> Hello,
> 
> I find that I learn easier when I go from specific examples to a more 
> general explanation of function's utility and I made a reference guide 
> that will eventually document all functions, classes and methods in 
> Python's Standard Library. For now, I covered about 20 most important 
> modules. I will be adding more modules and eventually I'll cover 
> everything. Here's my progress so far, let me know if this is useful;
> I'll be glad to hear comments/suggestions/etc:
> 
> http://www.lightbird.net/py-by-example/
> 

I uploaded an updated site incorporating most of the suggestions I 
received and fixing some errors along the way. I will be adding more 
examples to modules that are already covered and will try to add more 
modules during the following week. Thanks again to all who posted advice 
and comments!

-- 
  -ak
   Tobu | http://www.lightbird.net/tobu/ | Freeform DB / Tagger / PIM
   Python-by-Example | http://www.lightbird.net/py-by-example/ | Guide 
to LibRef
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Get all strings matching given RegExp

[Paul McGuire]

On Apr 3, 4:50 am, Alex9968 <[EMAIL PROTECTED]> wrote:
> Can I get sequence of all strings that can match a given regular
> expression? For example, for expression '(a|b)|(x|y)' it would be ['ax',
> 'ay', 'bx', 'by']
>

Actually, this regex will only match 'a', 'b', 'x', or 'y' (assuming
you meant to bracket it with leading '^' and trailing '$').  To get
the set you listed, you would need to invert '(a|b)(x|y)'.

The thread that Gabriel Genellina referenced includes a link to a
pyparsing regex inverter (http://pyparsing-public.wikispaces.com/space/
showimage/invRegex.py), which I used to test your regex - I've added
both regexes in the test cases of that script.  Some of the other test
cases include regexes for all US time zones, and all chemical symbols.

-- Paul
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Module not found in script that was found in command-line interpreter. Possible Path issue?

[Trent Mick]

Jacob Davis wrote:
> I just installed the MySQLdb module and I have been able to get it to 
> run in my command line interpreter. 
> 
> I am running Mac Leopard, and Python 2.5.
> 
>  I have tested importing and actually connecting and using a MySQL 
> database, although it issues some warning:
> 
> SnakeBite:MySQL-python-1.2.2 Snake$ python
> Python 2.5 (r25:51918, Sep 19 2006, 08:49:13) 
> [GCC 4.0.1 (Apple Computer, Inc. build 5341)] on darwin
> Type "help", "copyright", "credits" or "license" for more information.
 import MySQLdb
> /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/MySQL_python-1.2.2-py2.5-macosx-10.3-fat.egg/_mysql.py:3:
>  
> UserWarning: Module _mysql was already imported from 

 From that message it looks like this "python" is /usr/local/bin/python 
(i.e. a separate installation than Apple's system python at 
/usr/bin/python and /System/Library/Frameworks/Python.framework).

You can tell for sure by doing:

   $ which python

> However, while writing a .py script (with Komodo Edit) I try to simply 
> import the module and the in-Komodo interpreter returns an error:
> Traceback (most recent call last):
>   File 
> "/Users/Snake/Documents/NPS/Thesis/Source_Code/Genetics/mysql_connect_test.py",
>  
> line 11, in 
> import MySQLdb
> ImportError: No module named MySQLdb

I suspect that this is because your run of Komodo Edit doesn't have 
"/usr/local/bin" on its PATH and is using "/usr/bin/python" instead of 
the one you typically use on the command line.

You can configure Komodo to know about /usr/local/bin by adding a "PATH" 
setting in the "Environment" prefs panel. Arguably Komodo should just 
add /usr/local/bin to its runtime PATH by default, but unfortunately it 
currently doesn't. Komodo doesn't pick up your normal bash shell 
environment because of problems trying to get that information in general.

Please let me know (or on the komodo-discuss list [^1] or Komodo bug 
database [^2]) if you have any problems getting that going.

Cheers,
Trent

[1]: http://listserv.activestate.com/mailman/listinfo/Komodo-discuss
[2]: http://bugs.activestate.com/query.cgi?product=Komodo

-- 
Trent Mick
trentm at activestate.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python in High School

[marion]

On Apr 1, 1:27 pm, sprad <[EMAIL PROTECTED]> wrote:
> I'm a high school computer teacher, and I'm starting a series of
> programming courses next year (disguised as "game development" classes
> to capture more interest). The first year will be a gentle
> introduction to programming, leading to two more years of advanced
> topics.
>
> I was initially thinking about doing the first year in Flash/
> ActionScript, and the later years in Java. My reasoning is that Flash
> has the advantage of giving a quick payoff to keep the students
> interested while I sneak in some OOP concepts through ActionScript.
> Once they've gotten a decent grounding there, they move on to Java for
> some more heavy-duty programming.
>
> I've never used Python, but I keep hearing enough good stuff about it
> to make me curious.
>
> So -- would Python be a good fit for these classes? Could it equal
> Java as the later heavy-duty language? Does it have enough quickly-
> accessible sparklies to unseat Flash?
>
> I want to believe. Evangelize away.

I am a strong support of teaching programming in middle and high
school. Kids have the potential of being more than just passive
consumers of other programmers work. That is best illustrated by the
growth of game mod'rs writing their own levels for computer games.

I think I agree with all of the positive, supporting posts about
Python. I would just like to add that Python (and PyGame) are open
source and so your students can download it at home and have fun
exploring it on their own time (at their own pace). I think that is a
real positive.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange MySQL Problem...

[Gabriel Genellina]

En Thu, 03 Apr 2008 14:03:53 -0300, Victor Subervi  
<[EMAIL PROTECTED]> escribió:

> On Thu, Apr 3, 2008 at 9:34 AM, Gabriel Genellina  
> <[EMAIL PROTECTED]>
> wrote:
>
> Thank you. I believe you mean by bound, something like this, right?
> binary(picdata)
> I am now doing that, if I am not mistaken.

No, I mean execute(sqltext, arguments) instead of  
execute(sqltext_with_values_inserted_by_hand)

>> *post*? This is the server response. Return either text *or* an image  
>> (the
>> text may be an html referencing the image)
>
> Yes, post. But what do I write instead of this?
> print 'Content-Type: image/jpeg\r\n'

Perhaps you should read a little about how HTTP works.
Very shortly, the client (the browser, on behalf of the user) sends an  
HTTP Request (usually a GET request, sometimes POST). The server receives  
it, processes it and returns an HTTP Response. Whatever you 'print' in  
your CGI script is part of the Response, a response isn't a POST.
If the response is an HTML page, the Content-Type should say text/html,  
NOT image/jpeg.

>> Test it locally (just the database thing, no web), test the cgi script
>> without database interaction, only then join the two.
>
> Okay, but how do I upload an image into mysql without a script? And once  
> I can do that, everything´s (almost) solved!

I insist: test each step separately. First locally, later merge all steps.  
One doesn't "upload an image into mysql": one uploads an image, and some  
script on the server saves the image into mysql database.

>> But you can read the server logs at least?
> I am new to Plesk. I discovered I can read logs. More following...

Good thing!

>> Your code is throwing exceptions but you're not seeing them. Use the  
>> cgitb
>> module http://docs.python.org/lib/module-cgitb.html
>
> Now here you give me another laugh :) I added this line to the script
> import cgitb; cgitb.enable()
> and suddenly I can take out these junk lines that were throwing the HTTP  
> 200
> error:

Please stop refering to it as HTTP 200 error: it is *not* an error!

> I have a script called start.py which has this code in it:
>
>   col_names = ['id', 'name_en', 'name_es', 'name_it', 'category',
> 'title_en', 'title_es', 'title_it', \
> 'description_en', 'description_es', 'description_it', 'price',
> 'bedrooms', 'bathrooms', 'pic1', 'sort_factor']
> Just notice the pic1 there...
> Then I make a nice list of colnames like MySQL wants...
>
>   col_names_with_commas = ''
>   for name in col_names:
> col_names_with_commas += name + ', '
>   count = len(col_names_with_commas)-2
>   col_names_with_commas =
> col_names_with_commas[0:len(col_names_with_commas)-2]

The above code is a long and convoluted way to write:

 col_names_with_commas = ', '.join(col_names)

> for id in ids:
>   for d in id:
> print '\n'
> cursor.execute('select ' + col_names_with_commas + ' from  
> products
> where id = ' + str(d) + ';')

Remember to use parameters:
   cursor.execute('select ' + col_names_with_commas +
  ' from products '
  ' where id = %s;', (d,))

All this looks like the kind of spaghetti code common in cgi scripts  
around 10 years ago, mixing application logic, storage and presentation  
all at once.
At least try to separate database operations from html generation: make  
your queries, build some data structure, and generate the HTML using that  
structure as parameters.
There are many templating engines to choose for:  
http://wiki.python.org/moin/Templating

>   if x == 15:
> print '', field, '\n'
>   else:
> print '', field, '\n'
>   x += 1
> I have to do both of those for statements. The sticky point is in the  
> last
> if statement. 15 is the pic1 field. Should I use binary(field) instead?  
> It
> does not like that.

Binary is part of the DBAPI interface (you should read PEP 249 [1]). It  
has absolutely nothing to do with html pages.

> mod_python.cgihandler: NameError: global name 'binary' is not defined,

Python is case sensitive... Use MySQLdb.Binary instead. See [1] and the  
MySQLdb documentation.
I insist again: this would have been a lot easier to catch if you test  
locally; this error has nothing to do with the web part of the application.

[1] http://www.python.org/dev/peps/pep-0249/

-- 
Gabriel Genellina

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: splitting an XML file on the basis on basis of XML tags

[Diez B. Roggisch]

> I abuse it because I can (and because I don't generally work with XML
> files larger than 20-30meg) :)
> And the OP never said the XML file for 1TB in size, which makes things
> different.

Even with small xml-files your advice was not very sound. Yes, it's 
tempting to use regexes to process xml. But usually one falls flat on 
his face soon - because of whitespace or attribute order or  
versus  or .. or .. or.

Use an XML-parser. That's what they are for. And especially with the 
pythonic ones like element-tree (and the compatible lxml), its even more 
straight-forward than using rexes.


Diez
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python for low-level Windows programming

[Troels Thomsen]

>>
>> Is this possible without too much pain? I know I can code it with C#
>> or C++ but tha'ts a road to avoid, if possible.
>
> Well of course I don't know exactly what you'd need, but the
> answer's certainly Yes :)

Need I mention that it is also easy to write a windows service, using 
pyservice (i think ...) and ½ a page of python source

tpt

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Manipulate Large Binary Files

[Derek Tracy]

On Apr 3, 2008, at 3:03 AM, Paul Rubin <"http:// 
phr.cx"@NOSPAM.invalid> wrote:

> Derek Martin <[EMAIL PROTECTED]> writes:
>>> Both are clocking in at the same time (1m 5sec for 2.6Gb), are there
>>> any ways I can optimize either solution?
>
> Getting 40+ MB/sec through a file system is pretty impressive.
> Sounds like a RAID?
>
>> That said, due to normal I/O generally involving double-buffering,  
>> you
>> might be able to speed things up noticably by using Memory-Mapped I/O
>> (MMIO).  It depends on whether or not the implementation of the  
>> Python
>> things you're using already use MMIO under the hood, and whether or
>> not MMIO happens to be broken in your OS. :)
>
> Python has the mmap module and I use it sometimes, but it's not
> necessarily the right thing for something like this.  Each page you
> try to read from results in own delay while the resulting page fault
> is serviced, so any overlapped i/o you get comes from the OS being
> nice enough to do some predictive readahead for you on sequential
> access if it does that.  By coincidence there are a couple other
> threads mentioning AIO which is a somewhat more powerful mechanism.
>
> -- 
> http://mail.python.org/mailman/listinfo/python-list

I am running it on a RAID(stiped raid 5 using fibre channel), but I  
was expecting better performance.

I will have to check into AIO, thanks for the bone.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Manipulate Large Binary Files

[Derek Tracy]

---
Derek Tracy
[EMAIL PROTECTED]
---

On Apr 3, 2008, at 3:03 AM, Paul Rubin <"http:// 
phr.cx"@NOSPAM.invalid> wrote:

> Derek Martin <[EMAIL PROTECTED]> writes:
>>> Both are clocking in at the same time (1m 5sec for 2.6Gb), are there
>>> any ways I can optimize either solution?
>
> Getting 40+ MB/sec through a file system is pretty impressive.
> Sounds like a RAID?
>
>> That said, due to normal I/O generally involving double-buffering,  
>> you
>> might be able to speed things up noticably by using Memory-Mapped I/O
>> (MMIO).  It depends on whether or not the implementation of the  
>> Python
>> things you're using already use MMIO under the hood, and whether or
>> not MMIO happens to be broken in your OS. :)
>
> Python has the mmap module and I use it sometimes, but it's not
> necessarily the right thing for something like this.  Each page you
> try to read from results in own delay while the resulting page fault
> is serviced, so any overlapped i/o you get comes from the OS being
> nice enough to do some predictive readahead for you on sequential
> access if it does that.  By coincidence there are a couple other
> threads mentioning AIO which is a somewhat more powerful mechanism.
>
> -- 
> http://mail.python.org/mailman/listinfo/python-list
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: State of ctypes Support on HP-UX?

[Thomas Heller]

Phil Thompson schrieb:
> On Thursday 03 April 2008, Thomas Heller wrote:
>> Phil Thompson schrieb:
>> > Could somebody confirm how well ctypes is supported on HP-UX (for both
>> > PA-RISC and Itanium) for both Python v2.4 and v2.5?

>> I cannot answer your question, but if you want to try it out
>> yourself there is the HP testdrive program: http://www.testdrive.hp.com/
> 
> Thanks for the pointer. Unfortunately the answer is that there is no support 
> (at least for ctypes v1.0.2).

I tried out the current SVN version of Python myself.  ctypes doesn't compile
on the PA system (the libffi assembler code fails to compile),  but I did get
it to work on the Itanium system with gcc (I had to set LDSHARED="gcc -shared"
before configuring).  Even the ctypes unittests pass on this system.

In theory, the ctypes code should be backwards-compatible with python 2.4, 
although
in practice it currently is not, but IMO it should be possible to change it 
accordingly.

Would this be useful to you?

Thomas

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python in High School

[ajaksu]

On Apr 2, 5:01 pm, John Henry <[EMAIL PROTECTED]> wrote:
> However, once I start teaching him variables, expressions, loops, and
> what not, I found that (by surprise) he had great difficulties
> catching on.  Not soon after that, we had to quit.

This makes me curious: how much of videogamer are you? And your son?

I ask that because when I think about teaching programming to young
kids, I imagine using terms they know from gaming, like "save
slots" (variables/names), "memory cards" (containers),
"combos" (functions, loops), "life meters" (counters), "next
level" (conditionals, iteration, loops), "teammates" (helper
functions), "character classes" and "characters" (class and
instances), "confirm/cancel" (conditionals), etc.

But I've never really tried to put all those together and find a test
subject, so I'd like to know how fluent in this lingo you both were so
I can assess my pseudo-didatic approach by proxy :)

Regards,
Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Efficient way of testing for substring being one of a set?

[Pop User]

[EMAIL PROTECTED] wrote:
 > Dennis Benzinger:
 >> You could use the Aho-Corasick algorithm > Aho-Corasick_algorithm>.
 >> I don't know if there's a Python implementation yet.
 >
 > http://hkn.eecs.berkeley.edu/~dyoo/python/ahocorasick/
 >

http://nicolas.lehuen.com/download/pytst/ can do it as well.
-- 
http://mail.python.org/mailman/listinfo/python-list

python bisect questions

[ankitks . mital]

I am week on functional programming, and having hard time
understanding this:

class myPriorityQueue:
  def __init__(self, f=lamda x:x):
  self.A = []
  self.f = f

  def append(self, item)
  bisect.insort(self.A, (self.f(item), item))


now I know we are inserting items(user defined type objects) in list A
base on sorting order provided by function A.
but what I don't understand is bisect command
what does bisect.insort(self.A, (self.f(item), item)) doing

isn't it is returning truple of (self.f(item), item)).
why it is not
biset.insort(self.A, item)
A.sort(f)

thanks for your comments
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pexpect question.

[Jorgen Grahn]

On Wed, 02 Apr 2008 19:57:31 -0600, Paul Lemelle <[EMAIL PROTECTED]> wrote:
> Jorgen,
>
> Thanks for your reply. 
>
> The ssh function is just a small part of what I would like to
> accomplish.  And yes, chk is undefined, I was trying to figure out why
> control was not being returned from the sshcon funciton. 

OK. Hope my note helped.

> I looked for pexpect doucment on http://www.noah.org/wiki/Pexpect, but
> the documentaiton link appear to be broken. Could you recommend
> another site?

As I recall it, the online documentation is just the documentation
from the Python module itself, HTML-formatted.  You can simply type
"pydoc pexpect" and get the reference manual. I think it's pretty
good.

/Jorgen

-- 
  // Jorgen Grahn   R'lyeh wgah'nagl fhtagn!
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: function call - default value & collecting arguments

[Primoz Skale]

"OKB (not okblacke)" <[EMAIL PROTECTED]> wrote in message 
news:[EMAIL PROTECTED]
> Primoz Skale wrote:
>
>> OK, everything allright till so fair. But! :) Now define third
>> function as:
>>
>> def f(*a):
>>print a[0]
>>
>> In this case, function only prints first argument in the tuple:
>>
f(1,2,3)
>> 1
f(3)
>> 3
f()#no arguments passed
>> Traceback (most recent call last):
>>   File "", line 1, in 
>> f() #no arguments passed
>>   File "", line 2, in f
>> print a[0]
>> IndexError: tuple index out of range
>>
>> Then I tried to define the function as:
>>
>> def f(*a=(0,)):
>>   print a[0]  #this should come next, but we get error msg instead,
>>   saying
>>
>>   SyntaxError: invalid syntax
>>
>> but it does not work this way. Now my 'newbie' question: Why not?
>> :) I wanted to write function in this way, because then we can call
>> function without any arguments, and it would still print 0 (in this
>> case).
>>
>> What I wanted was something like this:
>>
>> def f(*a=(0,)):
>>   print a[0]
>>
f(1,2)
>> 1
f()  #no args passed 0
>
>When you use the *a syntax, you're saying that you want a to
> contain a tuple of all the arguments.  When you try *a=(0,), you seem to
> be saying that you want a to hold all the arguments, or, if there are
> none, you want to pretend that it was called with one argument, namely
> 0.
>
>Well, if you want to ensure that there is at least one argument,
> and print that first one, and make it zero if it's not supplied, why are
> you using the *a syntax?  You're clearly treating that first argument
> distinctly, since you want to apply a default value to it but not to any
> others.  Just do this:
>
> def f(a, *b):
>print a


Thanks! I never thought of that :)

P.


>
> -- 
> --OKB (not okblacke)
> Brendan Barnwell
> "Do not follow where the path may lead.  Go, instead, where there is
> no path, and leave a trail."
> --author unknown 


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: object-relational mappers

[M.-A. Lemburg]

On 2008-04-01 22:40, Aaron Watters wrote:
> I've been poking around the world of object-relational
> mappers and it inspired me to coin a corellary to the
> the famous quote on regular expressions:
> 
> "You have objects and a database: that's 2 problems.
> So: get an object-relational mapper:
> now you have 2**3 problems."
> 
> That is to say I feel that they all make me learn
> so much about the internals and features of the
> O-R mapper itself that I would be better off rolling
> my own queries on an as-needed basis without
> wasting so many brain cells.
> 
> comments?

I fully agree :-)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Apr 03 2008)
 >>> Python/Zope Consulting and Support ...http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/


 Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! 


eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
Registered at Amtsgericht Duesseldorf: HRB 46611
-- 
http://mail.python.org/mailman/listinfo/python-list