Freeze problem with Regular Expression

[Kirk]

Hi All,
the following regular expression matching seems to enter in a infinite 
loop:


import re
text = ' MSX INTERNATIONAL HOLDINGS ITALIA srl (di seguito MSX ITALIA) 
una '
re.findall('[^A-Z|0-9]*((?:[0-9]*[A-Z]+[0-9|a-z|\-]*)+\s*[a-z]*\s*(?:[0-9]
*[A-Z]+[0-9|a-z|\-]*\s*)*)([^A-Z]*)$', text)
#

No problem with perl with the same expression:

#
$s = ' MSX INTERNATIONAL HOLDINGS ITALIA srl (di seguito MSX ITALIA) una 
';
$s =~ /[^A-Z|0-9]*((?:[0-9]*[A-Z]+[0-9|a-z|\-]*)+\s*[a-z]*\s*(?:[0-9]*[A-
Z]+[0-9|a-z|\-]*\s*)*)([^A-Z]*)$/;
print $1;
#

I've python 2.5.2 on Ubuntu 8.04.
any idea?
Thanks!

-- 
Kirk
--
http://mail.python.org/mailman/listinfo/python-list

Re: Freeze problem with Regular Expression

[Kirk]

On Wed, 25 Jun 2008 15:29:38 -0700, John Machin wrote:

> Several problems:

Ciao John (and All partecipating in this thread),
first of all I'm sorry for the delay but I was out for business.

> (1) lose the vertical bars (as advised by others) (2) ALWAYS use a raw
> string for regexes; your \s* will match on lower- case 's', not on
> spaces

right! thanks!

> (3) why are you using findall on a pattern that ends in "$"? 

Yes, you are right, I started with a different need and then it changed 
over time...

> (6) as remarked by others, you haven't said what you are trying to do;

I reply here to all of you about such point: that's not important, 
although I appreciate very much your suggestions!
My point was 'something that works in Perl, has problems in Python'.
In respect to this, I thank Peter for his analysis.
Probably Perl has a different pattern matching algorithm.

Thanks again to all of you!

Bye!

-- 
Kirk
--
http://mail.python.org/mailman/listinfo/python-list

Re: Freeze problem with Regular Expression

[Kirk]

On Mon, 30 Jun 2008 13:43:22 -0700, John Machin wrote:

>> I reply here to all of you about such point: that's not important,
>> although I appreciate very much your suggestions! My point was
>> 'something that works in Perl, has problems in Python'.
> 
> It *is* important; our point was 'you didn't define "works", and it was

ok...

> near-impossible (without transcribing your regex into verbose mode) to
> guess at what you suppose it might do sometimes'.

fine: it's supposed to terminate! :-)

Do you think that hanging is an *admissible* behavior? Couldn't we learn 
something from Perl implementation?

This is my point.

Bye

-- 
Kirk
--
http://mail.python.org/mailman/listinfo/python-list

Pickling and inheritance are making me hurt

[Kirk Strauser]

I have a module that defines a Search class and a SearchResult class.  I use
these classes by writing other modules that subclass both of them as needed
to interface with particular search engines.

My problem is that Search defines a method (called automatically by __del__)
to save its results between invocations:

def _saveresults(self):
self._oldresults = self._results
file = open(self._storefile(), 'w')
pickle.dump(self._oldresults, file)
file.close()

The problem I'm having is the the pickle.dump call is failing whenever the
objects in "self.data" are instances of derivatives of SearchResult rather
than instances of SearchResult itself (which is pretty much always the
case):

Exception pickle.PicklingError:  in > ignored


Now, if I overload _saveresults inside a subclass of Search, then it works. 
It seems like the problem is that _saveresults is only looking inside the
same namespace as Search (where it's originally defined), even if it's
actually been inherited by another class in a different module.  Is there a
way around this?

-- 
Kirk Strauser
-- 
http://mail.python.org/mailman/listinfo/python-list

Python to do CDC on XML files

[Bruce Kirk]

Does anyone know of any existing projects on how to generate a change data 
capture on 2 very large xml files.

The xml structures are the same, it is the data within the files that may 
differ.

I need to take a XML file from yesterday and compare it to the XML file 
produced today and not which XML records have changed.

I have done a google search and I am not able to find much on the subject other 
than software vendors trying to sell me their products. :-)

Regards
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python to do CDC on XML files

[Bruce Kirk]

I agree, the challenge is the volume of the data to compare is 13. Million 
records. So it needs to be very fast

Sent from my iPad

> On Mar 23, 2016, at 4:47 PM, Bob Gailer  wrote:
> 
> 
> On Mar 23, 2016 4:20 PM, "Bruce Kirk"  wrote:
> >
> > Does anyone know of any existing projects on how to generate a change data 
> > capture on 2 very large xml files.
> >
> > The xml structures are the same, it is the data within the files that may 
> > differ.
> >
> It should not be too difficult to write a program that locates the tags 
> delimiting each record, then compare them.
-- 
https://mail.python.org/mailman/listinfo/python-list

Python article in Free Software Magazine

[Kirk Strauser]

I wrote this article which was published in Free Software Magazine:

http://www.freesoftwaremagazine.com/free_issues/issue_09/intro_zope_1/

It's intended as a high-level overview of the language, and therefore
glosses over some of the details.  For example, I describe its function
calling mechanism as pass-by-reference, because that's close enough for
newcomers to get the gist of it.

Anyway, the article's available under an open license.  If you like it, feel
free to pass it around.  Enjoy!
-- 
Kirk Strauser

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python article in Free Software Magazine

[Kirk Strauser]

On Sunday 01 January 2006 01:06 am, Steven D'Aprano wrote:

> I don't want to nit-pick all my way through the article,

There's nothing wrong with that, and if I hadn't been prepared for it, I
wouldn't have posted the link in here.

You have fair points.  Unfortunately, though, the word length of the article
just didn't provide enough space to go into the level of detail those
subjects would have required.  I tried to compromise by giving the answer
that most closely fit the situation without being exactly correct.

By the way, the author style guides say to write for an audience with
technical background.  That article was meant for you, not your boss.  :-)
-- 
Kirk Strauser

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python article in Free Software Magazine

[Kirk Strauser]

Terry Hancock wrote:

> I find that it's not difficult to explain Python object handling if you
> simply insist on the concept of "name binding" instead of "variable 
> assignment":  

That was a pretty good explanation.  I'll probably use that next time.
 
> Makes for a nice figure, too, which Free Software Magazine needs more of.

?
-- 
Kirk Strauser
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python article in Free Software Magazine

[Kirk Strauser]

Michele Simionato wrote:

> when I think Zope is the less Pythonic application I have ever seen;)

You do?  Why so?  I'm not arguing, but that's different than my experience
with it and I'm curious about how you reached that conclusion.
-- 
Kirk Strauser
-- 
http://mail.python.org/mailman/listinfo/python-list

Finding the name of a class

[Kirk Strauser]

Given a class:

>>> class foo(object):
>>> pass

how can I find its name, such as:

>>> b = foo
>>> print something(b)
'foo'

I'm writing a trace() decorator for the sake of practice, and am trying to
print the name of the class that a traced method belongs to.  This seems
like it should be easy, but I think I've been staring at the problem too
long.
-- 
Kirk Strauser
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Finding the name of a class

[Kirk Strauser]

Larry Bates wrote:

> print print b.__class__.__name__  gives what you want

That doesn't seem to do it, though.  Here's the result of importing a module
from my company's internally-developed library:

>>> from Daycos.TableCopier.copyfro import StateProcessor
>>> print StateProcessor.__class__.__name__
type

I'm looking for something that would print 'StateProcessor' but am not
having much luck.
-- 
Kirk Strauser
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Finding the name of a class

[Kirk Strauser]

Bruno Desthuilliers wrote:

> Kirk Strauser wrote:

>>>>> class foo(object):
>>>>> pass
>> 
>> how can I find its name, such as:
>> 
>>>>> b = foo

> I suppose you mean b = foo() ?

Actually, I meant 'b = foo' in this case - I want to find the name of the
class that b references, but the name of an instance (which may have zero
or many references to it).

> The name of a class is in the attribute '__name__' of the class. The
> class of an object is in the attribute '__class__' of the object.

I swear that didn't work earlier.  Honest.  :-)

OK, now for the good stuff.  In the code below, how can I find the name of
the class that 'bar' belongs to:

>>> class Foo(object):
... def bar(self):
... pass
...
>>> b = Foo.bar
>>> dir(b)
['__call__', '__class__', '__cmp__', '__delattr__', '__doc__', '__get__', 
'__getattribute__', '__hash__', '__init__', '__new__', '__reduce__', 
'__reduce_ex__', '__repr__', '__setattr__', '__str__', 'im_class', 'im_func', 
'im_self']
>>> b.__class__

>>> b.__class__.__name__
'instancemethod'

I was sort of hoping that it would print 'Foo'.
-- 
Kirk Strauser
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Question on try/except

[Kirk McDonald]

Dan wrote:
> While perusing sre.py in the standard library, I came upon this snippet 
> of code in the _compile() function definition:
> 
> try:
> p = sre_compile.compile(pattern, flags)
> except error, v:
> raise error, v # invalid expression
> 
> Is there some particular use in catching an exception and immediately 
> re-raising it?  Why catch it at all?
> 
> /Dan
> 

All I can think of is that it changes the traceback to point to the 
re-raise and not the original raise.

-Kirk McDonald
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: what is the keyword "is" for?

[Kirk McDonald]

daniel wrote:
> I'm so confused by the keyword "is" and "==" equal sign, it seems they
> could be exchanged in some contexts, but not in others, what's the
> difference between them in terms of comparation?
> 
> thanks...
> 
> daniel
> 

  'is' compares object identity. == compares values.

 >>> a = [1, 2, 3]
 >>> b = [1, 2, 3]
 >>> a is b
False
 >>> a == b
True

In this example, a and b refer to two separate lists that happen to hold 
the same values. Thus, 'is' returns False, and == returns True. On the 
other hand:

 >>> c = d = [4, 5, 6]
 >>> c is d
True
 >>> c == d
True

Here, c and d refer to the same list. Therefore, 'is' returns true (they 
both refer to the same object), as does == (an object is always equal to 
itself, unless you overload the equality check in a weird way).

The distinction can easily be seen if we try to mutate these lists:

 >>> a.append(4)
 >>> a is b
False
 >>> a == b
False
 >>> c.append(7)
 >>> c is d
True
 >>> c == d
True

When we mutate a, b is not affected. They are two different lists, and 
changing 'a' makes it so they are no longer equal.

When we mutate c, d IS affected; they refer to the same list.

You can easily confuse yourself if you ever talk about applying 'is' to 
(for example) integers. Python may re-use certain small integers when 
you might not expect it to; this is done in the interests of efficiency. 
If you only compare the /values/ of numbers (with ==), then you will 
never notice this.

 >>> a = 1
 >>> b = 1
 >>> c = 100
 >>> d = 100
 >>> a is b
True
 >>> c is d
False

-Kirk McDonald
-- 
http://mail.python.org/mailman/listinfo/python-list

TypeCheck vs IsInstance in C API

[Kirk McDonald]

I'm examining the C API, and I have a question.

http://docs.python.org/api/object.html

There are two functions that appear to do nearly same thing, and I just 
want to be certain I'm not missing something. First is PyObject_IsInstance:

int PyObject_IsInstance(PyObject *inst, PyObject *cls);
Returns 1 if inst is an instance of the class cls or a subclass of cls, 
or 0 if not...

Second is PyObject_TypeCheck:

int PyObject_TypeCheck(PyObject *o, PyTypeObject *type);
Return true if the object o is of type type or a subtype of type...

Now, I can see that IsInstance can take a tuple as the second argument 
and check the type of the first argument against every item in the 
tuple. I also see that TypeCheck was added in version 2.2. Why was it 
added? Its functionality already seems covered by IsInstance. Is it a 
new-style vs. old-style class thing?

-Kirk McDonald
-- 
http://mail.python.org/mailman/listinfo/python-list

API functions not working as expected

[Kirk McDonald]

... for reasons that are obvious in retrospect. Specifically, I am 
talking about the PyNumber_InPlace* family of functions. For example, 
the docs for InPlaceAdd say:

PyObject* PyNumber_InPlaceAdd(PyObject *o1, PyObject *o2)
 Return value: New reference.
 Returns the result of adding o1 and o2, or NULL on failure. The 
operation is done in-place when o1 supports it. This is the equivalent 
of the Python statement "o1 += o2".

But, of course, numbers are immutable. None of them support in-place 
addition. This is not the same as o1 += o2, as o1 is not actually 
changed when using this function.

Am I missing something here? Is there, in fact, no point to these 
InPlace* functions?

-Kirk McDonald
-- 
http://mail.python.org/mailman/listinfo/python-list

Turning a callback function into a generator

[Kirk McDonald]

Let's say I have a function that takes a callback function as a 
parameter, and uses it to describe an iteration:

def func(callback):
 for i in [1, 2, 3, 4, 5]:
 callback(i)

For the sake of argument, assume the iteration is something more 
interesting than this which relies on the callback mechanism. The 
function is an existing interface, and I cannot change it.

I want to somehow, in some way, provide an iteration interface to this 
function. Thoughts?

-Kirk McDonald
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Turning a callback function into a generator

[Kirk McDonald]

[EMAIL PROTECTED] wrote:
> Peter Otten wrote:
> 
>>Kirk McDonald wrote:
>>
>>
>>>Let's say I have a function that takes a callback function as a
>>>parameter, and uses it to describe an iteration:
>>>
>>>def func(callback):
>>> for i in [1, 2, 3, 4, 5]:
>>> callback(i)
>>>
> 
> 
> Which object is immutable? the callback or the function?  If its the
> callback then
> 
> def func(callback):
>for i in numbers:
>   yield callback(i)
> 
> If the function is immutable, then its a bit harder.  The callback has
> to be able to do the processing.  You can't use an iterator here
> because call stack gets in the way.  You could store the information
> being passed to the callback in a list, then iterate over the list
> afterwards.  Or you could have the callback be able to handle all of
> the work at once.
> 
> What do you intend to use this for?  Python has a lot of options and
> you may not be using the best one for the problem
> 

It is the function that is immutable.

I am writing a library for the D programming language that is not 
totally unlike Boost.Python:

http://dsource.org/projects/pyd/wiki

It is still in the fairly early stages, although the basic function and 
class wrapping do work.

This particular functionality is required to wrap D's basic iteration 
protocol, the opApply function:

http://www.digitalmars.com/d/statement.html#foreach

opApply works using a callback, as described. Because D does not (yet) 
have anything like Python's "yield", wrapping it with Python's iteration 
interface is turning out to be dreadfully annoying.

-Kirk McDonald
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Turning a callback function into a generator

[Kirk McDonald]

Alex Martelli wrote:
> Lawrence D'Oliveiro <[EMAIL PROTECTED]> wrote:
> 
> 
>>In article <[EMAIL PROTECTED]>,
>> Kirk McDonald <[EMAIL PROTECTED]> wrote:
>>
>>
>>>I want to somehow, in some way, provide an iteration interface to this
>>>function. Thoughts?
>>
>>Run it in a separate thread/process?
> 
> 
> Sounds best to me.  Specifically, given (e.g.) something like the OP's
> 
> def func(callback):
>  for i in [1, 2, 3, 4, 5]:
>  callback(i)
> 
> we might have a wrapper such as:
> 
> import thread
> import Queue
> 
> def wrap_cb_into_gen(func):
> q = Queue.Queue(1)   # maximum size of 1
> def callback(item):
> q.put(item)
> all_done_sentinel = object()
> def thread_skeleton():
> func(callback)
> q.put(all_done_sentinel)
> thread.start_new_thread(thread_skeleton, ())
> while True:
> item = q.get()
> if item is all_done_sentinel: break
> yield item
> 
> 
> Of course, there are lighter-weight options than a length-1 Queue for
> the purpose of synchronizing these two threads, but I always tend to
> prefer Queue (for its simplicity, generality, solidity) to other
> synchronization structures and architectures, unless there is some very
> strong reason to do otherwise in a specific case.  Also, I normally use
> module threading rather than the lower-level module thread -- here,
> clearly, either one will do just fine.
> 
> 
> Alex

Threads are probably a weak point of mine. I have little experience with 
them, and so I am inclined to paranoia while using them. What 
implications does this have for "func"? In my case, it is a function in 
an extension module. Does it have to be thread safe? If it isn't, what 
limitations does this impose on the use of this wrapper?

All that aside, I read the previous threads but it took until just now 
to understand how this works. :-) I'll probably have a go at 
implementing this in D, as that will be more convenient for the library...

-Kirk McDonald
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Augument assignment versus regular assignment

[Kirk McDonald]

nagy wrote:
> I do the following. First create lists x,y,z. Then add an element to x
> using the augumented assignment operator. This causes all the other
> lists to be changed also.
> But if I use the assignment x=x+[4] instead of using the augumented
> assignment, the y and z lists do not change.
> Why is that?
> This does not happen when I work with integer data type for example, as
> shown below.
> 
> Thanks for your help
> Nagy
> 
> 
x=y=z=[]

In this example, the '[]' creates a new list object. x, y, and z are all 
set to reference that object.

x+=[2]

This does an "in-place" operation on that list, modifying (or 
"mutating") the object directly.

x
> 

> [2]
> 
y
> 
> [2]
> 
z
> 
> [2]
> 
x=x+[4]

This creates a new list that is the concatenation of the list created 
above (the list [2]) with a new list (the list [4]). This brand new list 
is bound to the name 'x'. The names 'y' and 'z' are left unchanged. That 
is, they still point to the original list.


x
> 
> [2, 4]
> 
y
> 
> [2]
> 
z
> 
> [2]
> 
a=b=4

This binds the names 'a' and 'b' to the integer object 4.

b
> 
> 4
> 
a+=2

This attempts to mutate the integer object 4, by adding 2 to it. 
However, numbers in Python are immutable, and so the in-place operation 
fails. Thus, it creates a new integer object equal to 6 (actually, 
CPython keeps a cache of certain smaller integer objects and reuses 
them, but this does not matter in practice). This new integer object is 
bound to the name 'a'. The name 'b' remains bound to the original 4 object.

a
> 
> 6
> 
b
> 
> 4
> 
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Augument assignment versus regular assignment

[Kirk McDonald]

nagy wrote:
> Thanks, Kirk.
> I considered the += as only a shorthand notation for the assignment
> operator.
> Since for lists + is simply a concatetation, I am not sure it x=x+[2]
> is creating a brand
> new list. Could you refer me to any documentation on this?

Yes:

http://docs.python.org/ref/augassign.html
"An augmented assignment expression like x += 1 can be rewritten as x = 
x + 1 to achieve a similar, but not exactly equal effect. In the 
augmented version, x is only evaluated once. Also, when possible, the 
actual operation is performed in-place, meaning that rather than 
creating a new object and assigning that to the target, the old object 
is modified instead."

This behavior is only logical. Consider:

 >>> x = [2]
 >>> y = x + [4]

After these operations, we have two lists: x (the list [2]) and y (the 
list [2, 4]). This is because the expression "x + [4]" creates a new 
list. We then bind this new list to the name 'y', and leave the name 'x' 
alone.

If we then say this:

 >>> x = x + [6]

We are doing much the same operation. We are creating a new list (the 
list [2, 6]), and binding it to the name 'x'. The list [2], previously 
bound to 'x', is no longer bound to anything, so Python frees it.

The augmented assignment, as I went over previously, attempts to modify 
the list object directly. Any names bound to the object (or any other 
objects that reference the object) will see the changes.

-Kirk McDonald
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Augument assignment versus regular assignment

[Kirk McDonald]

Frank Millman wrote:
> nagy wrote:
> 
>>Thanks, Kirk.
>>I considered the += as only a shorthand notation for the assignment
>>operator.
>>Since for lists + is simply a concatetation, I am not sure it x=x+[2]
>>is creating a brand
>>new list. Could you refer me to any documentation on this?
>>Thanks,
>>Nagy
> 
> 
> My habit is to check the id.
> 
> 
>>>>x = [1,2]
>>>>id(x)
> 
> -1209327188
> 
>>>>x += [4]
>>>>x
> 
> [1,2,4]
> 
>>>>id(x)
> 
> -1209327188
> 
>>>>x = x + [6]
>>>>x
> 
> [1,2,4,6]
> 
>>>>id(x)
> 
> -1209334664
> 
> So it looks as if x +=  [] modifies the list in place, while x = x + []
> creates a new list.
> 
> I am not sure if this is 100% guaranteed,

It is. This is true for any mutable type.

> as I have noticed in the past
> that id's can be reused under certain circumstances. Perhaps one of the
> resident gurus can comment.
> 
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: merits of Lisp vs Python

[Kirk Sluder]

In article <[EMAIL PROTECTED]>,
 "mystilleef" <[EMAIL PROTECTED]> wrote:

> 1). More and better mature standard libraries (Languages don't matter,
> libraries do).

> On Lisp Macros:
> 
> I think they are overrated, and in general cause more harm than good.
> It's the reason I find Lisp-like programs difficult to grok, maintain
> and extend. Cos every smart ass wants to needlessly write his own mini
> language to the point of absolute obfuscation. Naturally, I'm supposed
> to be awed by his mischievous cleverness.

I've not seen a convincing explanation as to why imported macros 
from some library are so much more evil than imported functions. In 
both cases one might have to dig into documentation and/or comments 
to understand exactly what that imported snippit is doing.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: merits of Lisp vs Python

[Kirk Sluder]

In article 
<[EMAIL PROTECTED]>,
 Steven D'Aprano <[EMAIL PROTECTED]> wrote:
> Now, if you want to tell me that, despite all the talk, Lisp coders don't
> actually create new syntax or mini-languages all that often, that they
> just use macros as functions, then the question becomes: why do you need
> macros then if you are just using them as functions? Why not use functions?

Well, as a user of both lisp and python, it seems to me that python 
programs can create their own domain-specific mini-languages using 
mechanisms such as objects, classes and functions. It's quite 
possible to mask the bare-bones python behind functions and objects 
derived from imported libraries, which themselves might not be 
python-native.  And extreme advocates of functional programming 
raise the same arguments about objects: they create obfuscated 
constructs that are difficult to read and understand without 
unpacking their internals.  The questions could be asked, why do you 
need objects if you are just using them as containers for functions? 
Why not just use functions?

Python already is very explicit about supporting OOP, a paradigm 
that bundles functions and data into larger constructs. So why is it 
that many of these debates center around macros, which is another 
paradigm that bundles functions and data into larger constructs?  
Both objects and macros have IMO similar potential for abuse and 
obfuscation. 

But to answer your question, I use a macro when I have a problem 
that can't be easily reduced to a naive function. I also use object 
classes when I have a problem that can't be easily reduced to a 
simple data type.  When I don't need an macro or an object class, I 
just use naive functions and simple data types.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: merits of Lisp vs Python

[Kirk Sluder]

In article <[EMAIL PROTECTED]>,
 Pascal Bourguignon <[EMAIL PROTECTED]> wrote:

> Kirk  Sluder <[EMAIL PROTECTED]> writes:
> > I've not seen a convincing explanation as to why imported macros 
> > from some library are so much more evil than imported functions. In 
> > both cases one might have to dig into documentation and/or comments 
> > to understand exactly what that imported snippit is doing.
> 
> And the difference with a library function is?
> 
> (defpackage "LIBRARY" (:export "THIS-IS-A-FUNCTION"))
> 
> (library:this-is-a-function ???) ; ???

Well, my argument is that there is little difference. Functions, 
objects and macros all redefine some aspect of the system's 
language, and all of them can be vulnerable to obfuscation or 
unnecessary abstraction.  The question I have is why do critics 
single out macros and not other forms of abstraction such as 
objects, packages, libraries, and functions?

just as an example:
from foolib import *
bar.bar("somefile")

What does this program do? I have no idea. Its functionality is 
hidden behind multiple layers of abstraction (function, object, 
library.)
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: merits of Lisp vs Python

[Kirk Sluder]

In article 
<[EMAIL PROTECTED]>,
 Steven D'Aprano <[EMAIL PROTECTED]> wrote:

> On Sat, 09 Dec 2006 21:55:19 +, Kirk Sluder wrote:
> 
> Who says they do? All forms of abstraction have been criticized. Sometimes
> the criticism is valid. Sometimes it warns against abuses of the
> abstraction. Just because a feature is useful sometimes doesn't
> necessarily mean the benefit outweighs the cost.

The primary focus of the criticism in this discussion has been that 
macros are a bad from of abstraction compared to libraries, (and I'm 
admittedly extending that to objects, functions and classes.)

> But at least you know that foolib is a module or package. You know what
> from and import do, and that can't change. And you know that bar is an
> object with a method also called bar, it is being called, and the
> argument is a string "somefile". Those things can't change. Imagine a
> hypothetical language where those two lines could mean *anything*.

But as was pointed out earlier in the thread, those keywords can be 
overshadowed by custom functions in python as well. The only reason 
that we don't assume those two python lines could mean *anything* is 
because those mechanisms are not well advertised, and most people 
don't have a need to shadow "import."

Of course, it is possible to abstract away all of the lisp in such a 
way that you have a completely new programming language. But is this 
really a problem? Perl5 and bash are abstractions of C. Python has 
been implemented on both C and java, and chunks of perl6 have been 
implemented on top of Haskell.  Completely new programming languages 
and frameworks develop their own base of knowledge and expertise 
separate from the parent language.

> My point isn't whether or not their claims are correct (a "couple" of
> macros? really?) but that things like this feed the perception that Lisp
> is close to that hypothetical language where anything could be anything.
> If anything could be anything, do you really know what (+ 1 2) means
> without reading every line of code?

How do you know that operator has not been shadowed or overloaded in 
python?  In most cases what you have to do is trust both the 
underlying implementation, and the author(s) of the libraries you 
use.

Both python and lisp share mechanisms to protect namespaces is part 
of the answer. So one way to protect myself is to run my code in 
it's own namespace, and require explicit addressing of imported 
functions and constructs. In this way I can be pretty certain that 
(+ 1 2) uses the lisp primitives, and has not been shadowed, while 
(SOMEPACKAGE:+ 1 2) should be treated with suspicion.

The process of writing lisp often involves access to the REPL, so I 
can have lisp expand the definition of any expression: 
CL-USER> (macroexpand '(+ 1 2))
(+ 1 2)
NIL
CL-USER> (macroexpand '(loop for i upto 100 collect i))
(BLOCK NIL
   ..
)
T
CL-USER> 

This tells me that + is not a macro in my current environment. While 
loop is.

And at least one of the powerful aspects of lisp is than since lisp 
programs are nested stets of s-expressions, you can design something 
that walks through the code and highlights all macro calls, and 
foreign functions that shadow native functions. I don't how 
difficult this would be for languages such as python.

... extended argument snipped ...

> Or maybe it is only an advantage while Lisp programmers are a
> self-selected group of above-average skill. Wait until fifty million VB
> code monkeys start writing Lisp macros and maybe, just maybe, you'll wish
> they were using a less powerful and more restrictive language.

Perhaps it's because I'm a social scientist and not a programmer by 
training, but I find many arguments for *technical* solutions to 
*human performance* problems to be rather weak as a general 
practice.  In some cases, using a very restrictive language may be 
the best solution for the problem.  However, there are plenty of 
other ways around the problem that can be tailored to special needs.

In addition to sandboxing macros into packages and walking the code 
to detect macros, you could run a custom lisp image that does not 
permit DEFMACRO or enforces documentation on DEFMACRO. Or you could 
rely on social mechanisms like only using libraries from trusted 
sources, and better training of those code monkeys.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: merits of Lisp vs Python

[Kirk Sluder]

In article 
<[EMAIL PROTECTED]>,
 Steven D'Aprano <[EMAIL PROTECTED]> wrote:

> So it is good that English restricts the expressiveness and power of the
> syntax and grammar. While we're talking English, we can both understand
> each other, and in fact people who redefine words and ignore the common
> meaning of them are often covering weaknesses in their arguments.

Ohh, can the guy who does discourse analysis for a (meager) living 
respond to this?

To start with, English does not restrict the expressiveness and 
power of the syntax and grammar. People who use the English language 
in specific communities and in specific forms of discourse do. The 
key to how this happens occurs on another layer of how language 
works which is almost always left out of internet discussions of 
language: pragmatics. 

Here is the way it works on usenet.  You write, "...English 
restricts the expressiveness and power of the syntax and grammar." 
And I write, "You should probably do a bit of casual reading into 
human linguistics before you make such silly statements."

In general however, there is this great mechanism embedded in human 
communication called "feedback." Statements that are unclear or 
misunderstood are met with a blank stare, "huh?" or "???" depending 
on medium, mode and context variables.
 
> The same goes for programming languages. Extra expressiveness comes at the
> cost of reduced communication between programmers -- the code becomes
> harder to read for those who haven't already learnt how to read it.

Well, extending this analogy to programming languages, the same 
social mechanisms are in effect. In addition to the technical syntax 
of computer languages, there is also a fair amount of pragmatics 
involved.  Lisp code is expected to conform to a lisp style, and 
python code is expected to conform to a python style.  Developments 
that run counter to that style are more likely to be rejected than 
developments that conform to that style.

http://www.python.org/doc/essays/styleguide.html
http://www.lisp.org/table/style.htm
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: merits of Lisp vs Python

[Kirk Sluder]

In some cases lisp macros are a way of saying tomato to something 
that would be called "tomahto" in python.

One common use of macros is custom iteration constructs. In my 
social network analysis I wanted to do something to each and every 
sender-recipient pair in the header line of "mail" messages in my 
dataset. (And yes, I did have permission to use this data.) Each 
message had one sender, but possibly multiple recipients. Since I 
would be doing multiple forms of analysis, I might as well create 
one iterator to do this.

The DO-MAIL-RELATIONS macro takes the mail message (line), parses 
the sender field and assigns it to (sender), splits the recipient 
field and assigns it to (recip) and then performs the logic in the 
body which often involved looking up values in a database, and 
setting values in a matrix.

(kirk.matrix-process:do-mail-relations (message sender recip)
;;about 12 lines of logic snipped
)

To be pedantic about this call. "KIRK.MATRIX-PROCESS" identifies the 
package/namespace for the macro call. "DO-MAIL-RELATIONS" names the 
macro as part of the DO family of iteration constructs which are 
basic to lisp.

Now that I think about this, in python I'd probably do this using 
object logic that returned the recipient list as an iteratable 
object:

for sender, recipient in message.relationPairs:
   #about 12 lines of logic

Would it have been possible to implement DO-MAIL-RELATIONS as a 
function? Possibly, but I found trying to compact all of that logic 
into a single-use function that could be safely passed to another 
function to be more trouble.

Another common macro use is the "WITH-STREAM" family which opens a 
stream for IO and closes it at the end.  
(with-open-file (file-handle "filename" :direction :output)
   (format file-handle "Hello world.~%")
) 

The pythonic way to do this would be to create a class that 
implements file-like behaviors:

output = fileLike.open()
output.write("Hello world\n")
output.close()

You might want to use a custom "WITH-STREAM" macro or file-like 
object if you need to format, filter, or compress the outgoing 
stream in any way.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: merits of Lisp vs Python

[Kirk Sluder]

In article <[EMAIL PROTECTED]>,
 Paul Rubin  wrote:
> Don't be silly.  Some operators are more natural as infix and others
> as functions.  It's just like in natural language.  People have an
> innate ability to make such distinctions and it's fine for a
> programming language to use it. 

I don't think you can really make this case. In English, German and 
Spanish noun-verb order is flexible to accommodate differences in 
mood, tense, and emphasis:
"Add 2 and 2." imperative.
"The sum of 2 and 2" noun phrase.
"2 was added to 2." passive voice.

Personally, I've always preferred use the imperative to describe 
basic math rather than the passive. This would seem to map better to 
RPN than infix. 

And on top of that, some languages are even more flexible in regard 
to word order, preferring to communicate relationships through the 
modification of noun stems. In such a language, "He* her- the ring+ 
gave," is just as valid as "Her- the ring+ gave he*" and "He* gave 
her- the ring+."  ("+/-/*" signifying modifications which do not 
exist in modern English.) 

At any rate, I find this argument to be weak given that English 
doesn't always use the same word order. And English is just one 
language out of hundreds.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: merits of Lisp vs Python

[Kirk Sluder]

In article <[EMAIL PROTECTED]>,
 Paul Rubin <http://[EMAIL PROTECTED]> wrote:

> Kirk  Sluder <[EMAIL PROTECTED]> writes:
> > Personally, I've always preferred use the imperative to describe 
> > _basic_ math rather than the passive. This would seem to map better to 
> > RPN than infix. 

(emphasis added)
 
> For writing down complicated, nested expressions too?  That's very
> unusual.  E.g.
> 
>   n! = (n/e)**n * sqrt(2*pi*n) * (1 + (1/12n)) * ...
> 
> vs. the same thing in Lisp notation, and that's not even so complicated.

I wasn't even talking about Polish notation vs. other standard 
notations. I was talking about your claimed correspondence between 
infix and natural Languages.

(1/12n) - "divide 1 by the product of" of 12 and n"
sqrt(2*pi*n) - "calculate the square root of the product of 2 pi and 
n." 

If computer languages were to mimic natural languages on this point, 
they would support both forms of expression and be sensitive to mode 
and mood.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: merits of Lisp vs Python

[Kirk Sluder]

In article 
<[EMAIL PROTECTED]>,
 Kirk  Sluder <[EMAIL PROTECTED]> wrote:

> In article <[EMAIL PROTECTED]>,
>  Paul Rubin <http://[EMAIL PROTECTED]> wrote:
> 
> >   n! = (n/e)**n * sqrt(2*pi*n) * (1 + (1/12n)) * ...
>
> If computer languages were to mimic natural languages on this point, 
> they would support both forms of expression and be sensitive to mode 
> and mood.

And ok, bringing this around to lisp, many complex expressions such 
as polynomials can be viewed as lists of sub-expressions. So in this 
case, the expression can be re-written as:

(* (subexp1) (subexp2) (subexp3) (subexp4))

Which is probably how I'd solve the expression if I was using paper 
and pencil: simplify and combine.

And there is something that is missing here in arguing about 
computer language notations in relationship to human language 
readability, or correspondence to spoken language. I'm not writing 
code for another human, I'm writing code for a machine.  Often, the 
optimum expression of an mathematical concept for a machine is 
relatively baroque compared to the optimum expression for a human 
being.
-- 
http://mail.python.org/mailman/listinfo/python-list

Returning a value from code string

[Kirk McDonald]

Say I have a database containing chunks of Python code. I already have a 
way to easily load, edit, and save them. What is slightly baffling to me 
is how I can effectively pass this code some arguments, run it, and 
somehow get a return value.

(Acutally, in the process of writing this post, I figured out a pretty 
good way of doing it. Here's the rest of my post and the solution I came 
up with. Enjoy my thought process!)

Right now I'm thinking of just defining a standard function name, and 
using that as the entry/exit point for the DB-stored code. Is this 
reasonable? Is there some more Pythonic way?

I'll probably compile the code and store the bytecode to the database 
(in addition to the plain text) for faster execution time, as well. It 
shouldn't care whether the stored code is a string or a code object.

What if a stored chunk of code doesn't need to return anything? It would 
be more convenient to just execute the code without the standard 
function. Also, it'd be nice to wrap this in a class:

# module test.py
class Codeobj:
 def __init__(self, code=None):
 if code: self.code = code
 else: self.code = ""

 def __call__(self, args=None, **kwargs):
 # We can pass in a name/value dictionary
 if args: kwargs.update(args)
 exec self.code
 # We don't need a standard function if we're not returning
 # anything
 if locals().has_key('std_func_name'):
 return std_func_name(**kwargs)

if __name__ == "__main__":
 # Load code from the DB
 double = """\
def std_func_name(n):
 return n*2"""

 addthree = """\
def std_func_name(n):
 return n+3"""

 noreturn = "print 'arg = %s' % kwargs['arg']"

 a = Codeobj(double)
 b = Codeobj(addthree)
 c = Codeobj(noreturn)

 # Calling with name/value dictionary
 print a(args={'n':5})
 # Calling with specific named argument
 print b(n=4)
 c(arg='foo')
# EOF

$ python test.py
10
7
arg = foo

If I wanted to simplify the 'noreturn' example (well, if there were more 
lines of code this might make it simpler), I could have defined it like 
this:

 noreturn = """\
locals().update(kwargs)
print 'arg = %s' % arg"""

-Kirk McDonald
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Returning a value from code string

[Kirk McDonald]

Steven D'Aprano wrote:
> On Fri, 27 Jan 2006 20:33:53 -0800, Kirk McDonald wrote:
> 
> 
>> Say I have a database containing chunks of Python code. I already 
>> have a way to easily load, edit, and save them.
> 
> 
> Why?

I am implementing an Everything Engine-like system (see everything2.com
and everydevel.com) in Python. (The original is in Perl, and by all
accounts is some pretty ugly code.) The Everything idiom is that
everything in the engine is a 'node.' A node is a discrete unit that can
be loaded from and saved to the database. A node can be asked to display
itself in the page, it has an owner (and users are themselves nodes),
and so on.

One kind of node is a Superdoc, which is a document that mixes HTML with
code. (I have simply implemented these as mod_python PSP documents,
which is all hunky-dory.) Superdocs comprise essentially all of the
important bits of the site. Another kind of node (I'm still deciding
whether to call them Codenodes or Opcodes or maybe Pynodes) is a chunk
of code that can be asked to run itself, and which can be edited, on the 
fly, from within the website. Thus, one can both alter the functionality 
of the site, and add functionality, from the site itself (so long as you 
have the user permissions to do so).

-Kirk McDonald
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Returning a value from code string

[Kirk McDonald]

Fredrik Lundh wrote:
> executing the code in a custom namespace is a lot cleaner:
> 
> ns = {}
> ns["os"] = os # insert "preimported" values
> ns["app"] = app # insert "preloaded" objects
> exec self.code in ns
> try:
> func = ns["std_func_name"]
> except KeyError:
> pass
> else:
> func(**kwargs)
> 
> instead of using a standard function, you can let the code
> objects talk to the application via a preloaded object or an
> interface module.  e.g. instead of
> 
> def std_func_name(app, args):
> app.something()
> 
> your code object could simply become
> 
> app.something()
> 

These code objects should all return an instance of a certain class (or 
one of its descendants). In another branch of this topic, I described 
roughly what this is for. So, without going into too much detail, I'll 
just repeat that this is for a website backend. Whenever an HTML form is 
submitted, a hidden "op" element (short for operation) in the form is 
checked. This value will send the rest of the form data on to the code 
object it represents. (If you've read my other post: These code objects 
are nodes, the value for the "op" element is the ID number of one of 
them, or possibly one of a few mnemonics for some of the more important 
ops.)

So, taking another swipe at it with your suggestions:

def __call__(self, args=None, **kwargs):
 ns = {}
 ns.update(kwargs)
 ns["args"] = args
 ns["retval"] = None
 exec self.code in ns
 try:
 func = ns["std_func_name"]
 except KeyError:
 return ns["retval"]
 else:
 # Already passed kwargs into the namespace
 return func()

Then, a code object can either return a value by using whatever happens 
to be in 'retval' when it's done executing, or by defining a 
'std_func_name' function and using its return value.

So the interface for these Codenodes needs to be able to accept 
arguments as a name/value dictionary corresponding to the elements in 
the HTML form (the 'args' argument in my example above). This dictionary 
is passed on to the code object. The kwargs are a few important 
variables (not supplied by the form) that all Codenodes need to know, 
such as the current user and maybe a database cursor.

Each Codenode knows how to deal with the input from its associated form. 
(Or forms! There is, for example, a generalized "commit" Codenode that 
knows how to update the state of *any* node if a form follows a certain 
template.) When it is done, it needs to then return whichever node 
should be displayed next. If the operation being attempted violates the 
user's permissions, it might return the "access denied" node. (Actually, 
in that case it will raise an AccessDenied exception and expect 
something down the line to deal with it, so I can define what the 
"access denied" node is in a central place, but you get the idea.) If 
the operation resulted in a new node being created, it might return the 
new node. &c.

Sorry if I tend to ramble, heh. :-)

-Kirk McDonald
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: MYSql, CGI web page search code not working

[Kirk McDonald]

Fred wrote:
> No matter what I type in the form text box (or even if I leave it
> blank) I get all the records.

Try this:

#!/usr/local/bin/python
print "Content-Type: text/html\n"
import MySQLdb
import cgi

db=MySQLdb.connect(host = 'localhost', db = 'phone')
cursor=db.cursor()
cursor.execute("Select * from phone where name=%s order by name", (name,))

result = cursor.fetchall()
for record in result:
 print ''
 print record[0]
 print '--'
 print record[1]
 print '--'
 print record[2]
 print '--'
 print record[3]
 print ''

(Assuming the name of your text field is "name".)
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: MYSql, CGI web page search code not working

[Kirk McDonald]

Fred wrote:
> Yeah, I already tried that (except you have a , after name.
> 
> Your code produces the same error:
> 
>  NameError: name 'name' is not defined
> 
> I know I am close!! Just missing some small thing...
> 

Oh, duh. I forgot something:

#!/usr/local/bin/python
print "Content-Type: text/html\n"
import MySQLdb
import cgi

form = cgi.FieldStorage()

db=MySQLdb.connect(host = 'localhost', db = 'phone')
cursor=db.cursor()
cursor.execute("Select * from phone where name=%s order by name",
 (form['name'].value,))

result = cursor.fetchall()
for record in result:
 print ''
 print record[0]
 print '--'
 print record[1]
 print '--'
 print record[2]
 print '--'
 print record[3]
 print ''

The comma is intentional: the MySQLdb wants the argument(s) as a tuple.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Returning a value from code string

[Kirk McDonald]

Max wrote:
> Kirk McDonald wrote:
> 
>> Another kind of node (I'm still deciding
>> whether to call them Codenodes or Opcodes or maybe Pynodes) is a chunk
>> of code that can be asked to run itself, and which can be edited, on 
>> the fly, from within the website. Thus, one can both alter the 
>> functionality of the site, and add functionality, from the site itself 
>> (so long as you have the user permissions to do so).
>>
> 
> As Steven said, "U R pwn3d". 1f you d0n't sp3a|< l33t (if you don't 
> speak leet), that means you are screaming "hack me, use me to launch 
> attacks on other computers, and than attack my computer". Unless you 
> have some revolutionary ideas in code-security analysis. In which case 
> you can a lot more money than from implementing Everything2 in python.
> 
> --Max

Heavens! Normal users can't edit code! They won't even see it! I'm not a 
*total* moron. The only thing users will be able to enter is some 
simplified HTML. This is a convenience feature for the (trusted) admins 
of the site. There are some simple permission features built into the 
API. Every database-altering API call takes the current user as an 
argument, and if they're not allowed, it tells them to get bent.

Everything2 does this more or less the same way, and they've had few 
issues in the seven or so years they've been operating.

-Kirk McDonald
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: MYSql, CGI web page search code not working

[Kirk McDonald]

Fred wrote:
> OK one more... how would I do a "LIKE" instead of a = in this code?
> 
>  cursor.execute("Select * from phone where name=%s order by name",
>  (form['name'].value,))
> 
> Right off I think:
> 
>  cursor.execute("Select * from phone where name like %%s% order by
> name",
>  (form['name'].value,))
> 
> But it blows up...
> 

This should work:

cursor.execute("Select * from phone where name like %s order by name",
 ('%'+form['name'].value+'%',))

-Kirk McDonald
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: MYSql, CGI web page search code not working

[Kirk McDonald]

Dennis Lee Bieber wrote:
> On Sat, 28 Jan 2006 10:14:44 -0800, Kirk McDonald <[EMAIL PROTECTED]>
> declaimed the following in comp.lang.python:
> 
> 
> 
>>The comma is intentional: the MySQLdb wants the argument(s) as a tuple.
> 
> 
>   The DB-API wants tuples... But my last perusal of the MySQLdb Python
> code showed that it would work with naked singletons...

Ah! So it does. However, I still pass 'em as a tuple as a matter of 
course, since it's documented that way. *shrug* (Also, it saves that 
many keystrokes if I need to add arguments.)

-Kirk McDonald
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: String Manipulation Help!

[Kirk McDonald]

Dave wrote:
> OK, I'm stumped.
> 
> I'm trying to find newline characters (\n, specifically) that are NOT
> in comments.
> 
> So, for example (where "<-" = a newline character):
> ==
> 1: <-
> 2: /*<-
> 3: --<-
> 4: comment<-
> 5: --<-
> 6: */<-
> 7: <-
> 8: CODE CODE CODE<-
> 9: <-
> ==

[snip]

Well, I'm sure there is some regex that'll do it, but here's a stupid 
iterative solution:

def newlines(s):
 nl = []
 inComment = False
 for i in xrange(len(s)):
 if s[i:i+2] == '/*':
 inComment = True
 if s[i:i+2] == '*/':
 inComment = False
 if inComment: continue
 if s[i] == '\n':
     nl.append(i)
 return tuple(nl)

Your example returns:
(0, 64, 65, 80, 81)

This probably isn't as fast as a regex, but at least it works.

-Kirk McDonald
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Decoupling the version of the file from the name of the module.

[Kirk McDonald]

[EMAIL PROTECTED] wrote:
> I'm a newbie experimenting with Python. I want to incrementally develop
> a module called 'circle'. The problem is now that the file name is used
> for two purposes. To keep track of the version number and as the name
> for the module. So when I develop the first version of my file I have
> to call it circle_a.py. The name of the module then automatically
> becomes circle_a. But when I develop the next increment and call my
> file circle_b.py the module name changes as well.
> 
> Basically I want to decouple the version of my file from the name of
> the module.
> 
> Is there a *simple* way out of this dilemma.
> 

I would recommend just naming the file circle.py, and defining something 
like a variable named __version__ or maybe __revision__ at the top of 
the module. Then you can, I don't know, back up your old versions to 
other filenames or something.

Or, if you really want to do this right, you could install Subversion. :-)

-Kirk McDonald
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Decoupling the version of the file from the name of the module.

[Kirk McDonald]

[EMAIL PROTECTED] wrote:
> Now suppose I have make a new version with __version__ = 1.1. What
> shall I call this file and (I don't want to overwrite the old file if I
> need to go back to it) how do I import it from the shell. Your advice
> sounds nice, but I would appreciate if you could give me (or point me
> to) a simple example.
> 
> Thanks
> 

Before you make a new version, rename circle.py to, e.g., circle-1.0.py, 
and then create the new version as circle.py. Then you can access the 
new version just like you accessed the old. If you make yet another new 
version, then rename the current circle.py as circle-1.1.py, and lather, 
  rinse, repeat.

However, I'd still look into a version control system like Subversion. 
It can do all of this for you.

-Kirk McDonald
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Returning a value from code string

[Kirk McDonald]

Peter Hansen wrote:
> In the specific case in question, Kirk's first post gave little or no 
> hint about how much he knew about security issues and, by asking "why?", 
> with a single word Steven hoped to learn enough to say things like "oh, 
> there's a package designed to do that which you can just plug in" or 
> "oh, it's clear you are totally new to issues of network security and 
> are about to put your foot right in it, so here's some background to 
> save you from yourself", or whatever...

Heh heh. For what it's worth, I did have a perfectly good reason for 
trying to do what might otherwise be a suicidally insane idea. On the 
other hand, I've been re-evaluating my needs, and I'll probably be 
hardcoding the more important bits of code that I was going to store in 
the database (in that pulling these things from the database whenever I 
need to use them is remarkably slow compared to letting mod_python cache 
them, and these particular functions are unlikely to ever change). If I 
do need a way to have dynamic code in the database, I've added a thing 
that just reflects POST data back to the database-stored PSP document in 
which the form was found (if the form wants it to), letting the PSP 
document deal with it. That should be adequate.

In short, I learned something about exec and namespaces that I'm not 
actually going to use. Hooray!

-Kirk McDonald
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Storing lines from a text file

[Kirk McDonald]

[EMAIL PROTECTED] wrote:
> so this:
> a, b, c, d, e =f.readlines()
> 
> ..this will put the first line in a, second in b, etc? How do I
> accomplish this when I'm not sure how many lines there are going to be
> every time? Thanks.
> 

With a list:
http://python.org/doc/2.4.2/tut/node5.html#SECTION00514

-Kirk McDonald
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Storing lines from a text file

[Kirk McDonald]

[EMAIL PROTECTED] wrote:
> With what kind of list? I don't see how I can do it with a list unless
> I create one indefinate list and use the objects in the indefinate list
> for the names of  the lists to hold the lines of text. Is that how you
> are suggesting that I do it?
> 

You're thinking too hard. Say you want to read in all the lines from the 
file object f and just print them out one at a time:

lines = f.getlines()
for line in lines:
     print line

Simple.

-Kirk McDonald
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Storing lines from a text file

[Kirk McDonald]

[EMAIL PROTECTED] wrote:
> I keep getting an error when I try to use what you said Mr. McDonald. I
> think I've  got something wrong, but take a look if you can.
> 
> log = open('C:\log_0.txt')
> lines = log.getlines()
> for line in lines:
>  print line
> 
> When I debug it the error I get is the following:
>AttributeError: 'file' object has no attribute 'getlines'
> 

D'oh! That's because the method is readlines(). Stupid brain:

log = open('C:\log_0.txt')
lines = log.readlines()
for line in lines:
 print line

-Kirk McDonald
-- 
http://mail.python.org/mailman/listinfo/python-list

URL Character Decoding

[Kirk McDonald]

If you have a link such as, e.g.:

Main menu!

The space will be translated to the character code '%20' when you later 
retrieve the GET data. Not knowing if there was a library function that 
would convert these back to their actual characters, I've written the 
following:

import re

def sub_func(m):
 return chr(int(m.group()[1:], 16))

def parse_title(title):
 p = re.compile(r'%[0-9][0-9]')
 return re.sub(p, sub_func, title)

(I know I could probably use a lambda function instead of sub_func, but 
I come to Python via C++ and am still not entirely used to them. This is 
clearer to me, at least.)

I guess what I'm asking is: Is there a library function (in Python or 
mod_python) that knows how to do this? Or, failing that, is there a 
different regex I could use to get rid of the substitution function?

-Kirk McDonald
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: URL Character Decoding

[Kirk McDonald]

Kirk McDonald wrote:
> If you have a link such as, e.g.:
> 
> Main menu!
> 
> The space will be translated to the character code '%20' when you later 
> retrieve the GET data. Not knowing if there was a library function that 
> would convert these back to their actual characters, I've written the 
> following:
> 
> import re
> 
> def sub_func(m):
> return chr(int(m.group()[1:], 16))
> 
> def parse_title(title):
> p = re.compile(r'%[0-9][0-9]')
> return re.sub(p, sub_func, title)
> 
> (I know I could probably use a lambda function instead of sub_func, but 
> I come to Python via C++ and am still not entirely used to them. This is 
> clearer to me, at least.)
> 
> I guess what I'm asking is: Is there a library function (in Python or 
> mod_python) that knows how to do this? Or, failing that, is there a 
> different regex I could use to get rid of the substitution function?
> 
> -Kirk McDonald

Actually, I just noticed this doesn't really work at all. The URL 
character codes are in hex, so not only does the regex not match what it 
should, but sub_func fails miserably. See why I wanted a library function?

-Kirk McDonald
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: URL Character Decoding

[Kirk McDonald]

Kirk McDonald wrote:
> Actually, I just noticed this doesn't really work at all. The URL 
> character codes are in hex, so not only does the regex not match what it 
> should, but sub_func fails miserably. See why I wanted a library function?
> 
> -Kirk McDonald

Not to keep talking to myself, but looks like sub_func works fine, and 
the regex just needs to be r'%[0-9a-fA-F][0-9a-fA-F]'. But even so.

-Kirk McDonald
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: URL Character Decoding

[Kirk McDonald]

Paul McGuire wrote:
> "Kirk McDonald" <[EMAIL PROTECTED]> wrote in message
> news:[EMAIL PROTECTED]
> 
>>If you have a link such as, e.g.:
>>
>>Main menu!
>>
>>The space will be translated to the character code '%20' when you later
>>retrieve the GET data.
>>
>>I guess what I'm asking is: Is there a library function (in Python or
>>mod_python) that knows how to do this? Or, failing that, is there a
>>different regex I could use to get rid of the substitution function?
>>
>>-Kirk McDonald
> 
> 
> 
>>>>import urllib
>>>>urllib.quote("index.py?title=Main Menu")
> 
> 'index.py%3Ftitle%3DMain%20Menu'
> 
>>>>urllib.unquote("index.py%3Ftitle%3DMain%20Menu")
> 
> 'index.py?title=Main Menu'
> 
> 

Perfect! Thanks.

-Kirk McDonald
-- 
http://mail.python.org/mailman/listinfo/python-list

Introspection with classes

[Kirk McDonald]

Me again! Same project as before, if you're playing along at home.

The Everything Engine model (which I am blatantly copying in my project) 
is that everything in the engine is a 'node,' and that every node can be 
stored to and read from the database. Since this is the backend for a 
website, minimizing hits on the database is a worthy goal. Thus, it is 
paramount that reading in any given subclass of the base 'Node' class 
take (optimally) one query. (A simple, yet fairly simple-minded design 
-- and my first stab at the problem -- would be to have each subclass 
read in its own data after calling its parent's -- or parents'! -- read 
method(s). This has the disadvantage of cluttering a lot of code with 
database queries and making it quite difficult to create new subclasses.)

Here's a rough sketch of what I'm doing at the moment. I've invested 
enough time into it at this point that I've totally lost track of 
whether it's a good idea, and I would please like someone to tell me.

Every node has certain attributes. In this outline, let's say these are 
the node's unique id number, its title, and the id number of its owner.

class Node:
 dbInfo = { 'title': ['node', 'title',""]
'owner_id' : ['node', 'owner_id', None] }

 def __init__(self):
 self.default()

 def default(self):
 self.node_id = None
 for k, v in self.dbInfo.items():
 setattr(self, k, v[2])

 def read(self, db, node_id):
 # construct a database query from the table/column info in
 # dbInfo

 def commit(self, db):
 # write the current values to the db

 def insert(self, db):
 # insert a new node using the current values

 def nuke(self, db):
 # remove the node from the db, then:
 self.default()

So, simple enough. (The real version has signifigantly more stuff going 
on, as well as a permission system, but I have elided these for the sake 
of clarity.) By doing that dance with dbInfo, I can easily subclass Node.

Say we want a new node that holds some text. We just need to add a new 
attribute like so:

class Document(Node):
 dbInfo = {}
 dbInfo.update(Node.dbInfo)
 dbInfo['content'] = ['docs', 'text', ""]

... and that's it. That's the whole Document class. (Well, the real 
version has a render method that takes the content, parses it in 
interesting ways, and writes it to the client, but that's simple, too.) 
The shortness of this class is why this strikes me as a good idea.

The other advantage is I can make use of multiple inheritance. First, 
say we have a class that just holds a dictionary:

class DictNode(Node):
 dbInfo = {}
 dbInfo.update(Node.dbInfo)
 dbInfo['stuff'] = ['dictnode', 'dict', {}]

(Assume a mechanism is provided to automatically pickle the dictionary, 
which there is.)

Now say I want a nodetype that's just a document with a dictionary 
attached. This is almost exactly how the User class is implemented: the 
document holds the user's "homenode," which is just a brief bio or 
whatever the user wants to put in there, and the dictionary holds the 
user's settings. They snap together like Legos. It looks just like this:

class User(Document, DictNode):
 dbInfo = {}
 dbInfo.update(Document.dbInfo)
 dbInfo.update(DictNode.dbInfo)

If the User class wanted its own attributes (say, a password), they 
would just get added to the end.

This is my third stab at the problem (the first was mentioned above, the 
second was basically the same as this but dbInfo was an instance 
variable instead of a class variable which, uh, was sort of dumb).

Coming from C++, I'm still wrapping my brain around Python's 
introspection abilties. Can I generalize this more? Am I missing some 
idiom or language feature that would benefit me? Where's my time machine?

-Kirk McDonald
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: OO conventions

[Kirk McDonald]

Daniel Nogradi wrote:
> I'm relatively new to object oriented programming, so get confused
> about its usage once in a while. Suppose there is a class Image that
> has a number of methods, rotate, open, verify, read, close, etc. Then
> to use this class my natural guess would be to have something like
> 
> image = Image( )
> image.read( "myfile.jpg" )
> image.rotate( )
> image.close( )
> 
> But now it turns out that the PIL module uses this as
> 
> image = Image.open( "myfile.jpg" )
> image.verify( )
> image.rotate( )
> image.close( )
> 
> Perhaps the real Image class of PIL doesn't have these methods
> exactly, but doesn't matter, my point is the way it works. Is it
> normal that instead of first creating an instance of a class, it
> starts right away with one its methods? I of course understand that
> the PIL people simply made a choice that their module works this way,
> period, but I'm just wondering if it wouldn't have been more "logical"
> to do it along the way of my first example.
> 
> I guess it's just a matter of convention or how the programmer feels
> like, but there are no conventions of this type? Which would be more
> pythonic? Or I shouldn't worry and it's totally up to the developer?


If I were coding a class like that, I'd probably do it like this:

image = Image("myfile.jpg")
image.rotate()
image.close()

And, in addition, there would be an image.open() method. In the above, 
__init__ just calls it on its own when the filename is specified.

(Though, of course, I'm not at all familiar with the PIL module; there 
might be more stuff going on behind the scenes with the way they do it.)

-Kirk McDonald
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: cgi - secure sessions

[Kirk McDonald]

[EMAIL PROTECTED] wrote:
> Hey,
> 
> I was just wondering if / how would it be possible to create secure
> sessions for a website using Python CGI... I thought of using cookies,
> and things looked promising for a while; I could login through a form
> which pointed to a cgi script which created sent the user cookies, but
> I found that when a script to detect the cookies was run through a
> server side include line in the html, it couldn't get any cookies, but
> it would work fine when run directly through the browser (which is
> useless to me).
> 
> If anybody could help with this it would be great. Python is the only
> programming language that I'm relatively comfortable in at the moment,
> so using the usual PHP or Javascript just isn't an option for me
> unfortunately.
> 
> GazaM
> 

For what it's worth, mod_python supports sessions:

http://www.modpython.org/live/current/doc-html/pyapi-sess.html

I've been playing with them recently, and they seem to work. :-)

-Kirk McDonald
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: cgi - secure sessions

[Kirk McDonald]

GazaM wrote:
> What I have is a cgi script run through a server
> side include line in the html, which looks for the session cookie, if
> it is present will say 'logged in as "user"' and if the cookie isn't
> there will display a login form. Now, the big showstopper here is that,
> because session cookies are stored in http headers sent by the client
> (afaik) the cgi script can't get any, because the http headers are
> passed onto the html file and any cgi scripts inside don't get
> anything... is there a workaround possible? 

Python has a built-in Cookie module:

http://www.python.org/doc/2.4.2/lib/module-Cookie.html

It may simplify matters.

-Kirk McDonald
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: nDimensional sparse histogram in python.

[Kirk McDonald]

KraftDiner wrote:
> The dictionary is sorted by the point key.
> Is there a way to sort the dictionary by the value?
> Seems to me this goes against the purpose of a dictionary but
> thats what I need to do..
> 

Well, it's probably not all that efficient, but it is simple code:

sortedList = [(v, k) for k, v in self.histo.items()]
sortedList.sort()

Should do it...

-Kirk McDonald
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: nDimensional sparse histogram in python.

[Kirk McDonald]

KraftDiner wrote:
> Ok so this is nice.. Just one thing.. When you try to get a value from
> a dictionary
> and it isn't found in the dictionary things go bad...
> 
> Take this for example:
> 
> class histogram(object):
>   def __init__(self):
>   self.histo = {}
> 
>   def update(self, point):
>   if self.histo.get(point) != None:
>   self.histo[point] = self.histo[point] + 1
>   else:
>   self.histo[point] = 1
> 
>   def get(self, point):
>   return self.histo[point]
> 
> 
> hist = histogram()
> hist.update((0,0,0))
> hist.update((0,0,1))
> hist.update((0,0,1))
> hist.get((0,0,0))
> hist.get((0,0,1))
> hist.get((0,0,2))
> 
> spews out this error:
> 
> Traceback (most recent call last):
>   File "histogram.py", line 21, in ?
> hist.get((0,0,2))
>   File "histogram.py", line 12, in get
> return self.histo[point]
> KeyError: (0, 0, 2)
> 

Try this:

class histogram(object):
def __init__(self):
self.histo = {}

def update(self, point):
self.histo[point] = self.histo.get(point, 0) + 1

def get(self, point):
return self.histo.get(point, 0)

dict.get's default return value is your friend.

-Kirk McDonald
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: nDimensional sparse histogram in python.

[Kirk McDonald]

KraftDiner wrote:
> Many thanks everyone.
> 
> One last quick question...
> The dictionary, I believe, is sorted by the key.
> Is there a way to sort it by the value?
> Say I wanted to put out a list of the frequency sorted by highest to
> lowest?
> 

The dictionary itself is actually unordered; a C++ std::map this ain't. 
To sort its elements, you need to build a list of the items and sort 
that, e.g.:

items = [(v, k) for k, v in self.histo.items()]
items.sort()

This will give you a list of (value, key) tuples sorted by value.

-Kirk McDonald
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Automatic class attribute

[Kirk McDonald]

Franck PEREZ wrote:
> Hello all,
> 
> Considering the following code :
> 
> class C(object):
>...: observers = []
>...:
>...: @classmethod
>...: def showObservers(cls):
>...: print cls.observers
> 
> class D1(C):
>...: observers = [] #could it be moved in C ?
> 
> class D2(C):
>...: observers = [] #could it be moved in C ?
> 
> I want each child class of C to have it's own "observers" class attribute.
> 
> The code I provided works... but I'd like to avoid typing "observers =
> []" in each child class.
> 
> Is it possible to define something in C which would basically mean :
> "for each child class, automatically bind a new list attribute called
> observers" ?
> 
> Are metaclasses a way ? Is it possible to avoid them ?
> Thanks a lot,
> Franck

By an astounding coincidence, I was just working on a similar problem. 
Metaclasses can do this, no problem:

class M(type):
 def __init__(cls, name, bases, dict):
 cls.observers = []

 def showObservers(cls):
 print cls.observers

class C(object):
 __metaclass__ = M

class D1(C): pass
class D2(C): pass

-Kirk McDonald
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Global variables, Classes, inheritance

[Kirk McDonald]

DaveM wrote:
> Although I've programmed for fun - on and off - since the mid 70's, I'm
> definitely an OO (and specifically Python) beginner. 
> 
> My first question is about global variables. Are they, as I'm starting to
> suspect, a sin against God or just best avoided? Having got my current
> application working using them, I'm not sure whether I want to refactor it,
> but equally, I'd like to write the best code I can.

Python globals are actually remarkably safe, at least compared to some 
other languages. The key is that, when you import a module, its globals 
stay in its namespace. I've got nothing against them.

> 
> Secondly (and this is related),  if you have multiple instances of the same
> class, but need them all to have access to a dictionary created at run-time,
> is there a class way to this without calling the creation method multiple
> times? The best thought I've come up with is for the class instance to check
> its name and only if it's the first one do the dictionary creation, but I
> haven't tried it yet.

A class attribute would do this easily:

class Foo(object):
 d = {}

a = Foo()
b = Foo()
Foo.d = { 'a' : 'apple' }
print a.d
print b.d

That will print out {'a':'apple'} twice.

> 
> My third difficulty is with variables in a class. What's the difference
> between the following?:
> 
> class Foo:
> i = 12345
> ...
> 

i is a class attribute. It is bound to the class (which is, itself, an 
object). You can access i both from the class itself as well as from an 
instance of the class, i.e., Foo.i and Foo().i will both work.

> class Foo:
> self.i = 12345
> ...
> 

This does not work. There is no name 'self' bound to class Foo.

> class Foo:
> def __init__(self):
> self.i = 12345
> ...
> 

This binds the name 'i' to all instances of class Foo; if this were C++, 
we'd call self.i a member variable.

> class Foo:
> def __init(self):
> i = 12345
> ...
> 

(You forgot the trailing __, but whatever. :-) This binds the name 'i' 
to the local namespace of the __init__ function. It is just like any 
other local variable in a function.

Namespaces in Python are really great. It is worth reading up on 
globals() and locals() if you don't get them yet.

-Kirk McDonald
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: python's library support

[Kirk McDonald]

Sean wrote:
> I am a newbie in python, and I have a feeling that python provides less
> library support than perl www.cpan.org  This seems a big discussion
> topic.
> 
> I want to know if there is extensive algorithm library support in
> python. I know there is a pretty neat module in perl to implement graph
> theory. Is there a graph theory module in python?
> 
> Thanks,
> 

Python's standard library is actually quite extensive; it's one of the 
language's greatest strengths:

http://www.python.org/doc/2.4.2/lib/lib.html

However, it is perhaps true that it is lacking in the raw algorithm 
department. On the subject of graphs, a quick Google search brings up 
this nice-looking library:

https://networkx.lanl.gov/

For computationally intensive array manipulation, there's always good 
ol' numarray:

http://www.stsci.edu/resources/software_hardware/numarray

The standard library can do quite a lot, and if it fails you, there's 
more than likely a third-party library available.

-Kirk McDonald
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: UnboundMethodType and MethodType

[Kirk McDonald]

Schüle Daniel wrote:
> Hello all,
> 
>  >>> class Q:
> ... def bar(self):
> ... pass
> ...
>  >>> import types
>  >>> types.UnboundMethodType is types.MethodType
> True
>  >>>
>  >>> type(Q.bar)
> 
>  >>>
>  >>> q = Q()
>  >>> type(q.bar)
> 
>  >>>
>  >>> type(q.bar) is types.UnboundMethodType
> True
>  >>> q.bar
> >
>  >>>
> 
> I think is not very consistent
> notice q.bar is bounded although type(q.bar)
> says it's types.UnboundedMethodType
> what do you think?
> 
> Regard, Daniel
> 

I think it's perfectly consistent:

 >>> class B(object):
... def bar(self): pass
...
 >>> B.bar

 >>> type(B.bar)

 >>> b = B()
 >>> b.bar
>
 >>> type(b.bar)

 >>> id(B.bar)
-1211888788
 >>> id(b.bar)
-1211888788

It's the same function, whether it's bound or not. Thus, it should 
always have the same type. It's simply called in different ways. You can 
just as easily say:

 >>> B.bar(b)

As:

 >>> b.bar()

-Kirk McDonald
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: UnboundMethodType and MethodType

[Kirk McDonald]

Scott David Daniels wrote:
> To elaborate on this, once 'id' is called, you drop the reference.
> This allows quite surprising things like:
>  >>> id(7**8) == id(8**7)
> True
>  >>> a, b = 7**8, 8**7
>  >>> id(a) == id(b) # this time there are other references to a and b
> False
> 
> If you wanted to test the original code for identity match:
>  >>> B.bar is B().bar
> False
> is the appropriate test (the 'is' test holds the identities through
> the comparison).
> 
> By the by, this is tricky stuff, nobody should expect to understand
> it thoroughly without both study and testing.
> 
> --Scott David Daniels
> [EMAIL PROTECTED]

You know what? That makes perfect sense. Thank you.

-Kirk McDonald
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: calculating on matrix indices

[Kirk McDonald]

Brian Blais wrote:
> Hello,
> 
> In my attempt to learn python, migrating from matlab, I have the 
> following problem. Here is what I want to do, (with the wrong syntax):
> 
> from numpy import *
> 
> t=arange(0,20,.1)
> x=zeros(len(t),'f')
> 
> idx=(t>5)
> tau=5
> x[idx]=exp(-t[idx]/tau)  # <---this line is wrong (gives a TypeError)
> 
> #--
> 
> what is the best way to replace the wrong line with something that 
> works: replace all of the values of x at the indices idx with 
> exp(-t/tau) for values of t at indices idx?
> 
> I do this all the time in matlab scripts, but I don't know that the 
> pythonic preferred method is.
> 
> 
> 
> thanks,
> 
> bb
> 
> 

You're specifying the type of x but not of t. You need to change the 
line where you assign t to:

t = arange(0, 20, .1, 'f')

I'm not sure why it doesn't figure that own on its own (since it 
obviously does hold floats), but this does cause it to work.

-Kirk McDonald
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: embedding python in HTML

[Kirk McDonald]

John Salerno wrote:
> bruno at modulix wrote:
> 
>> You've got to understand that Python is *not* a 'ServerPage' language
>> (-> php, asp, jsp etc) in itself. Your server can now run python, fine,
>> but *how* ? CGI ? FastCGI ? mod_python ? other ? (hint: it's probably
>> just plain old CGI...)
> 
> 
> So does that mean I need to have something further on the server? Or is 
> this something I can do on my end? How do I find out what I need?

If you really want to use Python as a server page language, mod_python 
has support for Python Server Pages via its PSP handler:

Python Server Pages:
http://modpython.org/live/current/doc-html/pyapi-psp.html

PSP handler:
http://modpython.org/live/current/doc-html/hand-psp.html

This of course means your server needs to have mod_python installed and 
configured. (Consult your server administrator.) However, I've always 
found PSP to be somewhat fiddly, and mixing any serious code with the 
HTML text is hardly pretty.

A more common (and bare-metal) approach is CGI. In CGI, a request for a 
page runs a script, the output of which is the HTML page. I think this 
only requires that the server has Python installed, which you have said 
is the case. Python has signifigant standard library support for writing 
CGI.

You should examine Python's standard cgi module:
http://python.org/doc/2.4.2/lib/module-cgi.html
That page also has some nice examples to get you started.

And maybe its Cookie module, if you ever feel like messing with cookies:
http://python.org/doc/2.4.2/lib/module-Cookie.html

Slightly less bare-metal is using mod_python directly (rather than via 
its PSP module). This is probably preferable to plain CGI if mod_python 
is available, as it caches scripts as long as they are not changed. This 
is faster than reading them off the disk every time. By and large, 
mod_python's API replaces (or at least wraps) the standard library's CGI 
support if you go this route. Again, this is only available if your 
server has mod_python installed, which may or may not be the case.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: 2-dimensional data structures

[Kirk McDonald]

anthonyberet wrote:
> Thanks for the advice (to everyone in the thread).
> I think I will go with nested lists.
> However, I am running into a conceptual problem.
> My approach will be firstly to remove all the impossible digits for a 
> square by searching the row and column for other occurances.
> 
> However, I wondering how to approach the search of the nine regions of 
> the grid. I am thinking of producing another nested list, again 9x9 to 
> store the contents of each region, and to update this after each pass 
> through -and update of- the main grid (row and column).
> 
> I am not sure how to most efficiently identify which region any given 
> square on the grid is actually in - any thoughts, for those that have 
> done this? - I don't want a massive list of IF conditionals if I can 
> avoid it.
> 

When I wrote my Sudoku solver (a terribly inefficient and naive 
recursive algorithm that I originally wrote in C++ and was the first 
thing I ever wrote in Python), I used numarray. It allows 
two-dimensional slicing:

def inBlock(x, y, val):
 blockX = x/size * size
 blockY = y/size * size

 return val in sudoku[blockX:blockX+size, blockY:blockY+size]

'size' in this example is a global variable equal to the size of the 
puzzle, so 3 for a normal puzzle. 'sudoku' is a global two-dimensional 
array simply holding the values in the puzzle.

(There are any number of things in this can be improved, such as using 
the // floor division operator rather than /, not using global 
variables, and so on. What can I say? It was a straight conversion from 
C++. I hardly knew what "Pythonic" meant.)

-Kirk McDonald
-- 
http://mail.python.org/mailman/listinfo/python-list

Python & cgi on win98--tinyweb problems, etc

[Kirk Bailey]

RE: TinyWeb running python in windows

PROBLEM: getting python scripts to execute.
SOLUTION: Insure the script ends in the name extension .py.

Windows associates all such files with the pythonw.exe interpreter program, 
and will use it to interpret them. IT IGNORES THE SHEBANG (the first line in 
a script which in the lovely world of un*x points to the interpreter 
program). Some servers have config files to tell the server what to use. 
TinyWeb does not, and relies on windows file associations to direct it to 
the proper interpreter.

Also note that it is possible for windows to conceal name extensions in some 
configurations, and also to create name extensions it does not display, 
resulting in some interesting hair pulling evenings chasing bugs.

Also note that tinyweb checks for the existence of a default page to use if 
none is specified in an incoming request. IF YOU CHANGE THE FILE NAME AFTER 
TINY LOADS IT WILL BARK LIKE A DOG. For instance, index.htm or index.html 
are equally acceptable. You had .htm. then you decided to change it to 
.html- and the server started woofing. It thinks the file index.htm still is 
there someplace and is looking for it! If you change the file name, restart 
the server.

I used tinyweb in supporting the development of windows wiki, and in all 
that crazy alpha stage flakiness, it NOT ONCE blew out. It is BULLETPROOF.

But it is it's own strange beast, and has it's own peculiarities.

-- 
Salute!
    -Kirk Bailey
   Think
  +-+
  | BOX |
  +-+
   knihT

Fnord.
-- 
http://mail.python.org/mailman/listinfo/python-list

Auto-parallelizing with decorators?

[Kirk Strauser]

I was thinking about how a lot of Lisp proponents claim that Lisp is
inherently parallelizable because its functions don't have (or are not
supposed to have) side effects, and therefore the compiler can easily tell
which calls may be run in parallel instead of strictly serially.  I'm not a
Lisp expert so I can't say whether that's true or not, but it seems like an
interesting idea for Python.

Suppose that Python had a new decorator, say "@parallelizable".  Functions
so decorated would be eligible for multi-processed or multi-threaded
execution by such native constructs as list comprehensions, or the map()
function.  Their logic could be something along the lines of:

1) Is every function in the "function expression" of map() or a list
comprehension decorated with @parallelizable?  If not, use the normal
implementation.
2) Spawn an appropriate number of child threads and queue values to be
processed.
3) As results return, either store them in the result list (for map()), or
insert them into their place in the output queue to be consumed by users of
a list comprehension.

In #2, "appropriate" could default to something like 2 times the number of
CPUs detected in the system (which could be found the first time it was
needed so that programs that don't use this functionality don't pay for
finding that information).  I could imagine that it'd be useful to pass
hints to the decorator to more explicitly size the thread pool, such as:

@parallelizable(number=100)
def lightweightfunction:
"""Launch as many parallel copies as you can"""

@parallelizable(perproc=1)
def heavierfunction:
"""Only run one copy per CPU"""

@parallelizable(number=2)
def bigslowfunction:
"""This benefits from parallel execution, but don't kill the system"""

Would something like this have merit?  I like the idea because it's
completely optional; if you don't use it, you never have to deal with the
side effects.  However, if you take care to make your functions loosely
coupled and not dependent on execution order, you could get a nice speedup
every single time you give Python permission to schedule your repeated
function calls.
-- 
Kirk Strauser
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Auto-parallelizing with decorators?

[Kirk Strauser]

Stefan Behnel wrote:

> Wouldn't that require parallelism in the interpreter first? Mind the
> GIL...

That may be.  I'd bet though that I could whip up a native Python workalike
using os.fork() that would work around GIL, at least on Unix (just because
I don't know if Windows has os.fork(), having never looked for it).

I know something like this wouldn't be the easiest thing to implement, but
honestly, having such beautiful constructs as map() and list comprehensions
available is just begging for an application like this.
-- 
Kirk Strauser
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Auto-parallelizing with decorators?

[Kirk Strauser]

In article <[EMAIL PROTECTED]>,
 Kirk Strauser <[EMAIL PROTECTED]> wrote:

> I was thinking about how a lot of Lisp proponents claim that Lisp is
> inherently parallelizable because its functions don't have (or are not
> supposed to have) side effects, and therefore the compiler can easily tell
> which calls may be run in parallel instead of strictly serially.  I'm not a
> Lisp expert so I can't say whether that's true or not, but it seems like an
> interesting idea for Python.

By the way, I uploaded a sample implementation (in Python) of what I had 
in mind to http://www.honeypot.net/multi-processing-map-python .  Please 
let me know what you think of it and whether it seems remotely 
interesting or goofy.
-- 
Kirk Strauser
-- 
http://mail.python.org/mailman/listinfo/python-list

Recommended validating XML parser?

[Kirk Strauser]

We're looking for a current, supported, validating XML parser.  Since it
seems like there are a few thousand out there, I though we'd see what
everyone else is using.

Bonus points if it can do something like:

>>> foo = XMLParser("""

3000

""", dtd=file('/etc/weightfile.dtd'))

>>> print foo.weight
3000

...or some variant on that theme.
-- 
Kirk Strauser
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python does not play well with others

[Kirk Sluder]

In article <[EMAIL PROTECTED]>,
 Paul Rubin  wrote:

> I do think the core should have more stuff than it does, so that its
> functionality can be on a par with competing language distros like
> J2SE and PHP.  Both of those distros include database connectvity
> modules and web frameworks.  

Can't speak for PHP, but J2SE requires the additional installation 
of the Mysql Connector/J JDBC driver.  Java also does not include a 
web framework as far as I can tell, with many people bolting on 
struts and Tomcat for this purpose.

Now granted, it may or may not be a good idea for python to copy the 
java way of doing things by including the generic database API into 
the core libraries rather than expect database module developers to 
use the API.  But that doesn't solve the core problem that you can't 
get database connectivity just through the core language.

And for that matter, perl doesn't include the database drivers or a 
web framework either.  

php does, but I've always considered php to be a web framework with 
an embedded programming language.   Comparing python to php here 
strikes me as comparing apples and oranges.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python does not play well with others

[Kirk Sluder]

In article <[EMAIL PROTECTED]>,
 Paul Rubin  wrote:

> [EMAIL PROTECTED] writes:
> > Where would you stop?  At the boundaries of your particular application
> > interests?
> 
> If the Pythonistas are serious
> about such a claim, competitive analysis says they should be willing
> to look at what language L does to support application XYZ (example:
> PHP includes database connectivity), make a checklist of L's features,
> and see to it that Python achieves (at least) parity in those areas.

Well, digging into this, php includes database connectivity if 
compiled against the database client libraries.  So it's not as if 
you can just download php and have it work against the database you 
want.  

I also don't think that python as a general-purpose programming 
language is in direct competition to web frameworks like PHP and 
ColdFusion.  Perhaps the right answer is to say that PHP is a better 
solution for people wanting a web templating framework with embedded 
scripting, while Python is a better solution for people who want a 
general scripting/application language with broad support for 
specific applications via add-on modules. 

But then again, I'm one of those heterodox people who argue that one 
should fit the tool to the problem.  Common Lisp, bash, awk, sed and 
R are other great languages for different domains.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python does not play well with others

[Kirk Sluder]

In article <[EMAIL PROTECTED]>,
 "Paul Boddie" <[EMAIL PROTECTED]> wrote:

> Quite. I imagine that most GNU/Linux distributions (and various BSDs)
> provide at least some version of MySQLdb as a package.

Bingo, I've rarely installed python from python.org, or other
libraries from sourceforge, etc., etc.. Usually I've installed 
BSD-style ports, debian-style packages, or RedHat-style RPMs.



> 
> Paul
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python does not play well with others

[Kirk Sluder]

In article <[EMAIL PROTECTED]>,
 "Paul Boddie" <[EMAIL PROTECTED]> wrote:

> Would it benefit the Python community more if Python shipped with
> MySQL support out of the box? Is it likely that a user suddenly finds
> him/herself needing to connect to a MySQL database? 

The other problem is that it chains the python release schedule to 
that of MySQL AB (and postgresql, and whatever.)  One of the key 
advantages to independent modules is that I don't need to update my 
entire python system every time my database vendor releases a new 
library, nor do I have to accept lag time as the module is tested 
and rolled into the base distribution.  Database client libraries 
are much more of a moving target than core language features.

I find it interesting that PHP struggled with these issues and 
decided to abandon embedded MySQL support partly because they 
couldn't maintain their own parallel versions of the client 
libraries.
  
http://us2.php.net/manual/en/faq.databases.php#faq.databases.mysql.ph
p5

Among the other problems faced by both PHP and Python in bundling 
MySQL support is that they can't legally do it without adopting the 
GPL.  

Which leaves me wondering why the python core should adopt a feature 
that was abandoned by PHP, and was never highly recommended or used?

> Paul
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Definitions of editor and IDE?

[Kirk Sluder]

In article <[EMAIL PROTECTED]>,
 "Paddy" <[EMAIL PROTECTED]> wrote:

> On Feb 4, 9:01 am, Necmettin Begiter <[EMAIL PROTECTED]>
> wrote:
> > Can't give definitions, but here is the difference: if you use an editor, 
> > you
> > will have to run your script or program from outside the editor; if you use
> > an IDE, you will press a key or key combination in the IDE (say F9 or F5) 
> > and
> > your program/script will run...
> 
> Hmm, is vim an editor or an IDE?
> By the above definition it would be an IDE because under the tools
> menu you can run make or compilers/interpreters.
> It has probably got to the stage that its a continuum from something
> like Eclipse or SPE that are definitely IDE's through to ed and
> notepad which are editors.

I would define an IDE as a system that supports multiple tasks of 
the development process using a related set of interfaces.  Some of 
these tasks may include:

Searching and browsing reference documentation.
Writing source code.
Building/compiling projects. 
Evaluation using a shell or REPL.
Debugging and testing.
Production of graphical elements.
Production of documentation.
Packaging.
Project management and version tracking.

I'll agree that it's a spectrum with the most comprehensive IDEs 
supporting more different tasks of the development process.  

> 
> - Pad.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: "Subscribing" to topics?

[Kirk Sluder]

In article <[EMAIL PROTECTED]>,
 "Mizipzor" <[EMAIL PROTECTED]> wrote:

> On Feb 4, 9:55 pm, Toby A Inkster <[EMAIL PROTECTED]>
> wrote:
> > You discovered wrong -- Google does not provide a free newsserver. They no
> > doubt *have* several newsservers, but don't provide direct access to them
> > either free, or for a fee. They only provide a web interface to access the
> > contents of their newsservers, with a fraction of the features that a real
> > newsreader would.
> 
> Hehe, it has all the features I knew existed in a newsreader, except
> this "subscribe to topic" thing Im looking for.

If you have My Groups enabled, you can highlight a topic as a 
favorite. When reading a thread, click on the star near the upper 
left corner of the screen under the subject header.  This will add 
that subject to your favorites.  Then to see your favorite topics, 
click "favorites" at the top of the page, toward the right-hand side.

In an email client, you can set a filter that dumps messages from a 
list that don't match specific keywords into the trash or an archive 
folder.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Best Free and Open Source Python IDE

[Kirk Sluder]

In article <[EMAIL PROTECTED]>,
 Stef Mientki <[EMAIL PROTECTED]> wrote:

> Which brings me to some other questions on waste:
> - isn't it a pitty so many people are involved in writing another editor / 
> IDE ?

I don't know about that.  Most of the new editor development appears 
to involve one of the following:

1: Taking advantage of a specific graphical toolkit or interface, 
such as  KDE, Gnome, Cocoa, MS Foundation, SWING, and terminals. 

2: Building around a specific language for macro and script 
programming.  

3: Adding features that are useful for specific development models. 
Web/HTML authoring requires upload tools, C/C++ requires make, Java 
might use Ant, lisp and python can use shells.  

> - isn't it a waste for newbies to evaluate a dozen editors / IDE's ?

I think that in many cases, the choices are constrained to two or 
three depending on what the _newbie_ wants to do, and what Desktop 
Environment they use.  For general editors on OS X, I'd suggest 
Smultron, or TextWrangler.  For KDE, kick the tires on Kate a bit.  
Cross-platform and portable? jEdit.  Text consoles? Emacs, vim or 
jed.

And if the newbie has some specific itch that needs to be scratched, 
there again the choices usually boil down to two or three best of 
breed IDEs, many of which are segregated by platform.  

 > just some thoughts,
> of a some months old newbie,
> Stef Mientki
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Regex Speed

[Kirk Sluder]

In article <[EMAIL PROTECTED]>,
 "John Machin" <[EMAIL PROTECTED]> wrote:

> Getting back to the "It would be nice ..." bit: yes, it would be nice
> to have even more smarts in re, but who's going to do it? It's not a
> "rainy Sunday afternoon" job :-)

Well, just as an idea, there is a portable C library for this at 
http://laurikari.net/tre/ released under LGPL.  If one is willing to 
give up PCRE extensions for speed, it might be worth the work to 
wrap this library using SWIG.

The cheap way in terms of programmer time is to pipe out to grep or 
awk on this one.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: RegExp performance?

[Kirk Sluder]

In article <[EMAIL PROTECTED]>,
 Christian Sonne <[EMAIL PROTECTED]> wrote:

> Thanks to all of you for your replies - they have been most helpful, and 
> my program is now running at a reasonable pace...
> 
> 
> I ended up using r"\b\d{9}[0-9X]\b" which seems to do the trick - if it 
> turns out to misbehave in further testing, I'll know where to turn :-P

Anything with variable-length wildcard matching (*+?) is going to 
drag your performance down. There was an earlier thread on this very 
topic.  Another stupid question is how are you planning on handling 
ISBNs formatted with hyphens for readability?

In general I've found the following factors to be critical in 
getting good performance from re:

1: Reducing the number of times you call re.match or re.search.
2: Reducing the number of bytes that re has to search through.
3: Minimizing the use of wildcards in the expression.

If you can pre-filter your input with string.find before running 
re.match you will improve performance quite a bit compared to 
running re expressions over all 10 pages.  I played around a bit and 
attached some example code below searching over 21K of text for the 
ISBN number.  testPrefilter() runs about 1/5th the execution time of 
line-by-line re calls or a single re call over a 21K string.  
Interestingly this ratio scales up to something as big as Mobey 
Dick.  

The searchLabels() functions below beats all the other functions by 
searching for "ISBN", or "International Book" and then using RE on 
the surrounding 500 bytes.  You might also try searching for 
"Copyright" or "Library of Congress" since most modern books will 
have it all on the same page.  

A caveat here is that this only works if you can find a reasonably 
unique string at or near what you want to find with re.  If you need 
to run re.search on every byte of the file anyway, this isn't going 
to help.  

---
timing test code
---

#!/usr/bin/env python

from timeit import Timer
import re

textString = """The text of a sample page using with an ISBN 10
number ISBN 0672328976 and some more text to compare."""

#add the full text of Mobey Dick to make the search functions
#work for their bread.
fileHandle= open("./mobey.txt")
junkText = fileHandle.readlines()
junkText.append(textString)
textString=''.join(junkText)
#print textString

#compile the regex
isbn10Re = re.compile(r"\b\d{9}[0-9X]\b")

def testPrefilter():
"""Work through a pre-loaded array running re only on lines 
containing ISBN""" 
for line in junkText:
#search for 'ISBN"
if (line.find('ISBN') > -1):
thisMatch = isbn10Re.search(line)
if thisMatch:
return thisMatch.group(0)
  
def testNofilter():
"""Run re.search on every line."""
for line in junkText:
#seaching using RE
thisMatch = isbn10Re.search(line)
if thisMatch:
return thisMatch.group(0)


def testFullre():
"""Run re.search on a long text string."""
thisMatch = isbn10Re.search(textString)
if thisMatch:
return thisMatch.group(0)

def searchLabels():
#identify some text that might be near an ISBN number.
isbnLabels=["ISBN",
   "International Book"]

#use the fast string.find method to locate those 
#labels in the text
isbnIndexes = [textString.find(x) for x in isbnLabels]

#exclude labels not found in the text.
isbnIndexes = [y for y in isbnIndexes if y > -1]

#run re.search on a 500 character string around the text label.
for x in isbnIndexes:
thisMatch=isbn10Re.search(textString[x-250:x+250])
return thisMatch.group(0)



#print searchLabels()
#print testPrefilter()
#print testNofilter()
t = Timer("testNofilter()","from __main__ import testNofilter")
print t.timeit(100)
u = Timer("testPrefilter()","from __main__ import testPrefilter")
print u.timeit(100)
v = Timer("testFullre()","from __main__ import testFullre")
print v.timeit(100)
w = Timer("searchLabels()", "from __main__ import searchLabels")
print w.timeit(100)
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: setting file permissions on a web server

[Kirk McDonald]

Daniel Nogradi wrote:
> I have next to zero experience with windows but as far as I know
> windows doesn't have file permissions at all (anyone, please correct
> me if I'm wrong :)) so in windows land it doesn't make any sense to
> "change file permissions". Even if it has some sort of a notion of
> file permissions I wouldn't know how that gets translated into unix.

This is getting a little off-topic, I admit, but the NT-derived versions 
of Windows do indeed have file permissions. The Windows-equivalent of 
chmod (and chown) is cacls ("Change Access Control Lists"):

http://www.ss64.com/nt/cacls.html

In essence, each file has an owner and a list of other users and groups, 
who may each have "Full control" (meaning they can change permissions), 
read-only, write-only, read-write, or no access.

Windows 95 and its descendants don't have permissions any more than DOS 
does. (Except with respect to Windows network file-sharing.)

(Heh, I checked just before posting this and someone beat me to it. 
Here's my post anyway.) :-)

-Kirk McDonald
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: newbie question: if var1 == var2:

[Kirk Strauser]

At 2008-11-29T04:02:11Z, Mel <[EMAIL PROTECTED]> writes:

> You could try
>
> for item in fname:
> item = item.strip()

This is one case where I really miss Perl's "chomp" function.  It removes a
trailing newline and nothing else, so you don't have to worry about losing
leading or trailing spaces if those are important to you.
-- 
Kirk Strauser
The Day Companies
--
http://mail.python.org/mailman/listinfo/python-list

Re: Do more imported objects affect performance

[Kirk Strauser]

At 2008-12-01T11:30:44Z, Nick Craig-Wood <[EMAIL PROTECTED]> writes:

> Importing the module is actualy slower...  If you import the name into
> your namespace then there is only one lookup to do.  If you import the
> module there are two.

Note that if you're importing the entire module but want to call a function
that many times, you can do something like:

import timeit
Timer = timeit.Timer
for _ in xrange(100):
    Timer
-- 
Kirk Strauser
The Day Companies
--
http://mail.python.org/mailman/listinfo/python-list

Re: newbie question: if var1 == var2:

[Kirk Strauser]

At 2008-12-11T17:24:44Z, rdmur...@bitdance.com writes:

> >>> '  ab c  \r\n'.rstrip('\r\n')
> '  ab c  '
> >>> '  ab c  \n'.rstrip('\r\n')
> '  ab c  '
> >>> '  ab c  '.rstrip('\r\n')
> '  ab c  '

I didn't say it couldn't be done.  I just like the Perl version better.
-- 
Kirk Strauser
The Day Companies
--
http://mail.python.org/mailman/listinfo/python-list

Re: newbie question: if var1 == var2:

[Kirk Strauser]

At 2008-12-11T19:49:23Z, Steve Holden  writes:

> ... and it's so hard to write
>
>  item = item[:-1]

It's easy - and broken.  Bad things happen if you're using something other
than '\n' for EOL.
-- 
Kirk Strauser
The Day Companies
--
http://mail.python.org/mailman/listinfo/python-list

Re: newbie question: if var1 == var2:

[Kirk Strauser]

At 2008-12-12T15:35:11Z, "J. Cliff Dyer"  writes:

> Python has a version equally good:
>
> def chomp(s):
> return s.rstrip('\r\n')
>
> You'll hardly miss Perl at all. ;)

I haven't missed Perl in years!  I just wish there was a basestring.stripeol
method because I seem to end up writing the inline version of "chomp" every
time I iterate across a file.
-- 
Kirk Strauser
The Day Companies
--
http://mail.python.org/mailman/listinfo/python-list

Re: Removing None objects from a sequence

[Kirk Strauser]

At 2008-12-12T15:51:15Z, Marco Mariani  writes:

> Filip Gruszczyński wrote:
>
>> I am not doing it, because I need it. I can as well use "if not elem
>> is None",

> I suggest "if elem is not None", which is not quite the same.

So what's the difference exactly?  "foo is not None" is actually surprising
to me, since "not None" is True.  "0 is True" is False, but "0 is not None"
is True.  Why is that?
-- 
Kirk Strauser
The Day Companies
--
http://mail.python.org/mailman/listinfo/python-list

Re: (Very Newbie) Problems defining a variable

[Kirk Strauser]

At 2008-12-12T18:12:39Z, "Tim Rowe"  writes:

> Is there a tidy way of making rates and thresholds local to get_rate,
> without recalculating each time? I suppose going object oriented is
> the proper way.
>
> #Py3k,UTF-8
>
> rates = {0: 0.006, 1: 0.0085, 25000: 0.0124, 5: 0.0149, 10: 
> 0.0173}
> thresholds = list(rates.keys())
> thresholds.sort()
> thresholds.reverse()
>
> def get_rate(balance):
> for threshold in thresholds:
> if balance >= threshold:
> return rates[threshold]
> else:
> return 0.0

How 'bout:

def get_rate(balance):
for threshold, rate in ((10, .0173),
(5, .0149),
(25000, .0124),
(1, .0085),
(0, .006)):
    if balance > threshold:
return rate
return .0
-- 
Kirk Strauser
The Day Companies
--
http://mail.python.org/mailman/listinfo/python-list

Re: (Very Newbie) Problems defining a variable

[Kirk Strauser]

At 2008-12-12T19:20:52Z, John Machin  writes:

> (1) you meant "if balance > threshold:"

balance >= threshold.  We both mistyped.  :-)

> (2) sequential search can be very fast if the sequence is in
> descending order of probability of occurence ... you might like to
> consider reversing the order

Actually, I just wanted to point out a simplified version of the exact same
algorithm.  Given enough RAM and if the speed was indeed critical, you could
turn that into a tuple of interest rates and jump straight to rate[balance]
in O(1).
-- 
Kirk Strauser
The Day Companies
--
http://mail.python.org/mailman/listinfo/python-list

Re: tricky nested list unpacking problem

[Kirk Strauser]

At 2008-12-15T19:06:16Z, Reckoner  writes:

> The problem is that I don't know ahead of time how many lists there are or
> how deep they go. In other words, you could have:

Recursion is your friend.

Write a function to unpack one "sublist" and call itself again with the new
list.  For instance, something like:

def unpack(pattern):
# Find the first subpattern to replace
# [...]
results = []
for number in subpattern:
results.append(pattern.replace(subpattern, number))
return results

Calling unpack([1,2,3,[5,6],[7,8,9]]) would look cause it to call
unpack([1,2,3,5,[7,8,9]]) and unpack([1,2,3,6,[7,8,9]]), compile the
results, and return them.
-- 
Kirk Strauser
The Day Companies
--
http://mail.python.org/mailman/listinfo/python-list

Re: tricky nested list unpacking problem

[Kirk Strauser]

At 2008-12-15T20:03:14Z, "Chris Rebert"  writes:

> You just need a recursive list-flattening function. There are many
> recipes for these. Here's mine:

>>>> flattened = flatten([1,2,3,[5,6,[10, 11]],7,[9,[1, 2, 3, 4, 5 ]]])
>>>> flattened
> [1, 2, 3, 5, 6, 10, 11, 7, 9, 1, 2, 3, 4, 5]
>>>> '-'.join(str(num) for num in flattened)
> '1-2-3-5-6-10-11-7-9-1-2-3-4-5'

He doesn't want to flatten them directly.  He's using [1,2,3] sort of like a
regular expression, so that 1,[2,3],4 means "1,2,4" or "1,3,4", not
"1,2,3,4".
-- 
Kirk Strauser
The Day Companies
--
http://mail.python.org/mailman/listinfo/python-list

Re: Append a new value to dict

[Kirk Strauser]

At 2008-10-13T13:14:15Z, [EMAIL PROTECTED] writes:

> jdd:
>> foo = {'bar': 'baz'}
>> foo.update({'quux': 'blah'})
>
> That creates a new dict, to throw it away. Don't do that.

I use that if I'm changing many values at once, eg:

foo.update({
'quux': 'blah',
'baz' : 'bearophile',
'jdd' : 'dict',
})

instead of:

foo['quux'] = 'blah'
foo['baz'] = 'bearophile'
foo['jdd'] = 'dict'

because it seems to more clearly indicate what I'm doing, and has fewer
opportunities for types.  Still, there is a performance penalty.  Running
"my way" 10,000,000 times took 8.7s, and "your way" only took 4.7s.  If
you're doing this in an inner loop, that may be significant.  While we're on
the subject, use keyword arguments to dict like:

foo.update(dict(quux='blah', baz='bearophile', jdd='dict'))

was *much* slower, at 11.8s.
-- 
Kirk Strauser
The Day Companies
--
http://mail.python.org/mailman/listinfo/python-list

Re: Suggestion for the PythonDevelopment for next version

[Kirk Strauser]

At 2008-10-13T16:11:26Z, azrael <[EMAIL PROTECTED]> writes:

> I know that. enumerate is a great function. But this way it always
> adds some complexity. I think that it is more better to give a man a
> better tool then to let him play with a not so good one. People like
> Python because of his simplicity in comparison with c++. Maybe People
> would like him even more it would be a bit more simple but the same
> powerfull.

"enumerate" doesn't add complexity: it adds specificity.  When you use it,
there is no question of intent.  You are saying, explicitly, that you're
going to want to use the index.

Your way adds an enormous amount of complexity in that it's magic behavior
with no obviously good implementation.  What if the objects in my list
already have an attribute named "index" and I don't want your loop
contstruct to overwrite it?  What if I never use enumerate as it is and
don't want to pay for the extra overhead of modify every object I'll ever
get from an iterator?

Thanks, but no.  One of the things I love about Python is its lack of magic.
-- 
Kirk Strauser
The Day Companies
--
http://mail.python.org/mailman/listinfo/python-list

Re: RegExp: "wontmatch"-function

[Kirk Strauser]

At 2008-10-13T16:40:07Z, [EMAIL PROTECTED] writes:

def nomatch(value):
return not(value == '' or pattern.match(value))

-- 
Kirk Strauser
The Day Companies
--
http://mail.python.org/mailman/listinfo/python-list

Re: Question about scope

[Kirk Strauser]

At 2008-10-24T01:08:12Z, Pat <[EMAIL PROTECTED]> writes:

> ---> myGlobals.py file:
>
> class myGlobals():
> remote_device_enabled = bool
>
> ---> my initialize.py file:
>
> from myGlobals import *
> def initialize():
> myGlobals.remote_device_enabled = True
>
> ---> my main.py file:
>
> import from myGlobals import *
> RDE =  myGlobals.remote_device_enabled
>
> def main():
> if RDE:# this will not give me the correct value
> process_device()

If you *really* want to organize your settings like this, may I suggest:

---> myGlobals.py file:

# This does nothing more than create a placeholder
remote_device_enabled = None

---> my initialize.py file:

import myGlobals
def initialize():
myGlobals.remote_device_enabled = True

---> my main.py file:

import myGlobals
import initialize
initialize.initialize()

def main():
if myGlobals.remote_device_enabled:
process_device()


-- 
Kirk Strauser
The Day Companies
--
http://mail.python.org/mailman/listinfo/python-list

Re: Python barcode decoding

[Kirk Strauser]

At 2008-10-24T17:05:25Z, Robocop <[EMAIL PROTECTED]> writes:

> Does anyone know of any decent (open source or commercial) python
> barcode recognition tools or libraries.  I need to read barcodes from
> pdfs or images, so it will involve some OCR algorithm.  I also only
> need to read the code 93 symbology, so it doesn't have to be very
> fancy.  The most important thing to me is that it outputs in some
> python friendly way, or ideally that it is written in python.  Any
> tips would be great!

How precise are these images?  I mean, are they computer-generated, or
scanned in from other material?

If they're nice and crisp, I wrote a simple and pretty quick decoder at 
http://pypi.python.org/pypi/Daycos/1.0 .  Look in the
Daycos.Imageproc.Barcode module.
-- 
Kirk Strauser
The Day Companies
--
http://mail.python.org/mailman/listinfo/python-list

Re: Python/Numeric users be aware!

[Kirk Strauser]

At 2008-10-29T17:53:43Z, Benyang <[EMAIL PROTECTED]> writes:

> It is totally screwed up on 64-bit linux machines:
> [1 1 1 1 1 1 1 1 1 1]

And on 64-bit FreeBSD machines.
-- 
Kirk Strauser
The Day Companies
--
http://mail.python.org/mailman/listinfo/python-list

Re: Printing with interspersed element

[Kirk Strauser]

At 2008-10-30T21:10:09Z, "Paulo J. Matos" <[EMAIL PROTECTED]> writes:

> Thanks for the tip but that has an issue when dealing with potentially
> millions of objects. You are creating a string in memory to then dump
> to a file [or screen] while you could dump to the file [or screen] as
> you go through the original string. Right?

How about:

def alternator(lst, sep):
for index, item in enumerate(lst):
if index:
yield sep
yield item

for item in alternator(list_of_objects, 10):
print item,

-- 
Kirk Strauser
The Day Companies
--
http://mail.python.org/mailman/listinfo/python-list

Re: Weird behavior with lexical scope

[Kirk Strauser]

At 2008-11-06T16:57:39Z, mrstevegross <[EMAIL PROTECTED]> writes:

> class Outer:
>   class Inner:
> def __init__(self):
>   pass
>   def __init__ (self):
> a = Inner()
> Outer()

Try instead:

class Outer:
def __init__(self):
a = self.Inner()


-- 
Kirk Strauser
The Day Companies
--
http://mail.python.org/mailman/listinfo/python-list

Re: Python and Its Libraries--Who's on First?

[Kirk Strauser]

At 2008-11-17T11:44:00Z, "W. eWatson" <[EMAIL PROTECTED]> writes:

> See the post by Chris R.

In general, it is incumbent on the asker to provide additional information
as needed, rather than being the job of the would-be answerer to go
searching for it.
-- 
Kirk Strauser
--
http://mail.python.org/mailman/listinfo/python-list