Re: reading file to list

[Xah Lee]

On Jan 17, 10:25 am, Tino Wildenhain  wrote:
> > [[int(x) for x in line.split()] for line in open("blob.txt")]

Nice (python code).

Few comments:

• the above code is borderline of atypical. e.g. it is not a average
python code would produce or one'd seen in corporate python code.

• voodoo like the above makes me dislike python. To me, the one
advantage of python is its clarity enforced by its syntax.
Specifically, the forced indendation and quite simple semantics.
However, the way i've seen Guido's propensities and how python 3 is
moving to, it is becoming more mumbo jumbo of computer sciency OOP
jargons with syntax soup. (with iterators, enumerators, list
comprehension... shits forced upon the users)

The above line illustrate well the ad hoc syntax soup nature python is
moving into.

Further readings:

• Perl-Python Tutorial: List Comprehension
  http://xahlee.org/perl-python/list_comprehension.html

• Lambda in Python 3000
  http://xahlee.org/perl-python/python_3000.html

  Xah
∑ http://xahlee.org/

☄
--
http://mail.python.org/mailman/listinfo/python-list

Re: tp_base, ob_type, and tp_bases

[Carl Banks]

On Jan 17, 8:12 am, Jeff McNeil  wrote:
> On Jan 17, 11:09 am, Jeff McNeil  wrote:
>
>
>
> > On Jan 17, 10:50 am, "Martin v. Löwis"  wrote:
>
> > > > So, the documentation states that ob_type is a pointer to the type's
> > > > type, or metatype. Rather, this is a pointer to the new type's
> > > > metaclass?
>
> > > That's actually the same. *Every* ob_type field points to the object's
> > > type, e.g. for strings, integers, tuples, etc. That includes type
> > > objects, where ob_type points to the type's type, i.e. it's meta-type,
> > > also called metaclass (as "class" and "type" are really synonyms).
>
> > > > Next, we have tp_base.  That's defined as "an optional pointer to a
> > > > base type from which type properties are inherited."  The value of
> > > > tp_base is then added to the tp_bases tuple.  This is confusing me. On
> > > > the surface, it sound as though they're one in the same?
>
> > > (I don't understand the English "one in the same" - interpreting it
> > > as "as though they should be the same")
>
> > > No: tp_bases is a tuple of all base types (remember, there is multiple
> > > inheritance); tp_base (if set) provides the first base type.
>
> > > > I *think* (and dir() of a subclass of type seems to back it up), that
> > > > tp_base is only at play when the object in question inherits from
> > > > type?
>
> > > No - it is the normal case for single inheritance. You can leave it
> > > NULL, which means you inherit from object.
>
> > > Regards,
> > > Martin
>
> > Thank you! It was tp_base that was confusing me.  The tp_bases member
> > makes sense as Python supports multiple inheritance.  It wasn't
> > immediately clear that tp_base is there for single inheritance
> > reasons. It's all quite clear now.
>
> > Is that an optimization of sorts?
>
> Well, maybe not specifically for single inheritance reasons, I just
> didn't see an immediate reason to keep a separate pointer to the first
> base type.


The reason you need a separate tp_base is because it doesn't
necessarily point to the first base type; rather, it points to the
first base type that has added any fields or slots to its internal
layout (in other words, the first type with a tp_basicsize > 8, on 32-
bit versions).  I believe this is mainly for the benefit of Python
subclasses that define their own slots.  The type constructor will
begin adding slots at an offset of tp_base->tp_basicsize.

To see an example, int objects have a tp_basicsize of 12 (there are 4
extra bytes for the interger).  So if you multiply-inherit from int
and a Python class, int will always be tp_base.

class A(object): pass

class B(int,A): pass
print B.__base__ # will print 

class C(A,int): pass
print C.__base__ # will print 


A related issue is that you can't multiply inherit from two types that
have tp_basicsize > 8 unless one of them inherits from the other.
There can be only one tp_base.  For instance:

class D(int,tuple): pass # will raise TypeError

class E(object):
__slots__ = ['a','b']

class F(object):
__slots__ = ['c','d']

class G(E,G): pass # will raise TypeError

class H(E,int): pass  # will raise TypeError



Here's a bit more background (and by "a bit" I mean "a lot"):

In 32-bit Python, objects of types defined in Python are usually only
16 bytes long.  The layout looks like this.

instance dict
weak reference list
reference count
ob_type

The reference count, which is always the thing that the actual
PyObject* points at, isn't actually the first item in the object's
layout.  The dict and weakref list are stored at a lower address.
(There's a reason for it.)

If a Python class defines any __slots__, the type constructor will add
the slots to the object's layout.

instance dict (if there is one)
weak reference list (if there is one)
reference count
ob_type
slot
slot
slot

Note that, because you defined __slots__, the object might not have an
instance dict or weak reference list associated with it.  It might,
though, if one of the base classes had defined an instance dict or
weakref list.  Or, it might not, but then a subclass might have its
onw instance dict or weakref list.  That's why these guys are placed
at a lower address: so that they don't interfere with the layout of
subclasses.  A subclass can add either more slots or a dict.

Object of types defined in C can have arbitrary layout.  For instance,
it could have a layout that looks like this:

reference count
ob_type
PyObject* a
PyObject* b
long c
float d
instance dict

A problem with Python slots and C fields is that, if you want to
inherit from a type with a non-trivial object layout, aside from the
dict and weakrefs, all of the subtypes have to maintain the same
layout (see Liskov substitutability for rationale).

Subtypes can add their own fields or slots if they want, though.  So,
if a Python subtype wants to define its own slots on top of a type
with a non-trivial object layout, it has to know which base has the
largest layout so that it doesn't use the same memory for

Relax Syntax for Augmented Arithmetic?

[andrew cooke]

Context - http://docs.python.org/3.0/reference/datamodel.html?highlight=data
model#object.__iadd__

Just a suggestion I thought I'd throw out...  There's a restriction in
the language implementation on exactly what can go the left of an
augmented arithmetic expression.

For example:
>>> a = 3
>>> a **= 2

is ok, but:
>>> class Foo():
...   def __init__():
... self.a = 3
...   def __ipow__(self, x):
... self.a **= x
...
>>> Foo() **= 2
  File "", line 1
SyntaxError: illegal expression for augmented assignment

Now unless I've done something stupid above (always a possibility :o)
the implementation seems a bit strict (is it really a *syntax* error?
- I am not sure exactly what the restriction is).

This may seems like a small issue, but operators can really help with
making embedded DSLs in Python - they give quite a bit of "wiggle
room" to invent a syntax that is compact and intuitive.  The
restriction above cuts into that (OK, so it's still a small
issue... :o)

Cheers,
Andrew
--
http://mail.python.org/mailman/listinfo/python-list

Re: dynamic module import?

[James Stroud]

Lawson Hanson wrote:


So is there any way to get Python to import the named module
without just doing "from dummy import *", because I will
not know that the user wants to use the "dummy" module
until run-time ... I'm trying to import control data
for different run scenarios which will be defined in
differently named Python modules which the user will
specify at run-time with a command-line option

And help with this would be most appreciated


For the sake of humanity, I must try to talk you out of this. Well, it's 
not that serious, but dynamic import confuses programs that inspect code 
like py2exe, etc. I think it is likely that you will find the day that 
you regret trying so hard to 'import *' dynamically. A more maintainable 
way is to simply map the command line argument to an import statement 
and keep your namespaces clean:


"""importer_module"""
import sys
import dummy
import bonafide
modules = {"dummy" : dummy, "bonafide" : bonafide}
module = modules[sys.argv[1]]


If you have several modules that themselves might need the conditional 
imports, simply put the import statements in a separate module (e.g. 
"importer_module" and do something like



"""another_module"""
from importer_module import module


If you simply don't want to import a bunch of modules, use an if/then 
statement. In any event, I advise you to not design your code or usage 
around dynamic imports using the __import__() statement or exec().


James



--
James Stroud
UCLA-DOE Institute for Genomics and Proteomics
Box 951570
Los Angeles, CA 90095

http://www.jamesstroud.com
--
http://mail.python.org/mailman/listinfo/python-list

Re: Relax Syntax for Augmented Arithmetic?

[Chris Rebert]

On Sun, Jan 18, 2009 at 2:56 AM, andrew cooke  wrote:
> Context - http://docs.python.org/3.0/reference/datamodel.html?highlight=data
> model#object.__iadd__
>
> Just a suggestion I thought I'd throw out...  There's a restriction in
> the language implementation on exactly what can go the left of an
> augmented arithmetic expression.
>
> For example:
 a = 3
 a **= 2
>
> is ok, but:
 class Foo():
> ...   def __init__():
> ... self.a = 3
> ...   def __ipow__(self, x):
> ... self.a **= x
> ...
 Foo() **= 2
>  File "", line 1
> SyntaxError: illegal expression for augmented assignment
>
> Now unless I've done something stupid above (always a possibility :o)
> the implementation seems a bit strict (is it really a *syntax* error?
> - I am not sure exactly what the restriction is).

IIRC, you can only assign to:
- variables (x = ...)
- attributes (x.y = ...)
- elements (x[y] = ...)

Anything else doesn't make sense to assign to because it's not a
"storage box" so to speak. There's no way to work out what is meant.
In your case, you're assigning to a *value*, specifically a new
instance of the Foo class, which is nonsensical; instead of a "box",
you're trying to assign to a "value", something that gets stored in
boxes. By comparison, '2 = 5' and '[1,2] = 7' would seem to have some
sort of meaning under your system, which IMHO seems preposterous.
Now true, you are using augmented assignment, which in certain cases
is translated to a method call, but in principle the augmented
assignment (e.g. x += y) should have roughly the same effect as the
non-augmented equivalent (x = x + y), and the fact that a method call
is involved is merely an implementation detail of sorts.
Therefore, Python requires you to rewrite the code in some other way
that makes your intentions more clear. For instance, why not use the
<< operator instead?

Cheers,
Chris

-- 
Follow the path of the Iguana...
http://rebertia.com
--
http://mail.python.org/mailman/listinfo/python-list

Re: Relax Syntax for Augmented Arithmetic?

[andrew cooke]

> Therefore, Python requires you to rewrite the code in some other way
> that makes your intentions more clear. For instance, why not use the
> << operator instead?

Right, but you're guessing what the context is.  Within a DSL it often
makes a lot of sense to use operators for reasons that weren't
originally intended.

You even make the same case yourself indirectly.  The same argument
you make could be made to say that << should only operator on values
that can be shifted.  Now thankfully there is no way to test for that,
so there is no restriction and, consequently, it is now widely
accepted that no-one (even people arguing the case for constraints!)
think it odd to use << for something other than its initial use.

Obviously this kind of discussion has gone on since languages were
first invented - it's the "how much rope" argument.  So rather than
continue down that road I would just like to say that this feels like
an inconsistency.  The other operators are *not* as restricted and
this is making my life harder.

It may sound crazy, but this may force me to use * and ** instead (the
context is a language feature related to *args and **kargs, so the *
and ** help convey the meaning).  And they have a much much stronger
meaning to users, which will make my DSL harder to understand.  So in
this case a blunt knife is making life harder.

Andrew
--
http://mail.python.org/mailman/listinfo/python-list

Re: Dynamic Loading Modules

[Laszlo Nagy]

Riley Porter írta:

Hello all,

This is the first time I have posted to this group.  That being said 
if I am in the wrong place for this kind of support please let me know. 


OK,

So I am writing a log parsing program and wish to allow for the 
community to write "parsers".  Basically, what I have in place now is 
a "modules" directory that contains .py files that are going to be 
used to parse specific log types. 

The question is, is there a way to dynamically import whatever is in 
that modules directory and make is to the import name is actually the 
file name..
So after the modules directory is 'os.listdir' and it returns a 
listing of modules (actually just .py files) IE: ng1-fw.py, 
cisco_asa_fw.py, etc... I can just in the next line of code call:


ng1-fw.Parse(long_file)  #assuming the ng1-fw.py file has a Parse Function

right now using imp I got it to where i can say like:
x =  imp.load_source(md5.new(code_path).hexdigest(), code_path, f)  
#assuming that I gave it the path and filename for ng1-fw.py

I can do this:

x.Parse(log_file)

But what I am really after is the ability to do this:
ng1-fw.Parse(log_file) 

The reason again behind this is so that there is no need to "hard 
code" parser file names. 

Did I make myself clear?  Please let me know if I need to explain that 
better.

What you need is the imp module:

http://docs.python.org/library/imp.html

This module provides an interface to the mechanisms used to implement 
the import  
statement. It defines the following constants and functions:



You might find that using classes instead of modules is more handy.

Regards,

  Laszlo


--
http://mail.python.org/mailman/listinfo/python-list

Re: dynamic module import?

[alex23]

On Jan 17, 3:55 pm, Steven D'Aprano  wrote:
> Both very good points, but consider that you're not comparing apples with
> apples.
>
> >>> __import__("os", fromlist=["system"])
>  >>> system
> Traceback (most recent call last):
>   File "", line 1, in 
> NameError: name 'system' is not defined

I must confess I've rarely had a need to use __import__ and don't
think I've ever used the fromlist arg. I'm confused, though, because
the docstring states:

 The fromlist should be a list of names to emulate ``from name
import ...''

But it also states that __import__ always returns a module, so I'm
utterly confused as to the purpose of fromlist, or how to inject the
specified entries into the calling namespace. If anyone could explain
this for me, I'd really appreciate it.

> I mention this only to be pedantic, because I agree with your point that
> exec can introduce security issues, and be significantly slower.

Correcting misinformation isn't pedantry, especially when I've learned
something :)
--
http://mail.python.org/mailman/listinfo/python-list

Re: what's the point of rpython?

[andrew cooke]

Since this is a PyPy bashing thread, maybe it's an appropriate place
to suggest that the project has got a little bit waylaid by exploring
cool things instead of releasing a useful final result?

I am not questioning rpython directly - the case for something like
that is obvious.  But there's a question of balance.  It's possible to
go on building ever more complex systems which are theoretically
justified, but which postpone ever finishing the job.  At some point
there has to be a "good enough".

To some extent I am playing devil's advocate here, but as an outside
who looked at PyPy a while back, my uninformed and naive impression
was that the project was suffering from the kid of issues I have
caricatured above

Andrew

PS I guess you are aware of worse is better etc?  I think this may
also be a US/Euro culture issue...
--
http://mail.python.org/mailman/listinfo/python-list

Re: Totally confused by the str/bytes/unicode differences introduced in Pythyon 3.x

[John Machin]

On Jan 18, 2:02 pm, Terry Reedy  wrote:
> John Machin wrote:
> > On Jan 18, 9:10 am, Terry Reedy  wrote:
> >> Martin v. Löwis wrote:
> > Does he intend to maintain two separate codebases, one 2.x and the
> > other 3.x?
>  I think I have no other choice.
>  Why? Is theoretically possible to maintain an unique code base for
>  both 2.x and 3.x?
> >>> That is certainly possible! One might have to make tradeoffs wrt.
> >>> readability sometimes, but I found that this approach works quite
> >>> well for Django. I think Mark Hammond is also working on maintaining
> >>> a single code base for both 2.x and 3.x, for PythonWin.
> >> Where 'single codebase' means that the code runs as is in 2.x and as
> >> autoconverted by 2to3 (or possibly a custom comverter) in 3.x.
>
> >> One barrier to doing this is when the 2.x code has a mix of string
> >> literals with some being character strings that should not have 'b'
> >> prepended and some being true byte strings that should have 'b'
> >> prepended.  (Many programs do not have such a mix.)
>
> >> One approach to dealing with string constants I have not yet seen
> >> discussed here is to put them all in separate file(s) to be imported.
> >> Group the text and bytes separately.  Them marking the bytes with a 'b',
> >> either by hand or program would be easy.
>
> > (1) How would this work for somebody who wanted/needed to support 2.5
> > and earlier?
>
> See reposts in python wiki, one by Martin.

Most relevant of these is Martin's article on porting Django, using a
single codebase. The """goal is to support all versions that Django
supports, plus 3.0""" -- indicating that it supports at least 2.5,
which won't eat b"blah" syntax. He is using 2to3, and handles bytes
constants by """django.utils.py3.b, which is a function that converts
its argument to an ASCII-encoded byte string. In 2.x, it is another
alias for str; in 3.x, it leaves byte strings alone, and encodes
regular (unicode) strings as ASCII. This function is used in all
places where string literals are meant as bytes, plus all cases where
str() was used to invoke the default conversion of 2.x."""

Very similar to what I expected. However it doesn't answer my question
about how your "move byte strings to a separate file, prepend 'b', and
import the separate file" strategy would help ... and given that 2.5
and earlier will barf on b"arf", I don't expect it to.

> > (2) Assuming supporting only 2.6 and 3.x:
>
> > Suppose you have this line:
> > if binary_data[:4] == "PK\x03\x04": # signature of ZIP file
>
> > Plan A:
> > Change original to:
> > if binary_data[:4] == ZIPFILE_SIG: # "PK\x03\x04"
> > Add this to the bytes section of the separate file:
> > ZIPFILE_SIG = "PK\x03\x04"
> > [somewhat later]
> > Change the above to:
> > ZIPFILE_SIG = b"PK\x03\x04"
> > [once per original file]
> > Add near the top:
> > from separatefile import *
>
> > Plan B:
> > Change original to:
> > if binary_data[:4] == ZIPFILE_SIG: # "PK\x03\x04"
> > Add this to the separate file:
> > ZIPFILE_SIG = b"PK\x03\x04"
> > [once per original file]
> > Add near the top:
> > from separatefile import *
>
> > Plan C:
> > Change original to:
> > if binary_data[:4] == b"PK\3\4": # signature of ZIP file
>
> > Unless I'm gravely mistaken, you seem to be suggesting Plan A or some
> > variety thereof -- what advantages do you see in this over Plan C?
>
> For 2.6 only (which is much easier than 2.x), do C.  Plan A is for 2.x
> where C does not work.

Excuse me? I'm with the OP now, I'm totally confused. Plan C is *not*
what you were proposing; you were proposing something like Plan A
which definitely involved a separate file.

Why won't Plan C work on 2.x (x <= 5)? Because the 2.X will b"arf".
But you say Plan A is for 2.x -- but Plan A involves importing the
separate file which contains and causes b"arf" also!

To my way of thinking, one obvious DISadvantage of a strategy that
actually moves the strings to another file (requiring invention of a
name for each string (that doesn't have one already) so that it can be
imported is the amount of effort and exposure to error required to get
the same functional result as a strategy that keeps the string in the
same file ... and this disadvantage applies irrespective of what one
does to the string: b"arf", Martin's b("arf"), somebody else's _b
("arf") [IIRC] or my you-aint-gonna-miss-noticing-this-in-the-code
BYTES_LITERAL("arf").

Cheers,
John
--
http://mail.python.org/mailman/listinfo/python-list

Am I interacting with the database correctly?

[Hussein B]

Hey,
I'm new with database interactions in Python and I'm not sure if I'm
handling the cursor and transactions correctly:

cursor = db.cursor(MySQLdb.cursors.DictCursor)
cursor.execute(flate_rate_pkgs_sql)
rows = cursor.fetchall()
#I have for loop here to iterate over rows
   cursor.execute()
   rows = cursor.fetchall()
   # some more cursor.execute() calls but only SQL select statements
   # here is another for loop that contains try block
  # here are cursor.execute() calls, both insert and update
  db.commit()
  # in the except code block, I use db.rollback()

As you see, my script contains only one db object and one cursor
object and both the db and cursor objects are used multiple times, it
is ok?
As you might figured, this is a script for reports :)
Thanks.

--
http://mail.python.org/mailman/listinfo/python-list

Re: Relax Syntax for Augmented Arithmetic?

[Chris Rebert]

On Sun, Jan 18, 2009 at 3:42 AM, andrew cooke  wrote:
>> Therefore, Python requires you to rewrite the code in some other way
>> that makes your intentions more clear. For instance, why not use the
>> << operator instead?
>
> Right, but you're guessing what the context is.  Within a DSL it often
> makes a lot of sense to use operators for reasons that weren't
> originally intended.
>
> You even make the same case yourself indirectly.  The same argument
> you make could be made to say that << should only operator on values
> that can be shifted.  Now thankfully there is no way to test for that,
> so there is no restriction and, consequently, it is now widely
> accepted that no-one (even people arguing the case for constraints!)
> think it odd to use << for something other than its initial use.
>
> Obviously this kind of discussion has gone on since languages were
> first invented - it's the "how much rope" argument.  So rather than
> continue down that road I would just like to say that this feels like
> an inconsistency.  The other operators are *not* as restricted and
> this is making my life harder.

Indeed. Python happens to in this case draw the line at using the
augmented assignment operators for non-assignment. I personally see
this as reasonable because the = symbol has a consistent meaning in
Python (assignment) whereas the other plain operators, as you bring
up, consistently have no predetermined meaning; but I do agree that it
is in a sense an arbitrary restriction, like many programming language
design choices. However, Python was not explicitly designed for
creating DSLs, so it's kinda odd to complain about something Python
never claimed to support in the first place (although I do favor the
DSL in general-purpose-PL paradigm).

> It may sound crazy, but this may force me to use * and ** instead (the
> context is a language feature related to *args and **kargs, so the *
> and ** help convey the meaning).  And they have a much much stronger
> meaning to users, which will make my DSL harder to understand.  So in
> this case a blunt knife is making life harder.

Perhaps if you explained your particular predicament in more depth,
someone might be able to offer a workable suggestion.

Cheers,
Chris

-- 
Follow the path of the Iguana...
http://rebertia.com
--
http://mail.python.org/mailman/listinfo/python-list

Re: Relax Syntax for Augmented Arithmetic?

[andrew cooke]

Not sure if you were saying this, but the underlying technical reason
for this issue is that they are treated as assignment rather than
operators in the language spec -
http://docs.python.org/3.0/reference/simple_stmts.html#augmented-assignment-statements

I think this explains why they are not listed in the operator
precedence table http://docs.python.org/3.0/reference/expressions.html#summary

I think that's unfortunate in a language with mutable objects, but it
makes the decision seem much less arbitrary...

Cheers,
Andrew
--
http://mail.python.org/mailman/listinfo/python-list

Re: Relax Syntax for Augmented Arithmetic?

[andrew cooke]

On Jan 18, 9:01 am, Chris Rebert  wrote:
> Indeed. Python happens to in this case draw the line at using the
> augmented assignment operators for non-assignment. I personally see
> this as reasonable because the = symbol has a consistent meaning in
> Python (assignment) whereas the other plain operators, as you bring
> up, consistently have no predetermined meaning;

my argument was that *= is not treated as = and *, but as a completely
new operator (the docs even say that the implementation need not
return self which suggests some pretty extreme semantics were
envisaged).  however, as i've just commented elsewhere, this
commitment to operators was only half-baked because they are parsed as
assignments.

anyway, to reply to your comment - *= is not predetermined.  it is
determined by __imul__ which is user-definable.

> but I do agree that it
> is in a sense an arbitrary restriction, like many programming language
> design choices. However, Python was not explicitly designed for
> creating DSLs,

python is a general programming language.  as far as i can make any
sense at all of your argument it seems to be "you are asking for
change, but this is not how the current system works".  to which the
obvious answer is: if it did work that way i wouldn't be asking for
change.

andrew
--
http://mail.python.org/mailman/listinfo/python-list

Re: Relax Syntax for Augmented Arithmetic?

[Marc 'BlackJack' Rintsch]

On Sun, 18 Jan 2009 04:24:04 -0800, andrew cooke wrote:

> my argument was that *= is not treated as = and *, but as a completely
> new operator (the docs even say that the implementation need not return
> self which suggests some pretty extreme semantics were envisaged).

What do you mean by "suggests … extreme semantics"?  Most natural thing 
is to use numbers and there you *have* to be able to return something 
different than `self` to get anything useful.  For instance:

 n *= 3

with `n` bound to a number different from zero can't return `self` from 
`__imul__`.

Ciao,
Marc 'BlackJack' Rintsch
--
http://mail.python.org/mailman/listinfo/python-list

is there something like a module decorator ?

[Stef Mientki]

hello,

I wonder if there's something like a module decorator.
I could use it in debugging a large highly dynamical program.

thanks,
Stef Mientki
--
http://mail.python.org/mailman/listinfo/python-list

Re: Relax Syntax for Augmented Arithmetic?

[andrew cooke]

On Jan 18, 9:40 am, Marc 'BlackJack' Rintsch  wrote:
> On Sun, 18 Jan 2009 04:24:04 -0800, andrew cooke wrote:
> > my argument was that *= is not treated as = and *, but as a completely
> > new operator (the docs even say that the implementation need not return
> > self which suggests some pretty extreme semantics were envisaged).
>
> What do you mean by "suggests … extreme semantics"?  Most natural thing
> is to use numbers and there you *have* to be able to return something
> different than `self` to get anything useful.  For instance:
>
>  n *= 3
>
> with `n` bound to a number different from zero can't return `self` from
> `__imul__`.

in your example, n is not a number, it is a mutable variable, and its
value changes.

when n is an instance implementing __imul__ the natural analogue is
that the internal state of the instance changes.

either i have misundertstood you, or you have misunderstood __imul__,
or you are treating = as equality, or maybe you are thinking of a pure
language that creates new instances?  python is impure.

anyway, my original request is moot.  i was assuming that this was a
"capricious" restriction.  in fact it's related to what i thought were
operators actually being assignments, and so no change is possible
(until python 4.0 when guido will finally see the light and move to s-
expressions, at which point everyone will stop using the language ;o)

andrew
--
http://mail.python.org/mailman/listinfo/python-list

Re: Relax Syntax for Augmented Arithmetic?

[andrew cooke]

On Jan 18, 9:56 am, andrew cooke  wrote:

> either i have misundertstood you

ah, i see your point.  sorry, andrew
--
http://mail.python.org/mailman/listinfo/python-list

is there something like a module decorator ?

[Stef Mientki]

hello,

I wonder if there's something like a module decorator.
I could use it in debugging a large highly dynamical program.

thanks,
Stef Mientki
--
http://mail.python.org/mailman/listinfo/python-list

Re: is there something like a module decorator ?

[Diez B. Roggisch]

Stef Mientki schrieb:

hello,

I wonder if there's something like a module decorator.
I could use it in debugging a large highly dynamical program.


No, there isn't. This has been discussed a while ago:

http://groups.google.de/group/comp.lang.python/browse_thread/thread/215216a1e13ba2c6

Diez
--
http://mail.python.org/mailman/listinfo/python-list

Re: is there something like a module decorator ?

[Stef Mientki]

Diez B. Roggisch wrote:

Stef Mientki schrieb:

hello,

I wonder if there's something like a module decorator.
I could use it in debugging a large highly dynamical program.


No, there isn't. This has been discussed a while ago:

http://groups.google.de/group/comp.lang.python/browse_thread/thread/215216a1e13ba2c6 



Diez
--
http://mail.python.org/mailman/listinfo/python-list

thanks for the information.
cheers,
Stef
--
http://mail.python.org/mailman/listinfo/python-list

Re: Relax Syntax for Augmented Arithmetic?

[andrew cooke]

http://bugs.python.org/issue4986

Sorry for the noise,
Andrew
--
http://mail.python.org/mailman/listinfo/python-list

Re: Relax Syntax for Augmented Arithmetic?

[Aaron Brady]

On Jan 18, 6:56 am, andrew cooke  wrote:
> On Jan 18, 9:40 am, Marc 'BlackJack' Rintsch  wrote:
>
> > On Sun, 18 Jan 2009 04:24:04 -0800, andrew cooke wrote:
> > > my argument was that *= is not treated as = and *, but as a completely
> > > new operator (the docs even say that the implementation need not return
> > > self which suggests some pretty extreme semantics were envisaged).
>
> > What do you mean by "suggests … extreme semantics"?  Most natural thing
> > is to use numbers and there you *have* to be able to return something
> > different than `self` to get anything useful.  For instance:
>
> >  n *= 3
>
> > with `n` bound to a number different from zero can't return `self` from
> > `__imul__`.
>
> in your example, n is not a number, it is a mutable variable, and its
> value changes.
>
> when n is an instance implementing __imul__ the natural analogue is
> that the internal state of the instance changes.
>
> either i have misundertstood you, or you have misunderstood __imul__,
> or you are treating = as equality, or maybe you are thinking of a pure
> language that creates new instances?  python is impure.
>
> anyway, my original request is moot.  i was assuming that this was a
> "capricious" restriction.  in fact it's related to what i thought were
> operators actually being assignments, and so no change is possible
> (until python 4.0 when guido will finally see the light and move to s-
> expressions, at which point everyone will stop using the language ;o)
>
> andrew

Not sure if this ties in, but:

>>> ['a'].__imul__(2)
['a', 'a']
--
http://mail.python.org/mailman/listinfo/python-list

Re: Does Python really follow its philosophy of "Readability counts"?

[Russ P.]

Wow! That was an instant classic! I just have a couple of points to
add.

The suggestion was made (not by you) that data hiding is worthless
because it can be defeated anyway. According to that kind of
reasoning, locks are worthless because they can be picked, cut, or
just blown off. I know that the lock on my front door cannot stop
someone who is determined to get into my house, but I think I'll keep
it anyway. I'm just irrational that way, I guess.

As I said before, I don't know if enforced data hiding can be added to
Python without ruining the language (or if it can be added at all, for
that matter). But if it can, I think the cleanest and most elegant
syntax would be to add the keyword "private" or "priv" and use it
essentially the same way it is used in Java, C++, and now Scala. That
would eliminate the need for leading underscores, which I personally
find tacky.

Since new attributes can be added outside of the constructor, the
private declaration would also have to be allowed outside the
constructor and for existing attributes.

One more thing. If an airplane is going to flip upside down and fly
straight into the ground, the passengers may as well be nuns. They are
better prepared for the result.

--
http://mail.python.org/mailman/listinfo/python-list

Re: what's the point of rpython?

[Matimus]

The goals are listed here: 
http://codespeak.net/pypy/dist/pypy/doc/architecture.html

Speed is mentioned, but as a secondary concern. The main goal seems to
be to create a vehicle into exploring the concept of dynamic languages
themselves. If that seems amorphous then it is because it is a
research project.

Matt
--
http://mail.python.org/mailman/listinfo/python-list

s=ascii(hexlify(urandom(10)))

[gert]

I expected that py3 did not converted the b'...' indication too ?

b'afc76815e3fc429fa9d7'

--
http://mail.python.org/mailman/listinfo/python-list

output problem

[Jean-Paul VALENTIN]

Feature? the output of below Hello program he0.py started from command
line looks as follows:
F:\prompt>he0.py
Hello
F:\prompt>

'Hello' was sent with sys.stdout.write, so "without newline".
But why cannot be output (more logically):
F:\prompt>he0.py
HelloF:\prompt>

Is it normal? Is there any means or workaround to obtain that output if
I wish??

__
#!/usr/bin/python
#-*- coding: iso-8859-15 -*-
import sys
sys.stdout.write('Hello')

 
About Ingenico: Ingenico is the world's leading provider of payment solutions, 
with over 15 million terminals deployed across the globe. Delivering the very 
latest secure electronic payment technologies, transaction management and the 
widest range of value added services, Ingenico is shaping the future direction 
of the payment solutions market. Leveraging on its global presence and local 
expertise, Ingenico is reinforcing its leadership by taking banks and 
businesses beyond payment through offering comprehensive solutions, a true 
source of differentiation and new revenues streams.
 This message may contain confidential and/or privileged information. If you 
are not the addressee or authorized to receive this for the addressee, you must 
not use, copy, disclose or take any action based on this message or any 
information herein. If you have received this message in error, please advise 
the sender immediately by reply e-mail and delete this message. Thank you for 
your cooperation.
 P Please consider the environment before printing this e-mail
 
 
--
http://mail.python.org/mailman/listinfo/python-list

Re: English-like Python

[The Music Guy]

Wow, impressive responses.

It sounds like the general consensus is that English would not be a good
choice for programming even if there were an interpreter capable of
turning human language into machine language. But that makes sense; even
English professionals have trouble understanding each other sometimes.
Until that problem is somehow overcome, there's not much hope of
computers to overcome it.


--
http://mail.python.org/mailman/listinfo/python-list

bin = FieldStorage(fp=environ['wsgi.input'], environ=environ)

[gert]

in python3.0 this does not work

File "/usr/python/lib/python3.0/email/feedparser.py", line 99, in push
data, self._partial = self._partial + data,
TypeError: Can't convert 'bytes' object to str implicitly

So what do i need to wrap around environ['wsgi.input'] so it does
work ?
--
http://mail.python.org/mailman/listinfo/python-list

Re: PyQt, Cannot send events to objects owned by a different thread?

[icejobjob]

AE•X‰ÍŠúA‚»‚ê‚à’´AE•X‰ÍŠú‚ÉŽ„‚½‚¿‚Í’¼–Ê‚µ‚Ä‚¢‚Ü‚·

‚±‚́A'AEŠˆ“®'
(http://www.pwblog.com/user/xru01/syusyoku/)‚Í•À‘å’ï‚Ì“w—͂ł͏æ‚èØ‚邱‚Æ‚ª¢“ï‚ÆŒ¾‚í‚ê‚Ä‚¢‚Ü‚·B

•Ä‘AƒAƒƒŠƒJ‚̃Tƒuƒvƒ‰ƒCƒ€‚ð”çØ‚è‚Ɉø‚«‹N‚±‚³‚ꂽ¡‰ñ‚Ì•s‹µA‚»‚ꂱ‚»””NŠÔ‚ɂ킽‚Á‚āA¢ŠEŒoÏ‚ɃCƒ“ƒpƒNƒg‚ð—^‚¦‘±‚¯‚é‚Å‚µ‚傤B

‚±‚ÌŽžŠú‚́A¡‚Ì‚*‚È‚½‚ª’¼–Ê‚µ‚Ä‚¢‚éAEŠˆ“®‚Æ‚¢‚¤ƒCƒxƒ“ƒg‚́A‚*‚È‚½‚̐l¶AAEl¶‚Ì‘å‚«‚ȃEƒGƒCƒg‚ðè‚߂邱‚Æ‚Æ‚È‚è‚Ü‚·B

AEŠˆ“®‚́A‹Zp‚Æ’mŽ¯‚ŏæ‚èØ‚邱‚Æ‚ªo—ˆ‚Ü‚·BAE•X‰ÍŠúAâ‘΂Ɍã‰÷‚µ‚È‚¢‚悤‚É‘S—͂Ő킢‚Ü‚µ‚傤B


-- 
icejobjob

icejobjob's Profile: http://forums.yourdomain.com.au/member.php?userid=5313
View this thread: http://forums.yourdomain.com.au/showthread.php?t=18594

--
http://mail.python.org/mailman/listinfo/python-list

braces fixed '#{' and '#}'

[v4vijayakumar]

I saw some code where someone is really managed to import braces from
__future__. ;)

def test():
#{
print "hello"
#}
--
http://mail.python.org/mailman/listinfo/python-list

Re: Relax Syntax for Augmented Arithmetic?

[andrew cooke]

Improved link - 
http://docs.python.org/3.0/reference/datamodel.html#object.__iadd__
--
http://mail.python.org/mailman/listinfo/python-list

function argument dependent on another function argument?

[Reckoner]

I  would like to do:

def foo(self,x,y=self.a)

where the default value for y=self.a. Since this is not possible, I
wind up doing


def foo(self,x,y=None)
  if not y:
y=self.a

but that seems kind of clumsy.

Is there a better way to do this?

Thanks in advance
--
http://mail.python.org/mailman/listinfo/python-list

Re: output problem

[Pierre Bourdon]

IIRC, Windows automatically add a newline after the program output.

On Fri, Jan 16, 2009 at 4:24 PM, Jean-Paul VALENTIN
 wrote:
> Feature? the output of below Hello program he0.py started from command
> line looks as follows:
> F:\prompt>he0.py
> Hello
> F:\prompt>
>
> 'Hello' was sent with sys.stdout.write, so "without newline".
> But why cannot be output (more logically):
> F:\prompt>he0.py
> HelloF:\prompt>
>
> Is it normal? Is there any means or workaround to obtain that output if
> I wish??
> 
> __
> #!/usr/bin/python
> #-*- coding: iso-8859-15 -*-
> import sys
> sys.stdout.write('Hello')
>
>
> About Ingenico: Ingenico is the world's leading provider of payment 
> solutions, with over 15 million terminals deployed across the globe. 
> Delivering the very latest secure electronic payment technologies, 
> transaction management and the widest range of value added services, Ingenico 
> is shaping the future direction of the payment solutions market. Leveraging 
> on its global presence and local expertise, Ingenico is reinforcing its 
> leadership by taking banks and businesses beyond payment through offering 
> comprehensive solutions, a true source of differentiation and new revenues 
> streams.
>  This message may contain confidential and/or privileged information. If you 
> are not the addressee or authorized to receive this for the addressee, you 
> must not use, copy, disclose or take any action based on this message or any 
> information herein. If you have received this message in error, please advise 
> the sender immediately by reply e-mail and delete this message. Thank you for 
> your cooperation.
>  P Please consider the environment before printing this e-mail
>
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>
--
http://mail.python.org/mailman/listinfo/python-list

Re: Python 3: exec arg 1

[Alan G Isaac]

Alan G Isaac wrote:

Is it intentional that ``exec`` cannot handle a TextIOWrapper?
Bottom line: has ``execfile(filename)`` really become
``exec(open(filename).read())``? Is this a good thing?




On 1/17/2009 4:20 PM Terry Reedy apparently wrote:

Yes. Yes.




Alan G Isaac wrote:

OK.  Why?



On 1/17/2009 10:08 PM Terry Reedy apparently wrote:

This: execfile(filename)
is a trivial (9 keystroke) abbreviation of
this: exec(open(filename).read())
which is similar in effect to
this: from filename import *
so it is really not needed.



Well, that does not really answer my question, imo.
I do not much care about the disappearance of ``execfile``.
I was asking, why is it a **good thing** that
``exec`` does not accept a TextIOWrapper?
Or is it just not implemented yet?
What is the gain from this particular backwards
incompatibility (in the sense that ``exec(open(fname))``
no longer works)?

Context: I used to be able to tell students they
could run their scripts from the interpreter
prompt with ``execfile(fname)``.  I expected to
be able to tell them to ``exec( open(fname) )``,
which worked in Python 2.  Now instead it is
``exec( open(filename).read() )`` which is a bit
more mysterious to a newbie.  (Note: I teach
economics, not computer science.) And despite your
claim above, you know that ``import`` is not
really equivalent, and even more so now that
``reload`` must be imported.

Thanks,
Alan Isaac
--
http://mail.python.org/mailman/listinfo/python-list

Re: function argument dependent on another function argument?

[Aaron Brady]

On Jan 18, 8:19 am, Reckoner  wrote:
> I  would like to do:
>
> def foo(self,x,y=self.a)
>
> where the default value for y=self.a. Since this is not possible, I
> wind up doing
>
> def foo(self,x,y=None)
>   if not y:
>     y=self.a
>
> but that seems kind of clumsy.
>
> Is there a better way to do this?
>
> Thanks in advance

No.  The only alternative is really niche and probably not what you
want.  You'd use a decorator (which I don't have, btw):

@defval( y= selfattrget( 'a' ) )
def foo( self, x, y ).



--
http://mail.python.org/mailman/listinfo/python-list

Re: function argument dependent on another function argument?

[Steven D'Aprano]

On Sun, 18 Jan 2009 06:19:03 -0800, Reckoner wrote:

> I  would like to do:
> 
> def foo(self,x,y=self.a)
> 
> where the default value for y=self.a. Since this is not possible, I wind
> up doing
> 
> 
> def foo(self,x,y=None)
>   if not y:
> y=self.a
> 
> but that seems kind of clumsy.

It's also incorrect, because if you pass y=0 as an argument, or any other 
false value, it will be replaced by self.a.


> Is there a better way to do this?

def foo(self, x, y=None):
if y is None:
y = self.a


I don't find that clumsy in the least. I find it perfectly readable and a 
standard idiom.


-- 
Steven
--
http://mail.python.org/mailman/listinfo/python-list

Difference between Python 2.2.2 and Python 2.5

[Ravi]

I am developing for PyS60 1.4.4 which supports Python 2.2.2 while what
I know is Python 2.5  .

Can you please tell me differences between the two so that I can save
myself from incompatible code.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Python 3: exec arg 1

[Steven D'Aprano]

On Sun, 18 Jan 2009 14:36:15 +, Alan G Isaac wrote:

> Well, that does not really answer my question, imo. I do not much care
> about the disappearance of ``execfile``. I was asking, why is it a
> **good thing** that ``exec`` does not accept a TextIOWrapper? 


I'm not sure if this is a stupid question or not, but what's a 
TextIOWrapper? In the example you give:

exec(open(fname))

the argument to exec -- open(fname) -- is a file object:

>>> type(open('hello.py'))



BTW, exec is a statement. The brackets there are totally superfluous. You 
can, and should, write: 

exec open(fname)


> Or is it just not implemented yet?
> What is the gain from this particular backwards incompatibility (in the
> sense that ``exec(open(fname))`` no longer works)?

Simplicity of implementation?


> Context: I used to be able to tell students they could run their scripts
> from the interpreter prompt with ``execfile(fname)``.  I expected to be
> able to tell them to ``exec( open(fname) )``, which worked in Python 2. 
> Now instead it is ``exec( open(filename).read() )`` which is a bit more
> mysterious to a newbie.  (Note: I teach economics, not computer
> science.) 

Why don't you give your students a site.py module containing

def execfile(fname):
exec open(fname).read()

and tell them to do "from site import execfile"?

If you control the environment they are running their programs on (school 
computers?) then you can put a startup file in their path and have 
execfile imported automatically.


-- 
Steven
--
http://mail.python.org/mailman/listinfo/python-list

Re: function argument dependent on another function argument?

[Paul Rubin]

Steven D'Aprano  writes:
> def foo(self, x, y=None):
> if y is None:
> y = self.a
> 
> I don't find that clumsy in the least. I find it perfectly readable and a 
> standard idiom.

That has the same problem as the earlier version.  If the person
passes None, they get self.a.  I prefer:

sentinel = object()
...

def foo(x, y=sentinel):
  if y is sentinel:
  y = self.a
--
http://mail.python.org/mailman/listinfo/python-list

Re: output problem

[7stud]

On Jan 16, 8:24 am, "Jean-Paul VALENTIN"  wrote:
> Feature? the output of below Hello program he0.py started from command
> line looks as follows:
> F:\prompt>he0.py
> Hello
> F:\prompt>
>
> 'Hello' was sent with sys.stdout.write, so "without newline".
> But why cannot be output (more logically):
> F:\prompt>he0.py
> HelloF:\prompt>
>
> Is it normal? Is there any means or workaround to obtain that output if
> I wish??
> 
> __
> #!/usr/bin/python
> #-*- coding: iso-8859-15 -*-
> import sys
> sys.stdout.write('Hello')
>

Not on a Mac:

import sys
sys.stdout.write("hello")

output:
$ python 6test.py
hello$

--
http://mail.python.org/mailman/listinfo/python-list

Re: Difference between Python 2.2.2 and Python 2.5

[Steven D'Aprano]

On Sun, 18 Jan 2009 07:30:52 -0800, Ravi wrote:

> I am developing for PyS60 1.4.4 which supports Python 2.2.2 while what I
> know is Python 2.5  .
> 
> Can you please tell me differences between the two so that I can save
> myself from incompatible code.

Everything new mentioned here:

http://www.python.org/doc/2.5/whatsnew/whatsnew25.html
http://www.python.org/doc/2.4/whatsnew/whatsnew24.html
http://www.python.org/doc/2.3/whatsnew/whatsnew23.html

won't exist in Python 2.2.


-- 
Steven
--
http://mail.python.org/mailman/listinfo/python-list

Re: what's the point of rpython?

[Luis M . González]

On Jan 18, 8:56 am, andrew cooke  wrote:
> Since this is a PyPy bashing thread, maybe it's an appropriate place
> to suggest that the project has got a little bit waylaid by exploring
> cool things instead of releasing a useful final result?
>
> I am not questioning rpython directly - the case for something like
> that is obvious.  But there's a question of balance.  It's possible to
> go on building ever more complex systems which are theoretically
> justified, but which postpone ever finishing the job.  At some point
> there has to be a "good enough".
>
> To some extent I am playing devil's advocate here, but as an outside
> who looked at PyPy a while back, my uninformed and naive impression
> was that the project was suffering from the kid of issues I have
> caricatured above
>
> Andrew
>
> PS I guess you are aware of worse is better etc?  I think this may
> also be a US/Euro culture issue...

It's worth adding that, regarding the goal of having a faster python,
we have had for a long time a solid and useful proof-of-concept in
psyco. Pypy rolls up on this concept and adds many more useful
features, not all of them related to speed but to flexibility.
I have no doubt the project will be succesful. The question is how
long we will have to wait...

Luis
--
http://mail.python.org/mailman/listinfo/python-list

Re: function argument dependent on another function argument?

[Aaron Brady]

On Jan 18, 9:36 am, Paul Rubin  wrote:
> Steven D'Aprano  writes:
> > def foo(self, x, y=None):
> >     if y is None:
> >         y = self.a
>
> > I don't find that clumsy in the least. I find it perfectly readable and a
> > standard idiom.
>
> That has the same problem as the earlier version.  If the person
> passes None, they get self.a.  I prefer:
>
>     sentinel = object()
>     ...
>
>     def foo(x, y=sentinel):
>       if y is sentinel:
>           y = self.a

It is too bad that it is so much work to detect whether 'y' was passed
in the function call directly.  However, sentinel is just as good (or
nearly); at worst, you need one sentinel per argument per function,
which is possible to create, which has a specific meaning.  If you are
making systematic function calls, e.g. with a dictionary or list, you
can just use the sentinel in the dictionary.
--
http://mail.python.org/mailman/listinfo/python-list

Ordering attributes for dynamically generated class

[David Pratt]

Hi list. I use 'type' to generate classes but have a need to order  
the attributes for the generated class. Of course a dict is not going  
to maintain ordering. Is there any way to dynamically generate a  
class with attributes in specific order?


my_new_class = type( 'MyNewClass', tuple_of_bases, dict_of_attributes)

Many thanks,
David
--
http://mail.python.org/mailman/listinfo/python-list

Re: Am I interacting with the database correctly?

[John Fabiani]

Hussein B wrote:

> Hey,
> I'm new with database interactions in Python and I'm not sure if I'm
> handling the cursor and transactions correctly:
> 
> cursor = db.cursor(MySQLdb.cursors.DictCursor)
> cursor.execute(flate_rate_pkgs_sql)
> rows = cursor.fetchall()
> #I have for loop here to iterate over rows
>cursor.execute() 
>rows = cursor.fetchall()
># some more cursor.execute() calls but only SQL select statements
># here is another for loop that contains try block
>   # here are cursor.execute() calls, both insert and update
>   db.commit()
>   # in the except code block, I use db.rollback()
> 
> As you see, my script contains only one db object and one cursor
> object and both the db and cursor objects are used multiple times, it
> is ok?
> As you might figured, this is a script for reports :)
> Thanks.

I have never worked with MySQL.  I do work with others.  The first part
looks fine.  If you insert, update or delete then you need a 'commit' or
a 'rollback'.  Preparing data for a report it is unlikely that you need to
commit or rollback anything.  After all you are only using 'select'.

John
--
http://mail.python.org/mailman/listinfo/python-list

Re: Ordering attributes for dynamically generated class

[Aaron Brady]

On Jan 18, 9:52 am, David Pratt  wrote:
> Hi list. I use 'type' to generate classes but have a need to order  
> the attributes for the generated class. Of course a dict is not going  
> to maintain ordering. Is there any way to dynamically generate a  
> class with attributes in specific order?
>
> my_new_class = type( 'MyNewClass', tuple_of_bases, dict_of_attributes)
>
> Many thanks,
> David

Just a thought, you can subclass 'dict' and assign an instance of it
to the __dict__ member of your new instance.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Does Python really follow its philosophy of "Readability counts"?

[Bruno Desthuilliers]

Paul Rubin a écrit :

Bruno Desthuilliers  writes:

Once again, there's quite a lot to learn from
the story of Ariane 5.


Do you know what actually happened with Ariane 5?


*yes I do* - else I wouldn't mention it. Thanks.


 The failure was
because "smart" humans overrode the language enforced protection by
casting a floating point number down to a 16-bit integer, which worked
ok in Ariane 4, but failed with an overflow on Ariane 5 where bigger
numbers were involved.


The failure was because a module tested, QA'd and certified within a 
given context (in which it was ok to drop the builtin error handling) 
was reused in a context where it was not ok. And the point is exactly 
that : no *technology* can solve this kind of problem, because it is a 
*human* problem (in that case, not taking time to repass the whole specs 
/ tests / QA process given context change).

--
http://mail.python.org/mailman/listinfo/python-list

Python and threads

[vedrandekovic]

Hello again,

Thanks for previous help on "Start two threads in same time" it was
useful,but when I run this
two threads, I think they don't start at the same time, here is my
code snippet:


import threading

class ThreadedClass1(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)

def run(self):
a=True
while a==True:
bm=my_module.MyClass()
a=bm.get_Python_Process_Usage_Function()   #Returns True
or False

class ThreadedClass2(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)

def run(self):
os.popen("my_python_script.py")



threaded_obj = ThreadedClass1()
threaded_obj.start()
threaded_obj2 = ThreadedClass2()
threaded_obj2.start()

Regards,
John
--
http://mail.python.org/mailman/listinfo/python-list

Re: process command line parameter

[Thorsten Kampe]

* asit (Sat, 17 Jan 2009 13:28:12 -0800 (PST))
> Recently I was coding a link extractor. It's a command line stuff and
> takes parameter as argument.
> I found that the in operator is not always helpful.
> eg. if "--all" in sys.argv:
>print "all links will be printed"
> 
> its not helpful when some attribute value is sent in command line
> parameter.
> hence I want to process some data like find=".co.uk"
> 
> How can i do this ???

Use optparse
--
http://mail.python.org/mailman/listinfo/python-list

Re: Does Python really follow its philosophy of "Readability counts"?

[Bruno Desthuilliers]

Russ P. a écrit :

On Jan 17, 1:43 pm, Paul Rubin  wrote:

Bruno Desthuilliers  writes:

Once again, there's quite a lot to learn from
the story of Ariane 5.

Do you know what actually happened with Ariane 5?  The failure was
because "smart" humans overrode the language enforced protection by
casting a floating point number down to a 16-bit integer, which worked
ok in Ariane 4, but failed with an overflow on Ariane 5 where bigger


So this turns out to be an example of a failure due, not to the
*rigidity* of Ada, but to its *permissiveness* in allowing such a
cast.


Nope. This is an example of failure due to the *human* part of the 
process - this happened because of a lack of testing / QA, not because 
of a language feature or misfeature.


(snip)


This is one thing that Python gets right, automatically using bignums
rather than allowing int overflow.  In that sense, Python has more
enforced protection than Ada.


True, but Ada does not have the luxury of just using doubles and
"bignums" everywhere, because it needs to work on cheap processors
too.   But perhaps it could somehow be configured to do so by the user
if sufficiently powerful computers are being used.


Here the error was *not* to disable the overflow error checking (in the 
context of Ariane 4), but to reuse the module as-is in another context.


As I already stated, no technology can protect us from this kind of 
error. Ask yourself why this module was reused as-is, instead of going 
thru the whole specs / tests / QA process again, and *perhaps* you'll 
start to understand why I say that language-enforced access restrictions 
are the wrong solution to a real problem.


--
http://mail.python.org/mailman/listinfo/python-list

Re: dynamic module import?

[Duncan Booth]

alex23  wrote:

> I must confess I've rarely had a need to use __import__ and don't
> think I've ever used the fromlist arg. I'm confused, though, because
> the docstring states:
> 
>  The fromlist should be a list of names to emulate ``from name
> import ...''
> 
> But it also states that __import__ always returns a module, so I'm
> utterly confused as to the purpose of fromlist, or how to inject the
> specified entries into the calling namespace. If anyone could explain
> this for me, I'd really appreciate it.

Compare these:

>>> dom =  __import__('xml').dom
Traceback (most recent call last):
  File "", line 1, in 
AttributeError: 'module' object has no attribute 'dom'
>>> dom =  __import__('xml', fromlist=['dom']).dom
>>> dom


Then in a new session:
>>> import xml
>>> xml.dom
Traceback (most recent call last):
  File "", line 1, in 
AttributeError: 'module' object has no attribute 'dom'
>>> from xml import dom
>>> dom

>>>

When you import a package such as xml it only imports the top level. 
Modules and subpackages within the imported package aren't available until 
they are explicitly imported. The fromlist argument to __import__ allows 
you to force the lower modules to also import.

>>> xml = __import__('xml', fromlist=['dom'])

is effectively the same as doing:

>>> import xml.dom

After either of these you have a name 'xml' which has an attribute 'dom':

>>> xml.dom


To access the actual sub-objects using __import__ use getattr on the 
returned module. So far as injecting names into the calling namespace is 
concerned just assign to variables; you don't want to be injecting unknown 
names into your namespace.

For the same effect if you only have one name and you know it is a module 
you could do:

>>> xml = __import__('xml.dom')

but you need the fromlist parameter if you don't know which of the names 
are modules.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Ordering attributes for dynamically generated class

[David Pratt]

Hi Aaron. Yeah, definitely sounds like a possibility. I was able to  
locate an ordered dict implementation that subclasses dict. This  
might work fine.  Might be able to pass into type method directly  
since I think that dict passed into type is setting __dict__ I  
believe.  Let you know if that works out. Many thanks.


Regards,
David


On Jan 18, 2009, at 11:57 AM, Aaron Brady wrote:


On Jan 18, 9:52 am, David Pratt  wrote:

Hi list. I use 'type' to generate classes but have a need to order
the attributes for the generated class. Of course a dict is not going
to maintain ordering. Is there any way to dynamically generate a
class with attributes in specific order?

my_new_class = type( 'MyNewClass', tuple_of_bases,  
dict_of_attributes)


Many thanks,
David


Just a thought, you can subclass 'dict' and assign an instance of it
to the __dict__ member of your new instance.
--
http://mail.python.org/mailman/listinfo/python-list


--
http://mail.python.org/mailman/listinfo/python-list

Re: Does Python really follow its philosophy of "Readability counts"?

[Paul Rubin]

"Russ P."  writes:
> I don't know which variant of Ada was used here, but something called
> the "Ravenscar Profile" is a reduced subset of Ada that might have
> prevented this error (though I haven't verified this). Then there is
> Spark Ada, which supposed to be much safer than even Ada.

I'm not sure, but I think Ravenscar and Spark would both have
permitted the cast.  However, Spark is intended to be used with an
external tool called Spade (sort of a Pylint on steroids) that would
not have allowed the cast unless it was provable (starting from the
specification) that the number was in range before the cast.

I.e. the cast was wrong because of the failure of an unstated
assumption that a certain sensor reading was in a certain range.
Spark may still have allowed the cast only if the assumption was
stated explicitly in the specification.  Requiring the assumption to
be stated may have made the error more likely to be spotted as part of
the surrounding systems engineering.  Of course, the specification can
still be wrong, but that's hardly a problem with Ada or even with the
software.  It's not too much different than if the specification says
the rocket is 10 feet wide and they build the launch pad for that
size, but then the rocket turns out to actually be 12 feet wide.

I have Barnes's book about Spark at the office and will try to check
next week what it says about these casts.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Does Python really follow its philosophy of "Readability counts"?

[Paul Rubin]

Bruno Desthuilliers  writes:
> As I already stated, no technology can protect us from this kind of
> error. Ask yourself why this module was reused as-is, instead of going
> thru the whole specs / tests / QA process again, and *perhaps* you'll
> start to understand why I say that language-enforced access
> restrictions are the wrong solution to a real problem.

I have not seen anywhere that the specs stated that the sensor reading
that overflowed that variable was supposed to be in range.  It looks
to me (but I don't know for sure) that the development process allowed
an assumption to creep into the code that wasn't stated in the specs.
--
http://mail.python.org/mailman/listinfo/python-list

Re: changing URLs in webpages, python solutions?

[Stefan Behnel]

Simon Forman wrote:
> I want to take a webpage, find all URLs (links, img src, etc.) and
> rewrite them in-place, and I'd like to do it in python (pure python
> preferred.)

lxml.html has functions specifically for this problem.

http://codespeak.net/lxml/lxmlhtml.html#working-with-links

Code would be something like

html_doc = lxml.html.parse(b"http://.../xyz.html";)
html_doc.rewrite_links( ... )
print( lxml.html.tostring(html_doc) )

It also handles links in CSS or JavaScript, as well as broken HTML documents.

Stefan
--
http://mail.python.org/mailman/listinfo/python-list

calling an external program and capturing the output

[Eric]

Coming from a perl background I'm new to the Python world.  I need to
read a list of values, send each value to an external program and
capture and act on the output of that program. Reading and parsing the
initial values is not a problem but I can't seem to find anything on
the sending to and capturing from an external program.

Any suggestions would be greatly appreciated.

-Eric
--
http://mail.python.org/mailman/listinfo/python-list

Re: Python and threads

[Stefan Behnel]

vedrandeko...@yahoo.com wrote:
> Thanks for previous help on "Start two threads in same time" it was
> useful,but when I run this
> two threads, I think they don't start at the same time

That's normal. Threading is an unpredictable concurrency pattern. Things
often don't happen the way one would want them to.

Stefan
--
http://mail.python.org/mailman/listinfo/python-list

Re: Ordering attributes for dynamically generated class

[Duncan Booth]

Aaron Brady  wrote:

> On Jan 18, 9:52 am, David Pratt  wrote:
>> Hi list. I use 'type' to generate classes but have a need to order  
>> the attributes for the generated class. Of course a dict is not going  
>> to maintain ordering. Is there any way to dynamically generate a  
>> class with attributes in specific order?
>>
>> my_new_class = type( 'MyNewClass', tuple_of_bases, dict_of_attributes)
>>
>> Many thanks,
>> David
> 
> Just a thought, you can subclass 'dict' and assign an instance of it
> to the __dict__ member of your new instance.
> 
You can certainly pass a subclass of 'dict' to type(), in fact in Python 
3.0 you can use a metaclass with a __prepare__ method to substitute your 
own value for __dict__ in any class definition, but I don't think your own 
type will preserved when the instance is actually constructed.

A simple solution which works in any version of Python is just to maintain 
an attribute with the desired ordering and override __dir__ to return the 
attributes in that order.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Does Python really follow its philosophy of "Readability counts"?

[Bruno Desthuilliers]

Russ P. a écrit :

No one ever claimed that a programming language, no matter how
rigorous, can eliminate all bugs. All a language can do is to reduce
their rate of occurrence.

The Ariane fiasco was not a failure of Ada per se but rather a failure
of people using Ada. 


Almost right.


They attempted to re-use software written for one
rocket for another without proper testing. No language can prevent
that sort of error.


Now this is plain right.



We can argue forever about the usefulness of language-enforced
restriction of access to private data and methods. I have no doubt
whatsoever that it is very useful at least for the extreme cases of
very large, safety-critical systems.


And my POV is that it's just plain useless.


If you don't think access to
private data needs to be restricted for control of strategic nuclear
arsenals, for example, I think you're crazy, but that's just my
opinion.


If you think that this kind of access restriction makes softwares 
controling strategic nuclear arsenal any safer, then *you* are totally 
crazy. As *you* stated above, "no language can prevent this kind of error".



The only reasonable question in my mind is where the line should be
drawn between systems that should have enforced restrictions and those
that can rely on coding standards and voluntary cooperation among
programmers.


The only reasonable question in *my* mind is whether you think it's 
better, specially for safety-critical stuff, to hire people you can 
trust or to rely on technology.



A while back, I read something about the final integration of the
flight software on the Boeing 777, which was written mainly in Ada.
The claim was made that this integration took only three days, whereas
normally it would be expected to take more like three months with a
less rigorous language such as C++.  The reason for the simplified
integration is that Ada enforces interfaces and prevents access to
private data and methods.


C++ does it too. Or at least, that's a very common "claim" about C++.


Apparently such enforcement can improve the
efficiency of software production -- and that's not just in "theory."


My own experience is that too much rigidity in a language only leads to 
more boilerplate code and more workarounds (design patterns anyone ?), 
IOW more accidental complexity, hence more space for bugs to creep in.


My very humble (no don't say it) opinion about this is that what's 
important is how you handle and manage the whole project (including both 
technical and non-technical aspects), not the technology used (which 
might be relevant - I of course wouldn't use Python for anything 
real-time - or not - most of the monstruogigantic "entreprise" software 
written in Java would work just as well in Python or any other decent 
language).


No technology will prevent human errors - I think this point is clear, 
and that we both agree on it. OTHO, some technologies can at least help 
reducing the opportunities for human errors - and I think we also agree 
on this. Now the point is : *how* can a given techno helps wrt/ this 
second goal. Here you have two philosophies. One is that you'll reduce 
errors by improving simplicity and readability, and by promoting trust 
(whithin the team), sense of responsabily, and communication. The other 
one is that you'll reduce errors by not allowing those stupid 
code-monkeys to do anything that *might* (according mostly to unproved 
assertions) be "dangerous" - hence promoting distrust and 
irresponsability, and adding quite a lot of accidental complexity. BTW, 
do you know the by far most common Java idiom ? It's named the 
"do-nothing catch-all exception handler". As an excercise, explain the 
reasons behind this idiom, and it's practical results.


As a last point : I certainly don't think Python is perfect in any way, 
*nor* that it's the appropriate tool for each and any project. I just 
disagree with you on your assertion that Python is not ok for large or 
critical projects because of it's "lack" of language-enforced access 
restriction.

--
http://mail.python.org/mailman/listinfo/python-list

Re: Python and threads

[Diez B. Roggisch]

vedrandeko...@yahoo.com schrieb:

Hello again,

Thanks for previous help on "Start two threads in same time" it was
useful,but when I run this
two threads, I think they don't start at the same time, here is my
code snippet:


import threading

class ThreadedClass1(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)

def run(self):
a=True
while a==True:
bm=my_module.MyClass()
a=bm.get_Python_Process_Usage_Function()   #Returns True
or False

class ThreadedClass2(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)

def run(self):
os.popen("my_python_script.py")



threaded_obj = ThreadedClass1()
threaded_obj.start()
threaded_obj2 = ThreadedClass2()
threaded_obj2.start()


If you want to synchronize two or more threads, you need to do so 
manually using objects like Locks, Events, Conditions and Semaphores. 
See the threading module.


Even if you managed to get two threads started simultaneously (which the 
 OS doesn't even offer IINM), the would soon run out of sync.


How about you tell us what you *want*, and we tell you if and how it's 
possible to do that.


Diez

--
http://mail.python.org/mailman/listinfo/python-list

Re: calling an external program and capturing the output

[Diez B. Roggisch]

Eric schrieb:

Coming from a perl background I'm new to the Python world.  I need to
read a list of values, send each value to an external program and
capture and act on the output of that program. Reading and parsing the
initial values is not a problem but I can't seem to find anything on
the sending to and capturing from an external program.

Any suggestions would be greatly appreciated.


See the module subprocess.

Diez
--
http://mail.python.org/mailman/listinfo/python-list

Re: Does Python really follow its philosophy of "Readability counts"?

[Bruno Desthuilliers]

Russ P. a écrit :

On Jan 17, 11:49 am, Bruno Desthuilliers
 wrote:


Please educate yourself and learn about why Ariane 5 crashed on it's
first flight, due to an error in a module written in ADA (which is such
a psychorigid language that C++ and Java are even looser than Javascript
in comparison). Perhaps will it light a bulb for you.


The claim here regarding the Ariane 5 failure is one of those urban
myths that refuses to die.


You failed the test. Sorry.


It has been refuted over and over


What has been refuted exactly ?


(including on this thread already), but I would just like to add
something in defense of Ada.

Studies have found that Ada reduces both bugs and long-term
development costs by something like a factor of two compared to C.


This _might_ (or not - there are too many variables to get any 
scientific certitudes here) be true. But that was not the point. See my 
other posts here for more about it.


--
http://mail.python.org/mailman/listinfo/python-list

Re: function argument dependent on another function argument?

[Rob Williscroft]

Aaron Brady wrote in
news:6a10378f-addb-4d56-bc1b-0c382b3cb...@t26g2000prh.googlegroups.com
in comp.lang.python: 

> On Jan 18, 9:36 am, Paul Rubin  wrote:
>> Steven D'Aprano  writes:
>> > def foo(self, x, y=None):
>> >     if y is None:
>> >         y = self.a
>>
>> > I don't find that clumsy in the least. I find it perfectly readable
>> > and 
>  a
>> > standard idiom.
>>
>> That has the same problem as the earlier version.  If the person
>> passes None, they get self.a.  I prefer:
>>
>>     sentinel = object()
>>     ...
>>
>>     def foo(x, y=sentinel):
>>       if y is sentinel:
>>           y = self.a
> 
> It is too bad that it is so much work to detect whether 'y' was passed
> in the function call directly.  However, sentinel is just as good (or
> nearly); at worst, you need one sentinel per argument per function,

One per Module should be good enough. The only reason None doesen't
suffice is that it has other legitimate uses.  Though to be honest
I would always use None as the sentinel if it wasn't a legitimate
argument.

> which is possible to create, which has a specific meaning.  If you are
> making systematic function calls, e.g. with a dictionary or list, you
> can just use the sentinel in the dictionary.

IIUYC then, one sentinel is still only needed as the missing argument
is indicated by *both* position and value or by name and value (in the
case of a keyword-dictionary), so seperate distinct sentinel objects
aren't required, for example:

SENTINEL = object()

def f( a, b, c = SENTINEL, d = SENTINEL ):
  print( "values: %r" % ( ( a, b, c, d ), ) )
  if c is SENTINEL:
print( "c is missing" )
  if d is SENTINEL:
print( "d is missing" )

f( *( 1, 2, SENTINEL, SENTINEL ) )

f( **dict( a = 1 , b = 2, d = 4 ) )

f( **dict( a = 1 , b = 2, d = 4, c = SENTINEL ) )

Rob.
-- 
http://www.victim-prime.dsl.pipex.com/
--
http://mail.python.org/mailman/listinfo/python-list

Re: Am I interacting with the database correctly?

[Gabriel Genellina]

En Sun, 18 Jan 2009 13:54:06 -0200, John Fabiani   
escribió:



I have never worked with MySQL.  I do work with others.  The first part
looks fine.  If you insert, update or delete then you need a 'commit' or
a 'rollback'.  Preparing data for a report it is unlikely that you need  
to

commit or rollback anything.  After all you are only using 'select'.


Note that you have to commit/rollback a transaction *even* if you only  
execute select statements - at least when using isolation levels higher  
than "read uncommited".
By example, using "repeatable reads", a "select" on table A blocks any  
attempt to modify the involved rows until the transaction ends. And using  
"read commited", it won't see rows modified or added after the transaction  
began.


--
Gabriel Genellina

--
http://mail.python.org/mailman/listinfo/python-list

Re: Am I interacting with the database correctly?

[Gabriel Genellina]

En Sun, 18 Jan 2009 13:54:06 -0200, John Fabiani   
escribió:



I have never worked with MySQL.  I do work with others.  The first part
looks fine.  If you insert, update or delete then you need a 'commit' or
a 'rollback'.  Preparing data for a report it is unlikely that you need  
to

commit or rollback anything.  After all you are only using 'select'.


Note that you have to commit/rollback a transaction *even* if you only  
execute select statements - at least when using isolation levels higher  
than "read uncommited".
By example, using "repeatable reads", a "select" on table A blocks any  
attempt to modify the involved rows until the transaction ends. And using  
"read commited", it won't see rows modified or added after the transaction  
began.


--
Gabriel Genellina

--
http://mail.python.org/mailman/listinfo/python-list

Re: function argument dependent on another function argument?

[Aaron Brady]

On Jan 18, 10:44 am, Rob Williscroft  wrote:
> Aaron Brady wrote 
> innews:6a10378f-addb-4d56-bc1b-0c382b3cb...@t26g2000prh.googlegroups.com
> in comp.lang.python:
>
>
>
> > On Jan 18, 9:36 am, Paul Rubin  wrote:
> >> Steven D'Aprano  writes:
> >> > def foo(self, x, y=None):
> >> >     if y is None:
> >> >         y = self.a
>
> >> > I don't find that clumsy in the least. I find it perfectly readable
> >> > and
> >  a
> >> > standard idiom.
>
> >> That has the same problem as the earlier version.  If the person
> >> passes None, they get self.a.  I prefer:
>
> >>     sentinel = object()
> >>     ...
>
> >>     def foo(x, y=sentinel):
> >>       if y is sentinel:
> >>           y = self.a
>
> > It is too bad that it is so much work to detect whether 'y' was passed
> > in the function call directly.  However, sentinel is just as good (or
> > nearly); at worst, you need one sentinel per argument per function,
>
> One per Module should be good enough. The only reason None doesen't
> suffice is that it has other legitimate uses.  Though to be honest
> I would always use None as the sentinel if it wasn't a legitimate
> argument.
>
> > which is possible to create, which has a specific meaning.  If you are
> > making systematic function calls, e.g. with a dictionary or list, you
> > can just use the sentinel in the dictionary.
>
> IIUYC then, one sentinel is still only needed as the missing argument
> is indicated by *both* position and value or by name and value (in the
> case of a keyword-dictionary), so seperate distinct sentinel objects
> aren't required, for example:
>
> SENTINEL = object()
>
> def f( a, b, c = SENTINEL, d = SENTINEL ):
>   print( "values: %r" % ( ( a, b, c, d ), ) )
>   if c is SENTINEL:
>     print( "c is missing" )
>   if d is SENTINEL:
>     print( "d is missing" )
>
> f( *( 1, 2, SENTINEL, SENTINEL ) )
>
> f( **dict( a = 1 , b = 2, d = 4 ) )
>
> f( **dict( a = 1 , b = 2, d = 4, c = SENTINEL ) )
>
> Rob.
> --http://www.victim-prime.dsl.pipex.com/

I don't have a concrete example, so you may prove to be right, but I'm
not convinced.

If you have one function with an argument that defaults to an empty
list, and calls another with an argument that defaults to an empty
dict, then what is the meaning of passing sentinel to the first one?
Whereas, if each had their own, then passing the first one's default
would mean the empty list, and passing the second one's default would
mean the dict.

(Or, even if that evaluates correctly, perhaps there is no such useful
program.)
--
http://mail.python.org/mailman/listinfo/python-list

Re: Python 3: exec arg 1

[Rob Williscroft]

Steven D'Aprano wrote in news:018342f9$0$8693$c3e8...@news.astraweb.com
in comp.lang.python: 

> I'm not sure if this is a stupid question or not, but what's a 
> TextIOWrapper? In the example you give:
> 
> exec(open(fname))
> 
> the argument to exec -- open(fname) -- is a file object:
> 
 type(open('hello.py'))
> 
> 
> 
> BTW, exec is a statement. The brackets there are totally superfluous.
> You can, and should, write: 
> 
> exec open(fname)
> 

You must have missed the subject line: "Re: Python 3: exec arg 1"

Python 3.0 (r30:67507, Dec  3 2008, 19:44:23) [MSC v.1500 64 bit (AMD64)] 
on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> open( "hello.py" )

>>> exec "a = 1"
  File "", line 1
exec "a = 1"
   ^
SyntaxError: invalid syntax
>>> exec( "a = 1" )
>>> a
1
>>>

Rob.
-- 
http://www.victim-prime.dsl.pipex.com/
--
http://mail.python.org/mailman/listinfo/python-list

Re: calling an external program and capturing the output

[Xah Lee]

On Jan 18, 8:41 am, Eric  wrote:
> Coming from a perl background I'm new to the Python world.  I need to
> read a list of values, send each value to an external program and
> capture and act on the output of that program. Reading and parsing the
> initial values is not a problem but I can't seem to find anything on
> the sending to and capturing from an external program.
>
> Any suggestions would be greatly appreciated.

See:

• Making System Calls in Perl and Python
  http://xahlee.org/perl-python/system_calls.html

  Xah
∑ http://xahlee.org/

☄

--
http://mail.python.org/mailman/listinfo/python-list

Re: import urllib2 fails with Python 2.6.1 on Vista

[Scott MacDonald]

Yes, I see your point.  Not sure how that would happen.  It is possible to
have multiple versions of python on the same machine I assume?

During the installation I have specified the directory to install python in,
otherwise I have not changed anything.  Could it be an environment variable
or something like that?

Thanks,
Scott



On Sun, Jan 18, 2009 at 12:44 AM, Gabriel Genellina
wrote:

> En Sat, 17 Jan 2009 17:13:00 -0200, Scott MacDonald
>  escribió:
>
>  I googled a bit this morning search for an answer to this problem but have
>> come up empty so far.  Can anyone help?
>>
>> Python 2.6.1 (r261:67517, Dec  4 2008, 16:51:00) [MSC v.1500 32 bit
>>
>  ^
>
>> (Intel)]
>> on win32
>> Type "help", "copyright", "credits" or "license" for more information.
>>
>>> import urllib2
>
 Traceback (most recent call last):
>>  File "", line 1, in 
>>  File "C:\dev\opt\Python25\Lib\urllib2.py", line 92, in 
>>
>  
>
> It seems your have a confusing setup. Why is Python 2.6 using
> C:\dev\opt\Python25?
>
> --
> Gabriel Genellina
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>
--
http://mail.python.org/mailman/listinfo/python-list

Re: function argument dependent on another function argument?

[Rob Williscroft]

Aaron Brady wrote in
news:582ef883-0176-4984-9521-6c1894636...@a26g2000prf.googlegroups.com
in comp.lang.python: 

> On Jan 18, 10:44 am, Rob Williscroft  wrote:
>> Aaron Brady wrote
>> innews:6a10378f-addb-4d56-bc1b-0c382b3cb...@t26g2000prh 
> .googlegroups.com
>> in comp.lang.python:
>>

>> > It is too bad that it is so much work to detect whether 'y' was
>> > passed in the function call directly.  However, sentinel is just as
>> > good (or nearly); at worst, you need one sentinel per argument per
>> > function, 
>>
>> One per Module should be good enough. The only reason None doesen't
>> suffice is that it has other legitimate uses.  Though to be honest
>> I would always use None as the sentinel if it wasn't a legitimate
>> argument.
>>
>> > which is possible to create, which has a specific meaning.  If you
>> > are
>> > making systematic function calls, e.g. with a dictionary or list,
>> > you can just use the sentinel in the dictionary.
>>
>> IIUYC then, one sentinel is still only needed as the missing argument
>> is indicated by *both* position and value or by name and value (in
>> the case of a keyword-dictionary), so seperate distinct sentinel
>> objects aren't required, for example:
>>
>> SENTINEL = object()
>>
>> def f( a, b, c = SENTINEL, d = SENTINEL ):
>>   print( "values: %r" % ( ( a, b, c, d ), ) )
>>   if c is SENTINEL:
>>     print( "c is missing" )
>>   if d is SENTINEL:
>>     print( "d is missing" )
>>
>> f( *( 1, 2, SENTINEL, SENTINEL ) )
>>
>> f( **dict( a = 1 , b = 2, d = 4 ) )
>>
>> f( **dict( a = 1 , b = 2, d = 4, c = SENTINEL ) )

 
> I don't have a concrete example, so you may prove to be right, but I'm
> not convinced.

I'm afraid I can't think of a use case for passing default values around
eiither, and I suspect if we were to come up with one, a better solution
that didn't involve passing default values around could be found.
 
> If you have one function with an argument that defaults to an empty
> list, and calls another with an argument that defaults to an empty
> dict, then what is the meaning of passing sentinel to the first one?
> Whereas, if each had their own, then passing the first one's default
> would mean the empty list, and passing the second one's default would
> mean the dict.

If you *mean* to pass an "empty list" or "empty dict"'s you should do 
it like:

  function_taking_list( [] )
  function_taking_dict( {} )

Its when you don't (have reason to) care that you need default arguments.

> (Or, even if that evaluates correctly, perhaps there is no such useful
> program.)

I agree, though I think some confusion arises here as there are two
(at least) distinct meanings for default arguments in python:

1) provide a value for when the caller doesn't, eg: 

def f( a = 1 ): ...

2) provide a cache for the functions /private/ use, eg:

def f( cache = {} ): ...

If the two are mixed up, then I can imagine a situation where somebody
might want to start passing default caches around.  It could only end
in tears.

Rob.
-- 
http://www.victim-prime.dsl.pipex.com/
--
http://mail.python.org/mailman/listinfo/python-list

Re: import urllib2 fails with Python 2.6.1 on Vista

[Scott MacDonald]

Ah yes, with your help I seem to have solved my own problem.  I had
PYTHONPATH defined to point to the 2.5 directory.

Thanks!
Scott

On Sun, Jan 18, 2009 at 11:01 AM, Scott MacDonald <
scott.p.macdon...@gmail.com> wrote:

> Yes, I see your point.  Not sure how that would happen.  It is possible to
> have multiple versions of python on the same machine I assume?
>
> During the installation I have specified the directory to install python
> in, otherwise I have not changed anything.  Could it be an environment
> variable or something like that?
>
> Thanks,
> Scott
>
>
>
>
> On Sun, Jan 18, 2009 at 12:44 AM, Gabriel Genellina <
> gagsl-...@yahoo.com.ar> wrote:
>
>> En Sat, 17 Jan 2009 17:13:00 -0200, Scott MacDonald
>>  escribió:
>>
>>  I googled a bit this morning search for an answer to this problem but
>>> have
>>> come up empty so far.  Can anyone help?
>>>
>>> Python 2.6.1 (r261:67517, Dec  4 2008, 16:51:00) [MSC v.1500 32 bit
>>>
>>  ^
>>
>>> (Intel)]
>>> on win32
>>> Type "help", "copyright", "credits" or "license" for more information.
>>>
  import urllib2
>>
> Traceback (most recent call last):
>>>  File "", line 1, in 
>>>  File "C:\dev\opt\Python25\Lib\urllib2.py", line 92, in 
>>>
>>  
>>
>> It seems your have a confusing setup. Why is Python 2.6 using
>> C:\dev\opt\Python25?
>>
>> --
>> Gabriel Genellina
>>
>> --
>> http://mail.python.org/mailman/listinfo/python-list
>>
>
>
--
http://mail.python.org/mailman/listinfo/python-list

Re: uninstall before upgrade?

[Diez B. Roggisch]

waltbrad schrieb:

I want to upgrade from 2.5 to 2.6.  Do I need to uninstall 2.5 before
I do that? If so, what's the best way to uninstall it?  Thanks.


No, several versions of python can live happily together.

Diez
--
http://mail.python.org/mailman/listinfo/python-list

Re: pep 8 constants

[Francesco Bochicchio]

On Wed, 14 Jan 2009 08:13:30 +, Steven D'Aprano wrote:


> 
> Absolutely. It's rather sad that I can do this:
> 
> import math
> math.pi = 3.0
> 
> I like the ability to shoot myself in the foot, thank you very much, but 
> I should at least get a warning when I'm about to do so:
> 
> math.PI = 3.0  # use God-like powers to change a constant
> 
> 

Constants would be a nice addition in python, sure enough.
But I'm not sure that this can be done without a run-time check every time
the constant is used, and python is already slow enough. Maybe a check
that is disabled when running with optimizing flags ?

But I'm sure this discussion has been already made and the FINAL WORD has
been already spoken.

Ciao

FB
--
http://mail.python.org/mailman/listinfo/python-list

Re: Does Python really follow its philosophy of "Readability counts"?

[Bruno Desthuilliers]

Steven D'Aprano a écrit :

On Sat, 17 Jan 2009 20:49:38 +0100, Bruno Desthuilliers wrote:


Russ P. a écrit :


(snip)
 

Why leave to
coding standards and company policy what can be encoded right into the
language?

Because human are smarter than computers.


That's an awfully naive statement. It's a sound-byte instead of a 
reasoned argument. We're smarter than computers?


Obviously and definitively, yes. Not that we are that smart - it's just 
that computers are totally stupids. The differences between a human and 
a computer are that

1/ a human can eventually DWIM.
2/ a computer can do what is has been told way faster than any human

The first point is being smart. The second is being fast. Not quite the 
same thing !-)


Then why are we 
programming in languages like Python instead of directly in machine code? 
Why can optimizing C compilers make more efficient code than the best 
human assembly language programmers?


Because :
1/ this is something that can be solved by heavy computations
2/ these computations can be programmed
3/ these compuations would just take too long for a human being to do 
manually



Humans and computers are smart at different things.


s/smart/good/g


Why leave to humans (who are known to err) what can be automated
relatively easily? Isn't that what computers are for?

Error is human. For a real catastrophic failure, it requires a computer.


Oh rubbish. That's a sound-byte invented by nervous technophobes scared 
of computers.


Nope, that's a computer-user joke.




All those "setters" and
"getters" are a kludge. I think Python "properties" are a major step
forward here. Just for fun, I checked to see if Scala has properties.
Guess what? Not only does it have them, but they are generated
automatically for all member data. That's even better than Python
properties!

Oh yes ? "Better", really ? So it's better to have a language that
automagically breaks encapsulation (requiring an additionnal level of
indirection) than a language that do the right thing by default ? I'm
afraid I missed the point ???


You certainly do. How do properties "break" encapsulation rather than 
enforcing it?


Properties by themselves are not the problem, quite on the contrary - as 
you say, they actually help wrt/ encapsulation. What breaks 
encapsulation is *automatic generation* for properties for *each and 
any* implementation attribute. Might as well just makes them all public 
attribute then.






As I said before, enforced encapsulation may not be appropriate for
every application, but it is definitely appropriate for some.

No. It is appropriate for dummy managers hiring dummy programmers. The
project's size and domain have nothing to do with it.

Let me try to be very clear here. We are dealing with two separate but
related issues. The first is whether data hiding should be added to
Python.

No need to add it, it's already there : every name that starts with an
underscore is hidden !-)


That's not hidden. It's there in plain sight. 


Once again, I'm afraid you missed the joke - despite the smiley.


Whether it can be added without screwing up the language, I don't know.
If not, then so be it. That just limits the range of domains where
Python is suitable.

That's just plain stupid.


No it's not. It's *practical*. There are domains where *by law* code 
needs to meet all sorts of strict standards to prove safety and security, 
and Python *simply cannot meet those standards*.


Oh, sorry. I was talking about *technical* issues, not about legal ones. 
IANAL...


(snip)


I like to use the example of the flight software for a large commercial
transport aircraft, but many other examples could be given. How about
medical systems that control radiation therapy or chemotherapy? How
about financial systems that could take away your retirement account in
1.5 milliseconds. Or how about the control software for the strategic
nuclear arsenals of the US or Russia? When you consider the sea, air,
and land-based components, I'm sure that's one hell of a lot of code!

And ? Such systems have been written (and quite a lot are still running)
with languages way more permissive than Python. You know, languages like
C or assembly. 


Yes, and it is *hard* because the programmer has to worry about data 
hiding *on his own*.


Nope, it's hard because C and assembly are very low-level languages 
where you have to micro-manage each and everything. This has nothing to 
do with data hiding (unless you count memory management as data hiding ?)



One of my friends has worked for many years programming some pretty high-
powered banking software. Their approach is to move data-hiding into the 
database, or the operating system. Cobol doesn't enforce encapsulation, 
but the database and OS certainly do, with a permissions-based approach.


Guess what ? Whatever the language (Cobol, Java, Python, or even VB), 
chances are such an application would be written using a RDBMS (not even 
addressing the point about OS).

Re: calling an external program and capturing the output

[Eric]

Thanks guys. That helped point me int he right direction.

with your advice on the subprocess module I stumbled upon this
posting:
http://www.velocityreviews.com/forums/t359866-subprocess-module.html

for anyone else that might be interested here is the solution. It
simply calls a perl script called add.pl that reads 2 numbers from
stdin and adds them together.

Thanks again for the help.

-Eric


#!/usr/bin/env python

import subprocess

prog = "./add.pl"
args = "3 4"

app = subprocess.Popen
(prog ,stdout=subprocess.PIPE,stdin=subprocess.PIPE,stderr=subprocess.PIPE)

print "opened " + prog
#print app.stdout.read()

print "writing \'" + args + "\' to " + prog + " subprocess"
app.stdin.write(args)
app.stdin.write("\n")

print "wrote \'" + args + "\' to " + prog + " subprocess"
result = app.stdout.read()
result = result.rstrip('\n')

print "received: " + result



and here is the output:

./call_add.py
opened ./add.pl
writing '3 4' to ./add.pl subprocess
wrote '3 4' to ./add.pl subprocess
received: 7



--
http://mail.python.org/mailman/listinfo/python-list

Re: function argument dependent on another function argument?

[andrew cooke]

>     sentinel = object()
>     ...
>
>     def foo(x, y=sentinel):
>       if y is sentinel:
>           y = self.a

it just struck me you could also do:

 def foo(self, x, *y_args)
   y = y_args[0] if y_args self.a

which more directly checks whether an argument was passed, but has the
downside of making the method signature less clear in the declaration.

andrew
--
http://mail.python.org/mailman/listinfo/python-list

WSGI question: reading headers before message body has been read

[Ron Garret]

I'm writing a WSGI application and I would like to check the content-
length header before reading the content to make sure that the content
is not too big in order to prevent denial-of-service attacks.  So I do
something like this:

def application(environ, start_response):
status = "200 OK"
headers = [('Content-Type', 'text/html'), ]
start_response(status, headers)
if int(environ['CONTENT_LENGTH'])>1000: return 'File too big'

But this doesn't seem to work.  If I upload a huge file it still waits
until the entire file has been uploaded before complaining that it's
too big.

Is it possible to read the HTTP headers in WSGI before the request
body has been read?

Thanks,
rg

--
http://mail.python.org/mailman/listinfo/python-list

Re: pep 8 constants

[MRAB]

Francesco Bochicchio wrote:

On Wed, 14 Jan 2009 08:13:30 +, Steven D'Aprano wrote:



Absolutely. It's rather sad that I can do this:

import math
math.pi = 3.0

I like the ability to shoot myself in the foot, thank you very much, but 
I should at least get a warning when I'm about to do so:


math.PI = 3.0  # use God-like powers to change a constant




Constants would be a nice addition in python, sure enough.
But I'm not sure that this can be done without a run-time check every time
the constant is used, and python is already slow enough. Maybe a check
that is disabled when running with optimizing flags ?

But I'm sure this discussion has been already made and the FINAL WORD has
been already spoken.


>>> class Constants(object):
def __setattr__(self, key, value):
if key in self.__dict__:
raise ValueError("Can't change constant")
self.__dict__[key] = value


>>> c = Constants()
>>> c.PI = 3.0
>>> c.PI
3.0
>>> c.PI = 4.0

Traceback (most recent call last):
  File "", line 1, in 
c.PI = 4.0
  File "", line 4, in __setattr__
raise ValueError("Can't change constant")
ValueError: Can't change constant
--
http://mail.python.org/mailman/listinfo/python-list

Re: Python and threads

[vedrandekovic]

On 18 sij, 17:48, "Diez B. Roggisch"  wrote:
> vedrandeko...@yahoo.com schrieb:
>
>
>
> > Hello again,
>
> > Thanks for previous help on "Start two threads in same time" it was
> > useful,but when I run this
> > two threads, I think they don't start at the same time, here is my
> > code snippet:
>
> > import threading
>
> > class ThreadedClass1(threading.Thread):
> >     def __init__(self):
> >         threading.Thread.__init__(self)
>
> >     def run(self):
> >         a=True
> >         while a==True:
> >             bm=my_module.MyClass()
> >             a=bm.get_Python_Process_Usage_Function()   #Returns True
> > or False
>
> > class ThreadedClass2(threading.Thread):
> >     def __init__(self):
> >         threading.Thread.__init__(self)
>
> >     def run(self):
> >         os.popen("my_python_script.py")
>
> > threaded_obj = ThreadedClass1()
> > threaded_obj.start()
> > threaded_obj2 = ThreadedClass2()
> > threaded_obj2.start()
>
> If you want to synchronize two or more threads, you need to do so
> manually using objects like Locks, Events, Conditions and Semaphores.
> See the threading module.
>
> Even if you managed to get two threads started simultaneously (which the
>   OS doesn't even offer IINM), the would soon run out of sync.
>
> How about you tell us what you *want*, and we tell you if and how it's
> possible to do that.
>
> Diez

Hello,

and thanks for all previous help.I want to measure memory usage of
executed python script.I'am working on windows XP.

Regards,
John
--
http://mail.python.org/mailman/listinfo/python-list

Re: WSGI question: reading headers before message body has been read

[Diez B. Roggisch]

Ron Garret schrieb:

I'm writing a WSGI application and I would like to check the content-
length header before reading the content to make sure that the content
is not too big in order to prevent denial-of-service attacks.  So I do
something like this:

def application(environ, start_response):
status = "200 OK"
headers = [('Content-Type', 'text/html'), ]
start_response(status, headers)
if int(environ['CONTENT_LENGTH'])>1000: return 'File too big'

But this doesn't seem to work.  If I upload a huge file it still waits
until the entire file has been uploaded before complaining that it's
too big.

Is it possible to read the HTTP headers in WSGI before the request
body has been read?


AFAIK that is nothing that WSGI defines - it's an implementation-detail 
of your server. Which one do you use?


Diez
--
http://mail.python.org/mailman/listinfo/python-list

Re: WSGI question: reading headers before message body has been read

[Petite Abeille]

On Jan 18, 2009, at 8:01 PM, Ron Garret wrote:


def application(environ, start_response):
   status = "200 OK"
   headers = [('Content-Type', 'text/html'), ]
   start_response(status, headers)
   if int(environ['CONTENT_LENGTH'])>1000: return 'File too big'


How would that work for chunked transfer-encoding?

Cheers,

--
PA.
http://alt.textdrive.com/nanoki/

--
http://mail.python.org/mailman/listinfo/python-list

Re: Does Python really follow its philosophy of "Readability counts"?

[Russ P.]

On Jan 18, 9:22 am, Bruno Desthuilliers
 wrote:

> Properties by themselves are not the problem, quite on the contrary - as
> you say, they actually help wrt/ encapsulation. What breaks
> encapsulation is *automatic generation* for properties for *each and
> any* implementation attribute. Might as well just makes them all public
> attribute then.

Let me correct my statement about the automatic generation of
properties in Scala: it is only for public attributes, not all
attributes.

Getting back to the bigger point, I will gladly agree with you that
data hiding is not a magic bullet that will eliminate all bugs. The
idea that I or anyone else said that, however, is a red herring. Data
hiding is just one safeguard in a portfolio of safeguards that can
*help* to prevent certain kinds of bugs as well as deliberate acts of
sabotage or fraud. When you have a tough problem to solve, you need
all the help you can get.

You keep saying that if you hire competent, trustworthy developers,
you don't need data hiding. Well, maybe, but when you have a team of
dozens or hundreds of developers, your chances of avoiding any bad
ones is zero for all practical purposes.

And even if all your developers were excellent, data hiding would
still be a convenient mechanism to simplify their jobs so they can
focus on higher level problems -- and not have to rely on an ugly
naming convention.

Now, if developers become careless because they think data hiding will
save them, then that would be a problem. That much I will concede. But
I doubt that happens much.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Relax Syntax for Augmented Arithmetic?

[Terry Reedy]

andrew cooke wrote:

Context - http://docs.python.org/3.0/reference/datamodel.html?highlight=data
model#object.__iadd__

Just a suggestion I thought I'd throw out...  There's a restriction in
the language implementation on exactly what can go the left of an
augmented arithmetic expression.

For example:

a = 3
a **= 2


is ok, but:

class Foo():

...   def __init__():
... self.a = 3
...   def __ipow__(self, x):
... self.a **= x
...

Foo() **= 2


Calls return objects and therefore cannot be the target of an 
assignment, augmented or otherwise.  The target of an assignment is a 
name or collection slot, both of which are grammatical constructs, not 
objects.


--
http://mail.python.org/mailman/listinfo/python-list

Re: WSGI question: reading headers before message body has been read

[Ron Garret]

On Jan 18, 11:29 am, "Diez B. Roggisch"  wrote:
> Ron Garret schrieb:
>
>
>
> > I'm writing a WSGI application and I would like to check the content-
> > length header before reading the content to make sure that the content
> > is not too big in order to prevent denial-of-service attacks.  So I do
> > something like this:
>
> > def application(environ, start_response):
> >     status = "200 OK"
> >     headers = [('Content-Type', 'text/html'), ]
> >     start_response(status, headers)
> >     if int(environ['CONTENT_LENGTH'])>1000: return 'File too big'
>
> > But this doesn't seem to work.  If I upload a huge file it still waits
> > until the entire file has been uploaded before complaining that it's
> > too big.
>
> > Is it possible to read the HTTP headers in WSGI before the request
> > body has been read?
>
> AFAIK that is nothing that WSGI defines - it's an implementation-detail
> of your server. Which one do you use?

Apache at the moment, with lighttpd as a contender to replace it.

rg

--
http://mail.python.org/mailman/listinfo/python-list

Re: Python and threads

[Stefan Behnel]

vedrandeko...@yahoo.com wrote:
> and thanks for all previous help.I want to measure memory usage of
> executed python script.I'am working on windows XP.

Could you qualify "measure"? Do you mean:

a) "debug" (permanently high accuracy, potentially high runtime overhead)
b) "monitor" (high accuracy, low/medium-resolution surveillance, no runtime
  overhead)
c) "find out" (low accuracy or just max usage, no permanent observation)

?

Stefan
--
http://mail.python.org/mailman/listinfo/python-list

Re: WSGI question: reading headers before message body has been read

[Ron Garret]

On Jan 18, 11:43 am, Petite Abeille  wrote:
> On Jan 18, 2009, at 8:01 PM, Ron Garret wrote:
>
> > def application(environ, start_response):
> >    status = "200 OK"
> >    headers = [('Content-Type', 'text/html'), ]
> >    start_response(status, headers)
> >    if int(environ['CONTENT_LENGTH'])>1000: return 'File too big'
>
> How would that work for chunked transfer-encoding?

It wouldn't.  But many clients don't use chunked-transfer-encoding
when uploading files whose size is known.  In that case it would be
nice to let users know that their upload is going to fail BEFORE they
waste hours waiting for 10GB of data to go down the wire.

rg

--
http://mail.python.org/mailman/listinfo/python-list

Re: WSGI question: reading headers before message body has been read

[Diez B. Roggisch]

Ron Garret schrieb:

On Jan 18, 11:29 am, "Diez B. Roggisch"  wrote:

Ron Garret schrieb:




I'm writing a WSGI application and I would like to check the content-
length header before reading the content to make sure that the content
is not too big in order to prevent denial-of-service attacks.  So I do
something like this:
def application(environ, start_response):
status = "200 OK"
headers = [('Content-Type', 'text/html'), ]
start_response(status, headers)
if int(environ['CONTENT_LENGTH'])>1000: return 'File too big'
But this doesn't seem to work.  If I upload a huge file it still waits
until the entire file has been uploaded before complaining that it's
too big.
Is it possible to read the HTTP headers in WSGI before the request
body has been read?

AFAIK that is nothing that WSGI defines - it's an implementation-detail
of your server. Which one do you use?


Apache at the moment, with lighttpd as a contender to replace it.



Together with mod_wsgi?

Diez
--
http://mail.python.org/mailman/listinfo/python-list

libmsi.a import library from wine, and header files available (entirely free software), available for python-win32 builds under msys+wine

[Luke Kenneth Casson Leighton]

as part of building python2.5.2 under msys under wine on linux using
mingw, i thought i'd try building _msi.pyd just for kicks.  of course,
that required having an msi.lib import library, and associated header
files.  so, purely as an experiment, i've documented the process by
which it is possible to take wine 1.1.13 (current development release)
source code, modify the header files for use with mingw, create a
typelibrary (using dlltool) and then, surprise-surprise, _msi.pyd
successfully builds.

i say successfully builds: but then, running from _outside_ of a
wineconsole cmd, doing this:

/usr/local/bin/wine ./python.exe -c 'import _msi'

succeeds.

but if you do this:

  /usr/local/bin/wineconsole cmd
  c:/python2.5/bin/python.exde -c 'import _msi'

you get an "access violation - no access to memory" blah blah

to be honest, i don't care about that: this message is to let people
know that it _is_ possible, and, if anyone is interested in e.g.
adding -lmsi -lcabinet support to MinGW, here's where you can get the
necessary crud:

http://lkcl.net/msi.tgz

if the wine team have any objections, if they believe this is a bad
idea, please do say so :)

l.

p.s. if anyone would like to add a regression test to python called
"test_msi.py" - or if they know of one that exists, i'd love to hear
from you and try it out.  outside of a wine cmd.exe of course :)
--
http://mail.python.org/mailman/listinfo/python-list

Re: WSGI question: reading headers before message body has been read

[Ron Garret]

On Jan 18, 12:40 pm, "Diez B. Roggisch"  wrote:
> Ron Garret schrieb:
>
>
>
> > On Jan 18, 11:29 am, "Diez B. Roggisch"  wrote:
> >> Ron Garret schrieb:
>
> >>> I'm writing a WSGI application and I would like to check the content-
> >>> length header before reading the content to make sure that the content
> >>> is not too big in order to prevent denial-of-service attacks.  So I do
> >>> something like this:
> >>> def application(environ, start_response):
> >>>     status = "200 OK"
> >>>     headers = [('Content-Type', 'text/html'), ]
> >>>     start_response(status, headers)
> >>>     if int(environ['CONTENT_LENGTH'])>1000: return 'File too big'
> >>> But this doesn't seem to work.  If I upload a huge file it still waits
> >>> until the entire file has been uploaded before complaining that it's
> >>> too big.
> >>> Is it possible to read the HTTP headers in WSGI before the request
> >>> body has been read?
> >> AFAIK that is nothing that WSGI defines - it's an implementation-detail
> >> of your server. Which one do you use?
>
> > Apache at the moment, with lighttpd as a contender to replace it.
>
> Together with mod_wsgi?
>
> Diez

Yes.  (Is there any other way to run WSGI apps under Apache?)

rg

--
http://mail.python.org/mailman/listinfo/python-list

Re: WSGI question: reading headers before message body has been read

[Graham Dumpleton]

On Jan 19, 6:01 am, Ron Garret  wrote:
> I'm writing a WSGI application and I would like to check the content-
> length header before reading the content to make sure that the content
> is not too big in order to prevent denial-of-service attacks.  So I do
> something like this:
>
> def application(environ, start_response):
>     status = "200 OK"
>     headers = [('Content-Type', 'text/html'), ]
>     start_response(status, headers)
>     if int(environ['CONTENT_LENGTH'])>1000: return 'File too big'

You should be returning 413 (Request Entity Too Large) error status
for that specific case, not a 200 response.

You should not be returning a string as response content as it is very
inefficient, wrap it in an array.

> But this doesn't seem to work.  If I upload a huge file it still waits
> until the entire file has been uploaded before complaining that it's
> too big.
>
> Is it possible to read the HTTP headers in WSGI before the request
> body has been read?

Yes.

The issue is that in order to avoid the client sending the data the
client needs to actually make use of HTTP/1.1 headers to indicate it
is expecting a 100-continue response before sending data. You don't
need to handle that as Apache/mod_wsgi does it for you, but the only
web browser I know of that supports 100-continue is Opera browser.
Clients like curl do also support it as well though. In other words,
if people use IE, Firefox or Safari, the request content will be sent
regardless anyway.

There is though still more to this though. First off is that if you
are going to handle 413 errors in your own WSGI application and you
are using mod_wsgi daemon mode, then request content is still sent by
browser regardless, even if using Opera. This is because the act of
transferring content across to mod_wsgi daemon process triggers return
of 100-continue to client and so it sends data. There is a ticket for
mod_wsgi to implement proper 100-continue support for daemon mode, but
will be a while before that happens.

Rather than have WSGI application handle 413 error cases, you are
better off letting Apache/mod_wsgi handle it for you. To do that all
you need to do is use the Apache 'LimitRequestBody' directive. This
will check the content length for you and send 413 response without
the WSGI application even being called. When using daemon mode, this
is done in Apache child worker processes and for 100-continue case
data will not be read at all and can avoid client sending it if using
Opera.

Only caveat on that is the currently available mod_wsgi has a bug in
it such that 100-continue requests not always working for daemon mode.
You need to apply fix in:

  http://code.google.com/p/modwsgi/issues/detail?id=121

For details on LimitRequestBody directive see:

  http://httpd.apache.org/docs/2.2/mod/core.html#limitrequestbody

Graham
--
http://mail.python.org/mailman/listinfo/python-list

Re: Does Python really follow its philosophy of "Readability counts"?

[Tim Rowe]

2009/1/18 Paul Rubin <"http://phr.cx"@nospam.invalid>:

> I.e. the cast was wrong because of the failure of an unstated
> assumption that a certain sensor reading was in a certain range.
> Spark may still have allowed the cast only if the assumption was
> stated explicitly in the specification.

Unless it's changed since I used it, technically, SPADE doesn't allow
or disallow anything. It produces a predicate (a proof obligation)
that you have to prove is always true (or is it always false? It's
been 18 years since I worked on that stuff, and SPADE and MALPAS
produced their proof obligations with opposite values). So it's still
up to you to show that it won't overflow, it just gives you the
predicate calculus expression that you need to do that.

Since the value appears to come from a sensor, the only way one could
prove that there would be no overflow would be to state it as a part
of the specification of what is read in. If that specification doesn't
match the specification of the actual sensor, that's nothing to do
with the programming language or, for that matter, the program itself.
It's a specification mismatch.

I was actually at the European Space Agency's Toulouse site the week
after the Ariane 5 incident. I've been at jollier funerals. I can't
help thinking that thinking that the team would have benefited from
reading David Parnas's work on the specification of the A-7E avionics.

-- 
Tim Rowe
--
http://mail.python.org/mailman/listinfo/python-list

Re: WSGI question: reading headers before message body has been read

[Graham Dumpleton]

On Jan 19, 6:43 am, Petite Abeille  wrote:
> On Jan 18, 2009, at 8:01 PM, Ron Garret wrote:
>
> > def application(environ, start_response):
> >    status = "200 OK"
> >    headers = [('Content-Type', 'text/html'), ]
> >    start_response(status, headers)
> >    if int(environ['CONTENT_LENGTH'])>1000: return 'File too big'
>
> How would that work for chunked transfer-encoding?

Chunked transfer encoding on request content is not supported by WSGI
specification as WSGI requires CONTENT_LENGTH be set and disallows
reading more than defined content length, where CONTENT_LENGTH is
supposed to be taken as 0 if not provided.

If using Apache/mod_wsgi 3.0 (currently in development, so need to use
subversion copy), you can step outside what WSGI strictly allows and
still handle chunked transfer encoding on request content, but you
still don't have a CONTENT_LENGTH so as to check in advance if more
data than expected is going to be sent.

If wanting to know how to handle chunked transfer encoding in
mod_wsgi, better off asking on mod_wsgi list.

Graham
--
http://mail.python.org/mailman/listinfo/python-list

Re: WSGI question: reading headers before message body has been read

[Ron Garret]

On Jan 18, 1:21 pm, Graham Dumpleton 
wrote:
> On Jan 19, 6:01 am, Ron Garret  wrote:
>
> > I'm writing a WSGI application and I would like to check the content-
> > length header before reading the content to make sure that the content
> > is not too big in order to prevent denial-of-service attacks.  So I do
> > something like this:
>
> > def application(environ, start_response):
> >     status = "200 OK"
> >     headers = [('Content-Type', 'text/html'), ]
> >     start_response(status, headers)
> >     if int(environ['CONTENT_LENGTH'])>1000: return 'File too big'
>
> You should be returning 413 (Request Entity Too Large) error status
> for that specific case, not a 200 response.
>
> You should not be returning a string as response content as it is very
> inefficient, wrap it in an array.
>
> > But this doesn't seem to work.  If I upload a huge file it still waits
> > until the entire file has been uploaded before complaining that it's
> > too big.
>
> > Is it possible to read the HTTP headers in WSGI before the request
> > body has been read?
>
> Yes.
>
> The issue is that in order to avoid the client sending the data the
> client needs to actually make use of HTTP/1.1 headers to indicate it
> is expecting a 100-continue response before sending data. You don't
> need to handle that as Apache/mod_wsgi does it for you, but the only
> web browser I know of that supports 100-continue is Opera browser.
> Clients like curl do also support it as well though. In other words,
> if people use IE, Firefox or Safari, the request content will be sent
> regardless anyway.
>
> There is though still more to this though. First off is that if you
> are going to handle 413 errors in your own WSGI application and you
> are using mod_wsgi daemon mode, then request content is still sent by
> browser regardless, even if using Opera. This is because the act of
> transferring content across to mod_wsgi daemon process triggers return
> of 100-continue to client and so it sends data. There is a ticket for
> mod_wsgi to implement proper 100-continue support for daemon mode, but
> will be a while before that happens.
>
> Rather than have WSGI application handle 413 error cases, you are
> better off letting Apache/mod_wsgi handle it for you. To do that all
> you need to do is use the Apache 'LimitRequestBody' directive. This
> will check the content length for you and send 413 response without
> the WSGI application even being called. When using daemon mode, this
> is done in Apache child worker processes and for 100-continue case
> data will not be read at all and can avoid client sending it if using
> Opera.
>
> Only caveat on that is the currently available mod_wsgi has a bug in
> it such that 100-continue requests not always working for daemon mode.
> You need to apply fix in:
>
>  http://code.google.com/p/modwsgi/issues/detail?id=121
>
> For details on LimitRequestBody directive see:
>
>  http://httpd.apache.org/docs/2.2/mod/core.html#limitrequestbody
>
> Graham

Thanks for the detailed response!

rg

--
http://mail.python.org/mailman/listinfo/python-list

Can Python manipulate PE structure or bytes?

[seaworthyjeremy]

I'm interested in Python and wanted to know if Python can manipulate
PE structure and bytes. Also what are its limits?
--
http://mail.python.org/mailman/listinfo/python-list

Re: Does Python really follow its philosophy of "Readability counts"?

[bearophileHUGS]

r:
>Of course it would not run in C or Python but the point here is readability.<

With languages like Genie you can go close:
http://live.gnome.org/Genie

Or better Delight (based on the D language), plus some my "Python
compatibility" libs (plus Pyd, if you want) you can run that code:
http://delight.sourceforge.net/

Bye,
bearophile
--
http://mail.python.org/mailman/listinfo/python-list

Re: function argument dependent on another function argument?

[Steven D'Aprano]

On Sun, 18 Jan 2009 07:36:53 -0800, Paul Rubin wrote:

> Steven D'Aprano  writes:
>> def foo(self, x, y=None):
>> if y is None:
>> y = self.a
>> 
>> I don't find that clumsy in the least. I find it perfectly readable and
>> a standard idiom.
> 
> That has the same problem as the earlier version.

No it doesn't. The earlier version had the problem that *any* false 
object is replaced by self.a. I'm just following the standard Python 
idiom of using None as a sentinel. Have a look through the standard 
library and see how many times it is used.

Built-ins rarely accept None as a sentinel, slice() being a conspicuous 
exception. This is sometimes a nuisance when writing wrappers:

def my_find(S, sub, start=None, end=None):
"""Like string.find() only with pre-processing."""
pre_process()  # stub for something complicated
if end is None and start is None:
return S.find(sub)
elif end if None:
return S.find(sub, start)
else:
return S.find(sub, start, end)

or if you prefer:

def my_find(S, sub, start=None, end=None):
"""Like string.find()"""
pre_process()
args = [sub]
if start is not None:
args.append(start)
if end is not None:
args.append(end)
return S.find(*args)


Having said that, there are times where you need to pass None as a 
legitimate argument and not as a sentinel. In that case, your solution:

> If the person passes
> None, they get self.a.  I prefer:
> 
> sentinel = object()
> ...
> 
> def foo(x, y=sentinel):
>   if y is sentinel:
>   y = self.a

is an excellent one.


-- 
Steven
--
http://mail.python.org/mailman/listinfo/python-list

Re: Does Python really follow its philosophy of "Readability counts"?

[bearophileHUGS]

alex23:
> Paul, have you looked into Cython at all?

I can also suggest ShedSkin and the D language.

I have often used D for data munging, producing quick & short
programs, when Python isn't fast enough for a certain purpose.

Bye,
bearophile
--
http://mail.python.org/mailman/listinfo/python-list

Re: function argument dependent on another function argument?

[Paul Rubin]

Steven D'Aprano  writes:
> Having said that, there are times where you need to pass None as a 
> legitimate argument and not as a sentinel. 

I don't think it's worth trying to figure out which those times are.
The conclusion can be wrong, or can become wrong later because of
some faraway change in the code.  I prefer using the bulletproof
method from the beginning, whether it is needed or not.
--
http://mail.python.org/mailman/listinfo/python-list

Re: WSGI question: reading headers before message body has been read

[Diez B. Roggisch]

Ron Garret schrieb:

On Jan 18, 12:40 pm, "Diez B. Roggisch"  wrote:

Ron Garret schrieb:




On Jan 18, 11:29 am, "Diez B. Roggisch"  wrote:

Ron Garret schrieb:

I'm writing a WSGI application and I would like to check the content-
length header before reading the content to make sure that the content
is not too big in order to prevent denial-of-service attacks.  So I do
something like this:
def application(environ, start_response):
status = "200 OK"
headers = [('Content-Type', 'text/html'), ]
start_response(status, headers)
if int(environ['CONTENT_LENGTH'])>1000: return 'File too big'
But this doesn't seem to work.  If I upload a huge file it still waits
until the entire file has been uploaded before complaining that it's
too big.
Is it possible to read the HTTP headers in WSGI before the request
body has been read?

AFAIK that is nothing that WSGI defines - it's an implementation-detail
of your server. Which one do you use?

Apache at the moment, with lighttpd as a contender to replace it.

Together with mod_wsgi?

Diez


Yes.  (Is there any other way to run WSGI apps under Apache?)


Well, not so easy, but of course you can work with mod_python or even 
CGI/fastcgi to eventually invoke a WSGI-application.


However, the original question - that's a tough one.

According to this, it seems one can use an apache-directive to prevent 
mod_wsgi to even pass a request to the application if it exceeds a 
certain size.


http://code.google.com/p/modwsgi/wiki/ConfigurationGuidelines

Search for "Limiting Request Content"

However, I'm not sure how early that happens. I can only suggest you try 
& contact Graham Dumpleton directly, he is very responsive.



Diez

--
http://mail.python.org/mailman/listinfo/python-list