Re: More version woes

[Steven D'Aprano]

On Sun, 17 Jan 2010 23:15:28 -0800, Jive Dadson wrote:

> Sorry.  That deprecation warning has nothing to do with the slowness. It
> does torque my jaw, however.  Komodo costs money, and Python 2.6 broke
> it. @#^&!!! (Again.)
> 
> So, the new question is, does anyone know how to make Komodo 3.5 run at
> speed with Python 2.6?  Or perhaps better yet, can someone suggest a
> good IDE that doesn't cost much?

Perhaps you should be asking the Komodo forums?

Or since you paid for it, perhaps you should be asking customer support?

I suspect that they'll probably tell you that since the latest version of 
Komodo is 5.2, and you're using 3.5, you should stop using a version that 
is over four years old and almost certainly not supported.


-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: More version woes

[Jive Dadson]

Steven D'Aprano wrote:

On Sun, 17 Jan 2010 23:15:28 -0800, Jive Dadson wrote:


Sorry.  That deprecation warning has nothing to do with the slowness. It
does torque my jaw, however.  Komodo costs money, and Python 2.6 broke
it. @#^&!!! (Again.)

So, the new question is, does anyone know how to make Komodo 3.5 run at
speed with Python 2.6?  Or perhaps better yet, can someone suggest a
good IDE that doesn't cost much?


Perhaps you should be asking the Komodo forums?

Or since you paid for it, perhaps you should be asking customer support?

I suspect that they'll probably tell you that since the latest version of 
Komodo is 5.2, and you're using 3.5, you should stop using a version that 
is over four years old and almost certainly not supported.





Yep.  They want $295, and I cannot justify that for personal use.  I'll 
be looking for a cheaper one.

--
http://mail.python.org/mailman/listinfo/python-list

Re: C-API: Get scope in function

[moerchendiser2k3]

Perfect, this helped me a lot, thx! :)
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to install local module other than in "site-packages"?

[Jive Dadson]

Thankee.  I had just figgered that out.  I wrote everything up in a 
message titled "The answer," but I accidentally created a new thread 
with it. I'll post it in this thread.

--
http://mail.python.org/mailman/listinfo/python-list

Re: More version woes

[Steven D'Aprano]

On Mon, 18 Jan 2010 00:08:35 -0800, Jive Dadson wrote:

>> I suspect that they'll probably tell you that since the latest version
>> of Komodo is 5.2, and you're using 3.5, you should stop using a version
>> that is over four years old and almost certainly not supported.
>> 
>> 
>> 
> Yep.  They want $295, and I cannot justify that for personal use.  I'll
> be looking for a cheaper one.


This is why I use a text editor and the interactive console :)


-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: enhancing 'list'

[Peter Otten]

samwyse wrote:

> Lately, I've slinging around a lot of lists, and there are some simple
> things I'd like to do that just aren't there.
> 
> s.count(x[, cmp[, key]])
> - return number of i‘s for which s[i] == x.  'cmp' specifies a custom
> comparison function of two arguments, as in '.sort'.  'key' specifies
> a custom key extraction function of one argument.

What's your use case exactly? If I were to enhance count/index/rindex I 
would go for the simpler

>>> missing = object()  
>>> class List(list): 
... def count(self, value=missing, predicate=missing):
... if value is missing:
... if predicate is missing:
... raise TypeError
... return sum(1 for item in self if predicate(item))
... else:
... if predicate is not missing:
... raise TypeError
... return list.count(self, value)
...
>>> items = List(range(10))
>>> items.count(7)
1
>>> items.count(predicate=lambda item: item%3)
6

which nicely covers all applications I can imagine.

Peter
-- 
http://mail.python.org/mailman/listinfo/python-list

LAST CALL FOR PAPERS: TOOLS EUROPE 2010

[Lorenzo Bettini]

==
  LAST CALL FOR PAPERS
  (Deadline: January 22, 2010)

 TOOLS EUROPE 2010
   48th International Conference
Objects, Models, Components, Patterns
Co-located with
   *** International Conference on Model Transformation (ICMT 2010) ***
*** International Conference on Software Composition (SC 2010) ***
 *** International Conference on Tests and Proofs (TAP 2010) ***
  Málaga - Spain, 28 June - 02 July 2010
   http://malaga2010.lcc.uma.es/
==

TOOLS EUROPE is devoted to the combination of technologies that have
emerged as a result of object technology becoming "mainstream". Like its
predecessors, TOOLS EUROPE combines an emphasis on quality with a strong
practical focus.

Started in 1989, TOOLS conferences, held in Europe, the USA, Australia,
China and Eastern Europe, have played a major role in the development of
object technology; many of seminal concepts were first presented at
TOOLS. After an interruption of four years, the conference was revived
in 2007 to reflect the maturing of the field and the new challenges
ahead and has become a yearly event.

Contributions are solicited on all aspects of object technology and
related fields, in particular model-based development, component-based
development, and patterns (design, analysis and other applications);
more generally, any contribution addressing topics in advanced software
technology fall within the scope of TOOLS. Reflecting the practical
emphasis of TOOLS, contributions showcasing applications along with a
sound conceptual contribution are particularly welcome. Topics include:

* Object technology, including programming techniques, languages, tools
* Testing of object-oriented systems
* Patterns, pattern languages, tool support for patterns
* Distributed and concurrent object systems
* Real-time object-oriented programming and design
* Experience reports, including efforts at standardisation
* Applications to safety- and security-related software
* Component-based programming, modelling, tools
* Aspects and aspect-oriented programming and modelling
* Frameworks for component-based development
* Trusted and reliable components
* Model-driven development and Model-Driven Architecture
* Domain specific languages and language design
* Tools and frameworks for supporting model-driven development
* Language implementation techniques, compilers, run-time systems
* Practical applications of program verification and analysis
* Open source solutions & Reproduction studies

All contributions will be subject to a rigorous selection process by the
international Program Committee, with a stress on originality,
practicality and overall quality. The proceedings will be published in
Springer LNBIP. For detailed submission information see the conference
page.

Important Dates:

Papers submission deadline: January 22, 2010
Acceptance notification: March 24, 2010
Camera-ready final copy: April 5, 2010
Conference: June 28 -- July 02, 2010

Conference Chair: Bertrand Meyer, ETH Zürich and Eiffel Software
Program Chair: Jan Vitek, Purdue University
Publicity Chair: Osmar Santos, University of York

Program Committee:

Uwe Assman, University of Dresden, Germany
Elisa Baniassad, Chinese University of Hong Kong, Hong Kong
Alexandre Bergel, University of Chile, Chile
Lorenzo Bettini, University of Torino, Italy
Judith Bishop, Microsoft Research, USA
William Cook, University of Texas Austin, USA
Sophia Drossopolou, Imperial College London, UK
Catherine Dubois, ENSIIE, France
Stéphane Ducasse, INRIA Lille, France
Manuel Fahndrich, Microsoft Research, USA
Harald Gall, University of Zurich, Switzerland
Benoit Garbinato, University of Lausanne, Switzerland
Angelo Gargantini, University of Bergamo, Italy
Jeff Gray, University of Alabama Birmingham, USA
Kathryn Gray, University of Cambridge, UK
Thomas Gschwind, IBM Research, Switzerland
Matthias Hauswith, University of  Lugano, Switzerland
Nigel Horspool, University of Victoria, Canada
Tomas Kalibera, Charles University, Czech Republic
Gerti Kappel, Vienna University of Technology, Austria
Doug Lea, State University of New York Oswego, USA
Shane Markstrum, Brucknell University, USA
Peter Müller, ETH Zurich, Switzerland
Oscar Nierstrasz, University of Bern, Switzerland
James Noble, Victoria University of Wellington, New Zealand
Nate Nystrom, University of Texas Arlington, USA
Manuel Oriol, University of York, UK
Jonathan Ostroff, York University, Canada
Richard Paige, University of York, UK
Shaz Qadeer, Microsoft Research, USA
Awais Rashid, Lancaster University, UK
Vivek Sarkar, Rice University, USA
Doug Schmidt, Vanderbilt University, USA
Manuel Serrano, INRIA Sophia Antipolis, France
Peter Thiemann, University of Freiburg, German

ctypes: nested structures and pointers

[Gabriele Modena]

Hi all,
I am trying to learn ctypes and I  am facing some problems In wrapping
two nested structs.

<--- begin C code >
struct dev_callbacks;// Prototype the callback struct

typedef struct {
  const struct dev_callbacks* pdc;
  char acName[DEVICE_NAME_LENGTH];
  chip_type ct;
  dev_spec ds;
  bool bActive;
  bool bCrc;
  bool bPar;
  uint8_t ui8TxBits;
} dev_info;

struct dev_callbacks {
  const char* acDriver;
  dev_info* (*connect)(const uint32_t uiIndex);
  bool (*transceive)(const dev_spec ds, const byte_t* pbtTx, const
uint32_t uiTxLen, byte_t* pbtRx, uint32_t* puiRxLen);
  void (*disconnect)(dev_info* pdi);
};

< End C code >

I have two problems with this code:

 1. what is the correct (pythonic) way to capture the prototype
definition of dev_callbacks and the relation between that structure
and dev_info?

 2. is it correct to wrap "connect", "transceive" and "disconnect" in
separate structures and reference them within  pyDEV_CALLBACKS?

The (stub) code I wrote for now looks like this:

class pyDEV_CALLBACKS(Structure):
_fields_ = [("acDriver", c_char_p), ("connect", c_void_p),
("transceive", c_bool), ("disconnect", c_void_p) ]


class pyDEV_INFO(Structure):
_fields_ = [ ("pdc", POINTER(pyDEV_CALLBACKS)), ("ct", c_ubyte),
("ds", c_void_p), ("acName", c_char * 256), ("bActive",
c_bool), ("bCrc", c_bool), ("bPar", c_bool), ("ui8TxBits", c_ubyte) ]


Passing the data structures to wrapper functions seem to work (the
data is initialized), but the result is not what expected (I presume
the  problems are related to memory alignment due to wrong
declarations),

-- 
Gabriele
-- 
http://mail.python.org/mailman/listinfo/python-list

Generic Python Benchmark suite?

[Anand Vaidya]

Is there a generic python benchmark suite in active development? I am
looking forward to comparing some code on various python
implementations (primarily CPython 2.x, CPython 3.x, UnladenSwallow,
Psyco).

I am happy with something that gives me a relative number eg: ULS is
30% faster than CPy2.x etc

I found pybench which is probably not maintained actively.

What do you suggest?

PS: I think a benchmark should cover file / network, database  I/O,
data structures (dict, list etc), object creation/manipulation,
numbers, measure looping inefficiencies, effects of caching (memcache
etc) at the minimum
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: substitution

[superpollo]

superpollo ha scritto:

hi.

what is the most pythonic way to substitute substrings?

eg: i want to apply:

foo --> bar
baz --> quux
quuux --> foo

so that:

fooxxxbazyyyquuux --> barxxxquuxyyyfoo

bye


i explain better:

say the subs are:

quuux --> foo
foo --> bar
baz --> quux

then i cannot apply the subs in sequence (say, .replace() in a loop), 
otherwise:


fooxxxbazyyyquuux --> fooxxxbazyyyfoo --> barxxxbazyyybar --> 
barxxxquuxyyybar


not as intended...
--
http://mail.python.org/mailman/listinfo/python-list

substitution

[superpollo]

hi.

what is the most pythonic way to substitute substrings?

eg: i want to apply:

foo --> bar
baz --> quux
quuux --> foo

so that:

fooxxxbazyyyquuux --> barxxxquuxyyyfoo

bye
--
http://mail.python.org/mailman/listinfo/python-list

Re: More version woes

[Diez B. Roggisch]

Yep. They want $295, and I cannot justify that for personal use. I'll be
looking for a cheaper one.


There are bazillion discussions in this NG about IDEs and editors. GIYF.

Diez
--
http://mail.python.org/mailman/listinfo/python-list

Re: Generic Python Benchmark suite?

[Stefan Behnel]

Anand Vaidya, 18.01.2010 10:58:
> Is there a generic python benchmark suite in active development?
> [...]
> PS: I think a benchmark should cover file / network, database  I/O,
> data structures (dict, list etc), object creation/manipulation,
> numbers, measure looping inefficiencies, effects of caching (memcache
> etc) at the minimum

That doesn't sound generic at all. Maybe you should prefer an application
based benchmark instead.

Stefan
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: substitution

[Peter Otten]

superpollo wrote:

> superpollo ha scritto:
>> hi.
>> 
>> what is the most pythonic way to substitute substrings?
>> 
>> eg: i want to apply:
>> 
>> foo --> bar
>> baz --> quux
>> quuux --> foo
>> 
>> so that:
>> 
>> fooxxxbazyyyquuux --> barxxxquuxyyyfoo
>> 
>> bye
> 
> i explain better:
> 
> say the subs are:
> 
> quuux --> foo
> foo --> bar
> baz --> quux
> 
> then i cannot apply the subs in sequence (say, .replace() in a loop),
> otherwise:
> 
> fooxxxbazyyyquuux --> fooxxxbazyyyfoo --> barxxxbazyyybar -->
> barxxxquuxyyybar
> 
> not as intended...

If you want to avoid regular expressions:

def replace_many(s, pairs):
if len(pairs):
a, b = pairs[0]
rest = pairs[1:]
return b.join(replace_many(t, rest) for t in s.split(a))
else:
return s

assert replace_many("abc", ["ab", "bc", "ca"]) == "bca"
assert (replace_many("fooxxxbazyyyquuux",
 [("quuux", "foo"), ("foo", "bar"), ("baz", "quux")])
== "barxxxquuxyyyfoo")

Not tested.

Peter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: maintain 2 versions of python on my computer

[Gertjan Klein]

Duncan Booth wrote:

>   @(C:\Python26\Python -x %~f0 %* || pause) && goto:EOF
>   import sys
>   print sys.version
>   # raise RuntimeError # uncomment to trigger the 'pause'

That is nice! This should probably be in the documentation, here:

http://docs.python.org/using/cmdline.html#cmdoption-x

... instead of "This is intended for a DOS specific hack only.".

Gertjan.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: The answer

[Dave Angel]

Jive Dadson wrote:
alex23 
wrote:

>
> Actually, if you're using Python 2.6+/3.x, you can effectively skip
> steps 1-5, as these versions now support user site-packages.
>
> Rather than create a Module folder and modify your PYTHONPATH, add (if
> it doesn't exist already) the following folder:
> %APPDATA%/Python/Python26/site-packages
>
> Modules can sit directly in the folder, or within packages.
>
> For more details: http://www.python.org/dev/peps/pep-0370/

That requires a directory whose name embeds the Python version number, 
which is the evil from which I flee, or rather sought to flee.  
Imagine if all your C++ code had to go into directories that were 
named for some specific C++ compiler.  It's just WRONG.  It's a 
maintenance nightmare to have a bunch of different source files that 
all have to be updated whenever you fix a bug or add a feature.



As others have pointed out, you need a "deploy" script, which in your 
case would copy the files from source control to the appropriate 
production folder.  That's analogous to the compile, link and deploy 
steps of C++.  And if you want to be even more analogous, copy just the 
.pyc files, after building them.


Certainly you have lots more files in your version control system which 
are not intended to be copied to the "Modules" folder, such as the test 
suite.


DaveA

--
http://mail.python.org/mailman/listinfo/python-list

Re: Is python not good enough?

[Phlip]

On Jan 12, 7:09 am, ikuta liu  wrote:

> Go language try to merge low level, hight level and browser language.

Go uses := for assignment.

This means, to appease the self-righteous indignation of the math
professor who would claim = should mean "equality"...

...you gotta type a shift and 2 characters for a very common operator.

Pass!

--
  Phlip
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: substitution

[Steven D'Aprano]

On Mon, 18 Jan 2010 11:15:37 +0100, superpollo wrote:

> hi.
> 
> what is the most pythonic way to substitute substrings?
> 
> eg: i want to apply:
> 
> foo --> bar
> baz --> quux
> quuux --> foo
> 
> so that:
> 
> fooxxxbazyyyquuux --> barxxxquuxyyyfoo

For simple cases, just use replace:


>>> s = 'fooxxxbazyyyquuux'
>>> s = s.replace('foo', 'bar')
>>> s = s.replace('baz', 'quux')
>>> s = s.replace('quuux', 'foo')
>>> s == 'barxxxquuxyyyfoo'
True


In complicated cases, such as if there are conflicts or overlaps between 
strings, you may need to use a regular expression, or even parse the 
string yourself.

The problem is that "substitute multiple strings" is not well defined, 
because in general it depends on the order you perform them. You can 
define a function to do the replacements in one order, but for another 
use you might need a different order.

E.g.: replacing "a" -> "X" and "aa" -> "Y", if you have the string "aaa" 
what result do you expect? You could get "XXX", "XY" or "YX". None of 
these are wrong, it depends on which you prefer.


-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Is python not good enough?

[Tim Chase]

Phlip wrote:
On Jan 12, 7:09 am, ikuta liu  wrote: 

Go language try to merge low level, hight level and browser language.


Go uses := for assignment.

This means, to appease the self-righteous indignation of the math
professor who would claim = should mean "equality"...

...you gotta type a shift and 2 characters for a very common operator.

Pass!


Pass?!  no...Pascal! :-)

-tkc





--
http://mail.python.org/mailman/listinfo/python-list

Re: enhancing 'list'

[samwyse]

On Jan 18, 1:56 am, Terry Reedy  wrote:
> On 1/17/2010 5:37 PM, samwyse wrote:
>
>
>
>
>
> > Consider this a wish list.  I know I'm unlikely to get any of these in
> > time for for my birthday, but still I felt the need to toss it out and
> > see what happens.
>
> > Lately, I've slinging around a lot of lists, and there are some simple
> > things I'd like to do that just aren't there.
>
> > s.count(x[, cmp[, key]])
> > - return number of i‘s for which s[i] == x.  'cmp' specifies a custom
> > comparison function of two arguments, as in '.sort'.  'key' specifies
> > a custom key extraction function of one argument.
> > s.index(x[, i[, j[, cmp[, key)
> > - return smallest k such that s[k] == x and i<= k<  j.  'cmp' and
> > 'key' are as above.
> > s.rindex(x[, i[, j[, cmp[, key)
> > - return largest k such that s[k] == x and i<= k<  j.  'cmp' and
> > 'key' are as above.
>
> > There are two overlapping proposals here.  One is to add the .rindex
> > method, which strings already have.  The other is to extend the
> > optional arguments of .sort to all other methods that test for item
> > equality.
>
> > One last thing, the Python 2.6.2 spec says .count and .index only
> > apply to mutable sequence types.  I see no reason why they
> > (and .rindex) couldn't also apply to immutable sequences (tuples, in
> > particular).
>
> In 3.x, tuple does have those methods, even though the doc is not clear
> (unless fixed by now).

That's good to hear.  Perhaps I should have tried them directyly, but
my 3.1 docs still echo the 2.x docs, which only show them for
immutable sequences.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: enhancing 'list'

[samwyse]

On Jan 18, 3:06 am, Peter Otten <__pete...@web.de> wrote:
> samwyse wrote:
> > Lately, I've slinging around a lot of lists, and there are some simple
> > things I'd like to do that just aren't there.
>
> > s.count(x[, cmp[, key]])
> > - return number of i‘s for which s[i] == x.  'cmp' specifies a custom
> > comparison function of two arguments, as in '.sort'.  'key' specifies
> > a custom key extraction function of one argument.
>
> What's your use case exactly? If I were to enhance count/index/rindex I
> would go for the simpler
>
> >>> missing = object()                                                  
> >>> class List(list):                                                
>
> ...     def count(self, value=missing, predicate=missing):            
> ...             if value is missing:
> ...                     if predicate is missing:
> ...                             raise TypeError
> ...                     return sum(1 for item in self if predicate(item))
> ...             else:
> ...                     if predicate is not missing:
> ...                             raise TypeError
> ...                     return list.count(self, value)
> ...>>> items = List(range(10))
> >>> items.count(7)
> 1
> >>> items.count(predicate=lambda item: item%3)
>
> 6
>
> which nicely covers all applications I can imagine.
>
> Peter

That is a good idea.  However, I was looking more at the simplicity of
building of ideas that are already present in .sort.  And this
implementation is pretty simple as well.

>>> class List(list):
import __builtin__
def count(self, value, cmp=__builtin__.cmp):
return sum(1 for item in self if not cmp(item, value))


>>> items = List(range(10))
>>> items.count(7)
1
>>> items.count(3, lambda a, b: not a%b)  # My way
6
>>> items.count(Ellipsis, lambda a, b: not a%3)  # Your way
6

As a side note, wouldn't it be nice if '...' could be used in more
places than just slices?  IMHO, a useful idiom would be to use it to
signify "irrelevant" or "don't care", as opposed to 'None' which (in
my mind, at least) signifies "missing" or "unknown".
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: enhancing 'list'

[samwyse]

On Jan 17, 11:30 pm, Asun Friere  wrote:
> On Jan 18, 9:37 am, samwyse  wrote:
>
> > Consider this a wish list.  I know I'm unlikely to get any of these in
> > time for for my birthday, but still I felt the need to toss it out and
> > see what happens.
>
> > Lately, I've slinging around a lot of lists, and there are some simple
> > things I'd like to do that just aren't there.
>
> If memory serves me correctly, it has been possible to subclass 'built-
> in' types since Py2.2 or thereabouts.

True, but I've had bad experiences doing that.  See, for example,
http://groups.google.com/group/comp.lang.python/browse_thread/thread/10cfe2affc265ac
where I tried to subclass 'int'.  More importantly, subclassing means
that people have to keep re-inventing the same methods.  Having a
single implementation would save time, not to mention the speed
advantages of implementing them in the hosting language (C,
Java, .NET, etc).
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: enhancing 'list'

[Peter Otten]

samwyse wrote:

> On Jan 18, 3:06 am, Peter Otten <__pete...@web.de> wrote:
>> samwyse wrote:
>> > Lately, I've slinging around a lot of lists, and there are some simple
>> > things I'd like to do that just aren't there.
>>
>> > s.count(x[, cmp[, key]])
>> > - return number of i‘s for which s[i] == x.  'cmp' specifies a custom
>> > comparison function of two arguments, as in '.sort'.  'key' specifies
>> > a custom key extraction function of one argument.
>>
>> What's your use case exactly? If I were to enhance count/index/rindex I
>> would go for the simpler
>>
>> >>> missing = object()
>> >>> class List(list):
>>
>> ... def count(self, value=missing, predicate=missing):
>> ... if value is missing:
>> ... if predicate is missing:
>> ... raise TypeError
>> ... return sum(1 for item in self if predicate(item))
>> ... else:
>> ... if predicate is not missing:
>> ... raise TypeError
>> ... return list.count(self, value)
>> ...>>> items = List(range(10))
>> >>> items.count(7)
>> 1
>> >>> items.count(predicate=lambda item: item%3)
>>
>> 6
>>
>> which nicely covers all applications I can imagine.
>>
>> Peter
> 
> That is a good idea.  However, I was looking more at the simplicity of
> building of ideas that are already present in .sort.  And this
> implementation is pretty simple as well.

Note that the cmp() builtin and the cmp parameter for list.sort() are gone 
in Python 3.
 
 class List(list):
> import __builtin__
> def count(self, value, cmp=__builtin__.cmp):
> return sum(1 for item in self if not cmp(item, value))
> 
> 
 items = List(range(10))
 items.count(7)
> 1
 items.count(3, lambda a, b: not a%b)  # My way
> 6
 items.count(Ellipsis, lambda a, b: not a%3)  # Your way
> 6
> 
> As a side note, wouldn't it be nice if '...' could be used in more
> places than just slices?  IMHO, a useful idiom would be to use it to
> signify "irrelevant" or "don't care", as opposed to 'None' which (in
> my mind, at least) signifies "missing" or "unknown".

That is a pretty subtle distinction...

I prefer keyword arguments, but in Python 3 you can use the ellipsis literal 
freely:

>>> ... == ...
True
>>> [..., 42, ...].count(...)
2

Peter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: The answer

[samwyse]

On Jan 17, 8:30 pm, Jive Dadson  wrote:
> Okay, with your help I've figured it out.  Instructions are below, but
> read the caveat by Ben Fenny in this thread.  All this stuff is good for
> one default version of Python only.  The PYTHONPATH described below, for
> example, cannot specify a version number.  Yes, that's a pain in the
> butt, but there's no way around it.  If you switch versions, you may
> have to delete all the .pyc files that will show up in the module
> folders.  Python ought to check them to see if they are valid, but I do
> not know if it does so.
>
> These instructions are for MS Windows.
>
> 1) Create your modules folder. Let's say it's named "Modules."  The
> documentation calls it a "package."
>
> 2) In an explorer window or on the desktop, right click on My Computer,
> and select Properties.
>
> 3) Select the Advanced tab, and click on Environment Variables near the
> bottom.
>
> 4) Look for an environment variable named PYTHONPATH.
>
>     a) If you do not find one, create one using the New button(s). I
> don't know if it has to be in User Variables or System Variables.  To
> save time experimenting, I just put one in both. For the value, put the
> full path of the folder Modules.
>
>     b) If there's already a PYTHONPATH,  Edit it, adding a semi-colon
> and the full path of folder Module to the end.
>
> 5) Put your module folders into the folder Module.
>
> 6) (Here's a really arcane bit.) Into each module folder, put a file
> named __init__.py.  It will be executed when you load the module.  It
> can be empty, but it has to be there or else the module folder will be
> ignored.

In your original thread, you never quite said why you can't use site-
packages and .pth files.  Are you not allowed to modify your local
installation?  If you are writing something for distribution to
others, then site-packages and .pth files are the best way to go,
since they don't assume any particular operating system.  If you can't
(or won't) use them, then just create Module as a sub-directory of
wherever your program lives, since that directory is always prepended
to PYTHONPATH.  If you need to use the same module from multiple
directories, most modern operating systems support symbolic links; if
you're using Windows, well, here's a nickel kid, get yourself a better
computer (http://farm1.static.flickr.com/
89/240711122_f9888e5a3b_o.jpg).

I don't think that __init__.py is very arcane, since it is described
in detail in the documentation.  It's also a great place to use the
standard site.addsitedir() function, which is another platform
independent way to manipulate Python's search path.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: substitution

[Anthra Norell]

superpollo wrote:

hi.

what is the most pythonic way to substitute substrings?

eg: i want to apply:

foo --> bar
baz --> quux
quuux --> foo

so that:

fooxxxbazyyyquuux --> barxxxquuxyyyfoo

bye


Try the code below the dotted line. It does any number of substitutions 
and handles overlaps correctly (long over short)



Your case:

>>> substitutions = (('foo', 'bar'), ('baz', 'quux'), ('quuux', 
'foo'))   # Sequence of doublets

>>> T = Translator (substitutions)   # Compile substitutions -> translator
>>> s = 'fooxxxbazyyyquuux'   # Your source string
>>> d = 'barxxxquuxyyyfoo'# Your destination string
>>> print T (s)
barxxxquuxyyyfoo
>>> print T (s) == d
True


Frederic


-


class Translator:   


   r"""
   Will translate any number of targets, handling them correctly if 
some overlap.


   Making Translator
   T = Translator (definitions, [eat = 1])
   'definitions' is a sequence of pairs: ((target, 
substitute),(t2, s2), ...)
   'eat' says whether untargeted sections pass (translator) or 
are skipped (extractor).

   Makes a translator by default (eat = False)
   T.eat is an instance attribute that can be changed at 
any time.
   Definitions example: 
(('a','A'),('b','B'),('ab','ab'),('abc','xyz')   # ('ab','ab') see Tricks.

   ('\x0c', 'page break'), ('\r\n','\n'), ('   ','\t'))
   Order doesn't matter. 


   Running
   translation = T (source)

   Tricks
   Deletion:  ('target', '')
   Exception: (('\n',''), ('\n\n','\n\n')) # Eat LF except 
paragraph breaks.
   Exception: (('\n', '\r\n'), ('\r\n',\r\n')) # Unix to DOS, 
would leave DOS unchanged

   Translation cascade:
   # Rejoin text lines per paragraph Unix or DOS, inserting 
inter-word space if missing
   Mark_LF = Translator 
((('\n','+LF+'),('\r\n','+LF+'),('\r\n\r\n','\r\n\r\n'),('\n\n','\n\n')))
   # Pick positively identifiable mark for Unix and DOS end 
of lines
   Single_Space_Mark = Translator (((' +LF+', ' '),('+LF+', 
' '),('-+LF+', '')))

   no_lf_text = Single_Space_Mark (Mark_LF (text))
   Translation cascade:
   # Nesting calls
   reptiles = T_latin_english (T_german_latin (reptilien))

   Limitations
   1. The number of substitutions and the maximum size of input 
depends on the respective

   capabilities of the Python re module.
   2. Regular expressions will not work as such.

   Author:
   Frederic Rentsch (anthra.nor...@bluewin.ch).

   """


   def __init__ (self, definitions, eat = 0):

   '''
   definitions: a sequence of pairs of strings. ((target, 
substitute), (t, s), ...)
   eat: False (0) means translate: unaffected data passes 
unaltered.
True  (1) means extract:   unaffected data doesn't pass 
(gets eaten).
Extraction filters typically require substitutes to end 
with some separator,

else they fuse together. (E.g. ' ', '\t' or '\n')
   'eat' is an attribute that can be switched anytime.

   '''   
   self.eat = eat

   self.compile_sequence_of_pairs (definitions)
  
  
   def compile_sequence_of_pairs (self, definitions):


   '''
   Argument 'definitions' is a sequence of pairs:
   (('target 1', 'substitute 1'), ('t2', 's2'), ...)
   Order doesn't matter.


   '''
  
   import re

   self.definitions = definitions
   targets, substitutes = zip (*definitions)
   re_targets = [re.escape (item) for item in targets]
   re_targets.sort (reverse = True)
   self.targets_set = set (targets)  
   self.table = dict (definitions)

   regex_string = '|'.join (re_targets)
   self.regex = re.compile (regex_string, re.DOTALL)
  
  
   def __call__ (self, s):

   hits = self.regex.findall (s)
   nohits = self.regex.split (s)
   valid_hits = set (hits) & self.targets_set  # Ignore targets 
with illegal re modifiers.

   if valid_hits:
   substitutes = [self.table [item] for item in hits if item in 
valid_hits] + []  # Make lengths equal for zip to work right

   if self.eat:
   return ''.join (substitutes)
   else:   
   zipped = zip (nohits, substitutes)
   return ''.join (list (reduce (lambda a, b: a + b, 
[zipped][0]))) + nohits [-1]

   else:
   if self.eat:
   return ''
   else:
   return s



--
http://mail.python.org/mailman/listinfo/python-list

Re: substitution

[Iain King]

On Jan 18, 10:21 am, superpollo  wrote:
> superpollo ha scritto:
>
> > hi.
>
> > what is the most pythonic way to substitute substrings?
>
> > eg: i want to apply:
>
> > foo --> bar
> > baz --> quux
> > quuux --> foo
>
> > so that:
>
> > fooxxxbazyyyquuux --> barxxxquuxyyyfoo
>
> > bye
>
> i explain better:
>
> say the subs are:
>
> quuux --> foo
> foo --> bar
> baz --> quux
>
> then i cannot apply the subs in sequence (say, .replace() in a loop),
> otherwise:
>
> fooxxxbazyyyquuux --> fooxxxbazyyyfoo --> barxxxbazyyybar -->
> barxxxquuxyyybar
>
> not as intended...


Not sure if it's the most pythonic, but I'd probably do it like this:

def token_replace(string, subs):
subs = dict(subs)
tokens = {}
for i, sub in enumerate(subs):
tokens[sub] = i
tokens[i] = sub
current = [string]
for sub in subs:
new = []
for piece in current:
if type(piece) == str:
chunks = piece.split(sub)
new.append(chunks[0])
for chunk in chunks[1:]:
new.append(tokens[sub])
new.append(chunk)
else:
new.append(piece)
current = new
output = []
for piece in current:
if type(piece) == str:
output.append(piece)
else:
output.append(subs[tokens[piece]])
return ''.join(output)

>>> token_replace("fooxxxbazyyyquuux", [("quuux", "foo"), ("foo", "bar"), 
>>> ("baz", "quux")])
'barxxxquuxyyyfoo'

I'm sure someone could whittle that down to a handful of list comps...
Iain
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: enhancing 'list'

[samwyse]

On Jan 18, 6:20 am, Peter Otten <__pete...@web.de> wrote:
> Note that the cmp() builtin and the cmp parameter for list.sort() are gone
> in Python 3.

I've got Python 3 installed, and am using it for most new
development.  In this case case, however, I'm writing for the Google
App Engine, which is stuck at 2.5.  :(  (Curiously, no matter how I
order my PATH, the wrong version seems to appear first more than half
the time!  I'm seriously considering renaming all my Python 3 code to
use a .py3 file extension.)

> samwyse wrote:
> > As a side note, wouldn't it be nice if '...' could be used in more
> > places than just slices?  IMHO, a useful idiom would be to use it to
> > signify "irrelevant" or "don't care", as opposed to 'None' which (in
> > my mind, at least) signifies "missing" or "unknown".
>
> That is a pretty subtle distinction...

Hey, I'm a pretty subtle person...

> I prefer keyword arguments, but in Python 3 you can use the ellipsis literal
> freely:
>
> >>> ... == ...
> True
> >>> [..., 42, ...].count(...)
>
> 2
>
> Peter

I must have either missed or forgotten about that.  Thanks!
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: enhancing 'list'

[Stefan Behnel]

samwyse, 18.01.2010 13:49:
> Curiously, no matter how I
> order my PATH, the wrong version seems to appear first more than half
> the time!  I'm seriously considering renaming all my Python 3 code to
> use a .py3 file extension.

You should be able to start the interpreter as "python3.1" to be sure.

Stefan
-- 
http://mail.python.org/mailman/listinfo/python-list

Parse a log file

[kak...@gmail.com]

Hello to all!
I want to parse a log file with the following format for
example:
  TIMESTAMPEOperation FileName
Bytes
12/Jan/2010:16:04:59 +0200   EXISTS   sample3.3gp   37151
12/Jan/2010:16:04:59 +0200  EXISTSsample3.3gp   37151
12/Jan/2010:16:04:59 +0200  EXISTSsample3.3gp   37151
12/Jan/2010:16:04:59 +0200  EXISTSsample3.3gp   37151
12/Jan/2010:16:04:59 +0200  EXISTSsample3.3gp   37151
12/Jan/2010:16:05:05 +0200  DELETE  sample3.3gp   37151

How can i count the operations for a month(e.g total of 40 Operations,
30 exists, 10 delete?)
Any tips?

Thanks in advance
Antonis
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Is python not good enough?

[David Cournapeau]

On Mon, Jan 18, 2010 at 8:03 PM, Phlip  wrote:

> This means, to appease the self-righteous indignation of the math
> professor who would claim = should mean "equality"...

Much more likely, this is part of the stated goal of making go very
easy to analyse (to build tools and so that go is very fast to
compile), as stated in its FAQ.

cheers,

David
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: r"string" vs R"string

[Colin W.]

On 17-Jan-10 18:27 PM, Steven D'Aprano wrote:

On Sun, 17 Jan 2010 11:13:48 -0500, Roy Smith wrote:


In article,
  "Colin W."  wrote:


On 17-Jan-10 02:16 AM, Terry Reedy wrote:

On 1/17/2010 1:55 AM, Brendan Miller wrote:

Is there any difference whatsoever between a raw string beginning
with the captical R or one with the lower case r e.g. r"string" vs
R"string"?


No. Nor is there and difference between the strings created with raw
literals and cooked literals.



"cooked" literal  ??


I've never seen it referred to this way in the Python literature, but
"cooked" is a well-known term meaning, "not raw".  The usage goes back
decades.


I think the use of "cooked" meaning "not raw" goes back a little bit more
than decades. The verb "to cook" dates from the late 14th century, and
the adjective form "cooked" would follow soon after. The use of "cooked"
to mean manipulated (as in "cooking the books") comes from the 1630s.
Extending it to strings (as in raw versus cooked strings) is an obvious
extension.

Yes, I should have cottoned on, but perhaps "... between the strings, 
raw and other." would have conveyed the idea.


Colin W.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Parse a log file

[samwyse]

On Jan 18, 6:52 am, "kak...@gmail.com"  wrote:
> Hello to all!
> I want to parse a log file with the following format for
> example:
>               TIMESTAMPE            Operation     FileName
> Bytes
> 12/Jan/2010:16:04:59 +0200   EXISTS       sample3.3gp   37151
> 12/Jan/2010:16:04:59 +0200  EXISTS        sample3.3gp   37151
> 12/Jan/2010:16:04:59 +0200  EXISTS        sample3.3gp   37151
> 12/Jan/2010:16:04:59 +0200  EXISTS        sample3.3gp   37151
> 12/Jan/2010:16:04:59 +0200  EXISTS        sample3.3gp   37151
> 12/Jan/2010:16:05:05 +0200  DELETE      sample3.3gp   37151
>
> How can i count the operations for a month(e.g total of 40 Operations,
> 30 exists, 10 delete?)
> Any tips?
>
> Thanks in advance
> Antonis

time.strptime(string[, format])
Parse a string representing a time according to a format. The return
value is a struct_time as returned by gmtime() or localtime().

The format parameter uses the same directives as those used by strftime
(); it defaults to "%a %b %d %H:%M:%S %Y" which matches the formatting
returned by ctime(). If string cannot be parsed according to format,
or if it has excess data after parsing, ValueError is raised. The
default values used to fill in any missing data when more accurate
values cannot be inferred are (1900, 1, 1, 0, 0, 0, 0, 1, -1).

>>> import time
>>> ts='12/Jan/2010:16:04:59 +0200'
>>> time.strptime(ts[:-6], '%d/%b/%Y:%H:%M:%S')
time.struct_time(tm_year=2010, tm_mon=1, tm_mday=12, tm_hour=16,
tm_min=4, tm_sec=59, tm_wday=1, tm_yday=12, tm_isdst=-1)

I leave the conversion of the last six characters (the time zone
offset) as an exercise for the student.  :)
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: substitution

[Iain King]

On Jan 18, 12:41 pm, Iain King  wrote:
> On Jan 18, 10:21 am, superpollo  wrote:
>
>
>
> > superpollo ha scritto:
>
> > > hi.
>
> > > what is the most pythonic way to substitute substrings?
>
> > > eg: i want to apply:
>
> > > foo --> bar
> > > baz --> quux
> > > quuux --> foo
>
> > > so that:
>
> > > fooxxxbazyyyquuux --> barxxxquuxyyyfoo
>
> > > bye
>
> > i explain better:
>
> > say the subs are:
>
> > quuux --> foo
> > foo --> bar
> > baz --> quux
>
> > then i cannot apply the subs in sequence (say, .replace() in a loop),
> > otherwise:
>
> > fooxxxbazyyyquuux --> fooxxxbazyyyfoo --> barxxxbazyyybar -->
> > barxxxquuxyyybar
>
> > not as intended...
>
> Not sure if it's the most pythonic, but I'd probably do it like this:
>
> def token_replace(string, subs):
>         subs = dict(subs)
>         tokens = {}
>         for i, sub in enumerate(subs):
>                 tokens[sub] = i
>                 tokens[i] = sub
>         current = [string]
>         for sub in subs:
>                 new = []
>                 for piece in current:
>                         if type(piece) == str:
>                                 chunks = piece.split(sub)
>                                 new.append(chunks[0])
>                                 for chunk in chunks[1:]:
>                                         new.append(tokens[sub])
>                                         new.append(chunk)
>                         else:
>                                 new.append(piece)
>                 current = new
>         output = []
>         for piece in current:
>                 if type(piece) == str:
>                         output.append(piece)
>                 else:
>                         output.append(subs[tokens[piece]])
>         return ''.join(output)
>
> >>> token_replace("fooxxxbazyyyquuux", [("quuux", "foo"), ("foo", "bar"), 
> >>> ("baz", "quux")])
>
> 'barxxxquuxyyyfoo'
>
> I'm sure someone could whittle that down to a handful of list comps...
> Iain

Slightly better (lets you have overlapping search strings, used in the
order they are fed in):

def token_replace(string, subs):
tokens = {}
if type(subs) == dict:
for i, sub in enumerate(subs):
tokens[sub] = i
tokens[i] = subs[sub]
else:
s = []
for i, (k,v) in enumerate(subs):
tokens[k] = i
tokens[i] = v
s.append(k)
subs = s
current = [string]
for sub in subs:
new = []
for piece in current:
if type(piece) == str:
chunks = piece.split(sub)
new.append(chunks[0])
for chunk in chunks[1:]:
new.append(tokens[sub])
new.append(chunk)
else:
new.append(piece)
current = new
output = []
for piece in current:
if type(piece) == str:
output.append(piece)
else:
output.append(tokens[piece])
return ''.join(output)
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: substitution

[Peter Otten]

Iain King wrote:

> Not sure if it's the most pythonic, but I'd probably do it like this:
> 
> def token_replace(string, subs):
> subs = dict(subs)
> tokens = {}
> for i, sub in enumerate(subs):
> tokens[sub] = i
> tokens[i] = sub
> current = [string]
> for sub in subs:
> new = []
> for piece in current:
> if type(piece) == str:
> chunks = piece.split(sub)
> new.append(chunks[0])
> for chunk in chunks[1:]:
> new.append(tokens[sub])
> new.append(chunk)
> else:
> new.append(piece)
> current = new
> output = []
> for piece in current:
> if type(piece) == str:
> output.append(piece)
> else:
> output.append(subs[tokens[piece]])
> return ''.join(output)
> 
> >>> token_replace("fooxxxbazyyyquuux", [("quuux", "foo"), ("foo", "bar"), 
("baz", "quux")])
> 'barxxxquuxyyyfoo'
> 
> I'm sure someone could whittle that down to a handful of list comps...

I tried, but failed:

def join(chunks, separator):
chunks = iter(chunks)
yield next(chunks)
for chunk in chunks:
yield separator
yield chunk

def token_replace(string, subs):
tokens = {}

current = [string]
for i, (find, replace) in enumerate(subs):
tokens[i] = replace
new = []
for piece in current:
if piece in tokens:
new.append(piece)
else:
new.extend(join(piece.split(find), i))
current = new

return ''.join(tokens.get(piece, piece) for piece in current)

You could replace the inner loop with sum(..., []), but that would be really 
ugly.

Peter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: substitution

[superpollo]

it looked simpler when i posted, but i realize that the problem is non 
trivial.


thanks to everybody.

i guess that the algorithm would be easier if it was known in advance 
that the string to substitute must have some specific property, say:


1) they all must start with "XYZ"
2) they all have the same length N (e.g. 5)

like this:

qweXYZ12asdXYZ1345XYZ ---> qweIWAS12asdIWAS1345XYZ

bye
--
http://mail.python.org/mailman/listinfo/python-list

Re: Is python not good enough?

[Anh Hai Trinh]

On Jan 18, 6:03 pm, Phlip  wrote:
> On Jan 12, 7:09 am, ikuta liu  wrote:
>
> > Go language try to merge low level, hight level and browser language.
>
> Go uses := for assignment.

Except that it doesn't. := is a declaration.

s := "foo"

is short for

var s string = "foo"


Cheers,

aht
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Changing var names

[Jean-Michel Pichavant]

Victor Subervi wrote:
On Fri, Jan 15, 2010 at 3:15 PM, Adam Tauno Williams 
mailto:awill...@opengroupware.us>> wrote:


On Fri, 2010-01-15 at 13:27 -0400, Victor Subervi wrote:
> Hi;
> Well it took me *less than a day* to fix the following problems:
> -- bare excepts (accidentally left a couple I think)
> -- sql injection attacks
> -- recreating tables to make them more reasonable
> Now, I believe someone once mentioned that there is some s/w
tool out
> there for changing the names of my variables more easily than going
> through every script one-by-one. If so, please share with me again
> what it is.

This process is called 'refactoring' [a good term to Google], and
every
decent IDE provides some support [if it doesn't, it isn't a "decent"
IDE]


Thanks. I'll work with Eric, I guess, once I get my Mac.
beno
Here are 2 well known IDEs, that help with refactoring (and many other 
things).

http://netbeans.org/
http://www.eclipse.org/

Both supports python.

I personnaly will stick to vim :o)

JM
--
http://mail.python.org/mailman/listinfo/python-list

Re: Inheriting methods but over-riding docstrings

[Gabriel Genellina]

En Sun, 17 Jan 2010 23:23:45 -0300, Steve Holden   
escribió:

Gabriel Genellina wrote:



Methods don't have docstrings; functions do. So one has to "clone" the
function to set a new docstring.


On behalf of all methods I would like to say "We demand the right to
docstrings".


print abs.__doc__

abs(number) -> number

Return the absolute value of the argument.




Perhaps I am misunderstanding "don't have docstrings". This code:


Sorry, I wasn't clear at all. What I wanted to say is that docstrings are  
actually stored as *function* attributes; methods inherit the docstring  
from the function they are built on (as a read-only attribute).



py> class Base(object):
...   def method(self):
... "my docstring"
...
py> Base.method.__doc__
'my docstring'
py> Base.method.__doc__ = "another docstring"
Traceback (most recent call last):
  File "", line 1, in 
AttributeError: attribute '__doc__' of 'instancemethod' objects is not  
writable

py> Base.method.im_func.__doc__
'my docstring'
py> Base.method.im_func.__doc__ = "another docstring"
py> Base.method.__doc__
'another docstring'


So, if you want Base.foo and Derived.foo to share the same behavior but  
have separate docstrings (what Steven asked for), you have to create  
separate functions for both. That's what my already posted code does.



So, are the demands of the methods reasonable, or do they already have
what they are so loudly demanding?


Well, they could demand the right to have a docstring of their own. Since  
3.x eliminated a whole category of methods (unbound methods are gone),  
their ability to press for demands was greatly reduced. I don't think they  
could get such right granted in the near future...


--
Gabriel Genellina

--
http://mail.python.org/mailman/listinfo/python-list

substitution

[Adi Eyal]

> From: superpollo 
> To:
> Date: Mon, 18 Jan 2010 11:15:37 +0100
> Subject: substitution
> hi.
>
> what is the most pythonic way to substitute substrings?
>
> eg: i want to apply:
>
> foo --> bar
> baz --> quux
> quuux --> foo
>
> so that:
>
> fooxxxbazyyyquuux --> barxxxquuxyyyfoo
>
> bye


Using regular expressions the answer is short (and sweet)

mapping = {
"foo" : "bar",
"baz" : "quux",
"quuux" : "foo"
}

pattern = "(%s)" % "|".join(mapping.keys())
repl = lambda x : mapping.get(x.group(1), x.group(1))
s = "fooxxxbazyyyquuux"
re.subn(pattern, repl, s)
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: substitution

[Iain King]

On Jan 18, 2:17 pm, Adi Eyal  wrote:
> > From: superpollo 
> > To:
> > Date: Mon, 18 Jan 2010 11:15:37 +0100
> > Subject: substitution
> > hi.
>
> > what is the most pythonic way to substitute substrings?
>
> > eg: i want to apply:
>
> > foo --> bar
> > baz --> quux
> > quuux --> foo
>
> > so that:
>
> > fooxxxbazyyyquuux --> barxxxquuxyyyfoo
>
> > bye
>
> Using regular expressions the answer is short (and sweet)
>
> mapping = {
>         "foo" : "bar",
>         "baz" : "quux",
>         "quuux" : "foo"
>
> }
>
> pattern = "(%s)" % "|".join(mapping.keys())
> repl = lambda x : mapping.get(x.group(1), x.group(1))
> s = "fooxxxbazyyyquuux"
> re.subn(pattern, repl, s)

Winner! :)

Iain
-- 
http://mail.python.org/mailman/listinfo/python-list

python gui ide under linux..like visual studio ;) ?

[ted]

Hi at all...
Can someone please give me some advice, about a good IDE with control
GUI under Linux ?

Actually i know QT Creator by Nokia which i can use with Python (but i
don't know how).

And, a good library for access to database (mysql, sql server, oracle) ?

Thank you very much !

bye
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: multithreading, performance, again...

[Oktaka Com]

On 30 Aralık 2009, 17:44, mk  wrote:
> Hello everyone,
>
> I have figured out (sort of) how to do profiling of multithreaded
> programs with cProfile, it goes something like this:
>
> #!/usr/local/bin/python
>
> import cProfile
> import threading
>
> class TestProf(threading.Thread):
>      def __init__(self, ip):
>
>          threading.Thread.__init__(self)
>          self.ip = ip
>
>      def run(self):
>          prof = cProfile.Profile()
>          retval = prof.runcall(self.runmethod)
>          prof.dump_stats('tprof' + self.ip)
>
>      def runmethod(self):
>          pass
>
> tp = TestProf('10.0.10.10')
>
> tp.start()
> tp.join()
>
> The problem is, now that I've done profiling in the actual program
> (profiled version here:http://python.domeny.com/cssh_profiled.py) with
> 9 threads and added up stats (using pstats.Stats.add()), the times I get
> are trivial:
>
>  >>> p.strip_dirs().sort_stats('cumulative').print_stats(10)
> Wed Dec 30 16:23:59 2009    csshprof9.156.44.113
> Wed Dec 30 16:23:59 2009    csshprof9.156.46.243
> Wed Dec 30 16:23:59 2009    csshprof9.156.46.89
> Wed Dec 30 16:24:00 2009    csshprof9.156.47.125
> Wed Dec 30 16:24:00 2009    csshprof9.156.47.17
> Wed Dec 30 16:24:00 2009    csshprof9.156.47.29
> Wed Dec 30 16:24:01 2009    csshprof9.167.41.241
> Wed Dec 30 16:24:02 2009    csshprof9.168.119.15
> Wed Dec 30 16:24:02 2009    csshprof9.168.119.218
>
>           39123 function calls (38988 primitive calls) in 6.004 CPU seconds
>
>     Ordered by: cumulative time
>     List reduced from 224 to 10 due to restriction <10>
>
>     ncalls  tottime  percall  cumtime  percall filename:lineno(function)
>          9    0.000    0.000    6.004    0.667 cssh.py:696(runmethod)
>        100    0.004    0.000    5.467    0.055 threading.py:389(wait)
>         82    0.025    0.000    5.460    0.067 threading.py:228(wait)
>        400    5.400    0.013    5.400    0.013 {time.sleep}
>          9    0.000    0.000    5.263    0.585 cssh.py:452(ssh_connect)
>          9    0.003    0.000    5.262    0.585 client.py:226(connect)
>          9    0.001    0.000    2.804    0.312
> transport.py:394(start_client)
>          9    0.005    0.001    2.254    0.250 client.py:391(_auth)
>         18    0.001    0.000    2.115    0.117
> transport.py:1169(auth_publickey)
>         18    0.001    0.000    2.030    0.113
> auth_handler.py:156(wait_for_response)
>
> 
>
> It's not burning CPU time in the main thread (profiling with cProfile
> indicated smth similar to the above), it's not burning it in the
> individual worker threads - so where the heck it is burning this CPU
> time? bc 'top' shows heavy CPU load during most of the time of the
> program run.
>
> help...
>
> regards,
> mk

See http://code.google.com/p/yappi/ if you want to profile
multithreaded python app.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: substitution

[Mel]

superpollo wrote:

> hi.
> 
> what is the most pythonic way to substitute substrings?
> 
> eg: i want to apply:
> 
> foo --> bar
> baz --> quux
> quuux --> foo
> 
> so that:
> 
> fooxxxbazyyyquuux --> barxxxquuxyyyfoo

This is simple -- maybe a bit brutal -- and if the strings get long it will 
be slow:


def replaced (input, replacements):
output = []
while input:
for target in replacements:
if input.startswith (target):
output.append (replacements [target])
input = input [len (target):]
break
else:
output.append (input[0])
input = input [1:]
return ''.join (output)

original = 'fooxxxbazyyyquux'
replacements = {'quux':'foo', 'foo':'bar', 'baz':'quux'}
print original, '-->', replaced (original, replacements)




Mel.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: basic Class in Python

[bartc]

"Wolfgang Rohdewald"  wrote in message 
news:mailman.1056.1263771299.28905.python-l...@python.org...

On Monday 18 January 2010, BarryJOgorman wrote:

TypeError: object._new_() takes no parameters



def _init_(self, name, job=None, pay=0):


__init__ needs two underscores left and right


Any particular reason why two, and not one (or three)? In some fonts it's 
difficult to tell how many as they run together.


--
Bartc 


--
http://mail.python.org/mailman/listinfo/python-list

Proper display of XMLRPCserver exceptions

[Jean-Michel Pichavant]

To all using xmlrpclib,

I had trouble getting a proper display of my exceptions occuring on the 
server. Basically, xmlrpclib only display the exception itself without 
any traceback, on the client side.


Something like
:2">

Ok then there's a key error, but how the hell am I supposed to know 
where it occured ?


So I wrote this:
from SimpleXMLRPCServer import SimpleXMLRPCServer
server = SimpleXMLRPCServer(('localhost', 8000), logRequests=False, 
allow_none=True)

old_dispatch = server._dispatch
def _new_dispatch(method, params):
   try:
   return old_dispatch(method,params)
   except Exception, exc:
   import traceback
   traceback.print_exc()
   raise
server._dispatch = _new_dispatch


Pretty dirty, isn't it, accessing the private attributes (too lazy to 
subclass) ! I guess there's an obvious way to do this with xmlrpclib 
that I just missed. Anyone knows ?


JM
--
http://mail.python.org/mailman/listinfo/python-list

Re: substitution

[Nobody]

On Mon, 18 Jan 2010 11:21:54 +0100, superpollo wrote:

>> what is the most pythonic way to substitute substrings?

> say the subs are:
> 
> quuux --> foo
> foo --> bar
> baz --> quux
> 
> then i cannot apply the subs in sequence (say, .replace() in a loop), 
> otherwise:
> 
> fooxxxbazyyyquuux --> fooxxxbazyyyfoo --> barxxxbazyyybar --> 
> barxxxquuxyyybar
> 
> not as intended...

Are there any characters which are guaranteed never to occur in the
string? If so:

s = s.replace('quuux', '@1').replace('foo', '@2').replace('baz', '@3')
s = s.replace('@1', 'foo').replace('@2', 'bar').replace('@3', 'quux')

E.g.:

def replace(subs, s):
  for i, (src, dst) in enumerate(subs):
s = s.replace(src, '@%06d' % i)
  for i, (src, dst) in enumerate(subs):
s = s.replace('@%06d' % i, dst)
  return s

Not the most efficient solution (and problematic if there aren't any
"unused" characters), but somewhat shorter than the other solutions.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Is python not good enough?

[Phlip]

On Jan 18, 5:59 am, Anh Hai Trinh  wrote:

> > Go uses := for assignment.
>
> Except that it doesn't. := is a declaration.

Ah, and that's why Go is easy for cheap parsers to rip.

Tx all!

I was formerly too mortified to proceed - now I'm back in the Go camp.
They fixed the hideous redundancy of Java without the ill-defined
scope issues of Python & Ruby, and without the tacky little 'var' of
JavaScript!

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Generic Python Benchmark suite?

[Dotan Cohen]

> What do you suggest?
>

$ man time


-- 
Dotan Cohen

http://what-is-what.com
http://gibberish.co.il
-- 
http://mail.python.org/mailman/listinfo/python-list

not and is not problem

[superpollo]

hi:

#!/usr/bin/env python
data = "seq=123"
name , value = data.split("=")
print name
print value
if not name == "seq":
print "DOES NOT PRINT OF COURSE..."
if name is not "seq":
print "WTF! WHY DOES IT PRINT?"

help please.

bye
--
http://mail.python.org/mailman/listinfo/python-list

ANN: shpaml 0.94b

[Steve Howell]

Hi everybody, I would like to announce the latest version of shpaml,
which is an indentation-based markup language similar to haml, but
different.

Shpaml allows you to author HTML-like content with an indentation-
based syntax that eliminates the need to write and read close tags and
angle brackets.  It also has haml-like syntax for specifying divs,
classes, and ids, which also makes your documents easier to read,
particularly when you are also using CSS and jQuery-like tools.
Finally, it is very transparent with respect to HTML semantics, as
well as letting most HTML syntax pass through as needed.

It is probably best understood by simply looking at a few examples:

http://shpaml.webfactional.com/examples

There is also a tutorial, and you can try out your own examples there,
so need to download the software if you just want to poke around a
bit:

http://shpaml.webfactional.com/tutorial/1

If you do decide to download it, it is a pretty small module at 225
lines or so:

http://shpaml.webfactional.com/source_code

The project is about two months old now, and it is being actively
maintained.   We have a mailing list, users submitting patches, all
that good stuff.

I am labelling the current version as "beta," but it has been well
tested on over a thousand lines of markup.

Cheers,

Steve
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: not and is not problem

[Stephen Hansen]

On Mon, Jan 18, 2010 at 7:43 AM, superpollo  wrote:

> #!/usr/bin/env python
> data = "seq=123"
> name , value = data.split("=")
> print name
> print value
> if not name == "seq":
>print "DOES NOT PRINT OF COURSE..."
> if name is not "seq":
>print "WTF! WHY DOES IT PRINT?"
>

Because name really is not "seq"; "seq" is an entirely new and different
string. The "is" operator tests for object identity, == tests for equality.
"name is name" will always return True, everything else will always return
False.

For example, you can do:
other = name
if name is other:
print "OK"

And it'll work, because those are the exact same object.

You only use 'is' and 'is not' when you really want to ask, 'Is this the
precisely same object?' Generally, that's only when you're testing if
something is or is not None. Sometimes is or is not True/False, but usually
'if x' is preferred.

--S
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: not and is not problem

[D'Arcy J.M. Cain]

On Mon, 18 Jan 2010 16:43:25 +0100
superpollo  wrote:
> hi:
> if name is not "seq":
>  print "WTF! WHY DOES IT PRINT?"

Perhaps they aren't the same object.  Some values are guaranteed to be
the same (e.g. True and False) but not all.  Try "!=" instead of "is
not" in that experssion.

-- 
D'Arcy J.M. Cain  |  Democracy is three wolves
http://www.druid.net/darcy/|  and a sheep voting on
+1 416 425 1212 (DoD#0082)(eNTP)   |  what's for dinner.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: substitution

[Anthra Norell]

Anthra Norell wrote:

superpollo wrote:

hi.

what is the most pythonic way to substitute substrings?

eg: i want to apply:

foo --> bar
baz --> quux
quuux --> foo

so that:

fooxxxbazyyyquuux --> barxxxquuxyyyfoo

bye


So it goes. The more it matters, the sillier the misatakes. The method 
__init__ () was incomplete and __call__ () was missing, Sorry abount 
that. Here the whole thing again:



class Translator:  
  r"""
  Will translate any number of targets, handling them correctly if 
some overlap.


  Making Translator
  T = Translator (definitions, [eat = 1])
  'definitions' is a sequence of pairs: ((target, 
substitute),(t2, s2), ...)
  'eat' says whether untargeted sections pass (translator) or 
are skipped (extractor).

  Makes a translator by default (eat = False)
  T.eat is an instance attribute that can be changed at any 
time.
  Definitions example: 
(('a','A'),('b','B'),('ab','ab'),('abc','xyz')   # ('ab','ab') see Tricks.

  ('\x0c', 'page break'), ('\r\n','\n'), ('   ','\t'))
  Order doesn't matter.
  Running

  translation = T (source)

  Tricks
  Deletion:  ('target', '')
  Exception: (('\n',''), ('\n\n','\n\n')) # Eat LF except 
paragraph breaks.
  Exception: (('\n', '\r\n'), ('\r\n',\r\n')) # Unix to DOS, 
would leave DOS unchanged

  Translation cascade:
  # Rejoin text lines per paragraph Unix or DOS, inserting 
inter-word space if missing
  Mark_LF = Translator 
((('\n','+LF+'),('\r\n','+LF+'),('\r\n\r\n','\r\n\r\n'),('\n\n','\n\n')))
  # Pick positively identifiable mark for Unix and DOS end 
of lines   Single_Space_Mark = Translator (((' +LF+', ' 
'),('+LF+', ' '),('-+LF+', '')))

  no_lf_text = Single_Space_Mark (Mark_LF (text))
  Translation cascade:
  # Nesting calls
  reptiles = T_latin_english (T_german_latin (reptilien))

  Limitations
  1. The number of substitutions and the maximum size of input 
depends on the respective

  capabilities of the Python re module.
  2. Regular expressions will not work as such.

  Author:
  Frederic Rentsch (anthra.nor...@bluewin.ch).
  """

   def __init__ (self, definitions, eat = 0):

   '''
   definitions: a sequence of pairs of strings. ((target, 
substitute), (t, s), ...)
   eat: False (0) means translate: unaffected data passes 
unaltered.
True  (1) means extract:   unaffected data doesn't pass 
(gets eaten).
Extraction filters typically require substitutes to end 
with some separator,

else they fuse together. (E.g. ' ', '\t' or '\n')
   'eat' is an attribute that can be switched anytime.

   '''   
   self.eat = eat

   self.compile_sequence_of_pairs (definitions)
  
  
   def compile_sequence_of_pairs (self, definitions):


   '''
   Argument 'definitions' is a sequence of pairs:
   (('target 1', 'substitute 1'), ('t2', 's2'), ...)
   Order doesn't matter.


   '''
  
   import re

   self.definitions = definitions
   targets, substitutes = zip (*definitions)
   re_targets = [re.escape (item) for item in targets]
   re_targets.sort (reverse = True)
   self.targets_set = set (targets)  
   self.table = dict (definitions)

   regex_string = '|'.join (re_targets)
   self.regex = re.compile (regex_string, re.DOTALL)
  
  
   def __call__ (self, s):

   hits = self.regex.findall (s)
   nohits = self.regex.split (s)
   valid_hits = set (hits) & self.targets_set  # Ignore targets 
with illegal re modifiers.

   if valid_hits:
   substitutes = [self.table [item] for item in hits if item in 
valid_hits] + []  # Make lengths equal for zip to work right

   if self.eat:
   return ''.join (substitutes)
   else:   
   zipped = zip (nohits, substitutes)
   return ''.join (list (reduce (lambda a, b: a + b, 
[zipped][0]))) + nohits [-1]

   else:
   if self.eat:
   return ''
   else:
   return s

--
http://mail.python.org/mailman/listinfo/python-list

Re: not and is not problem

[Arnaud Delobelle]

On 18 Jan, 15:43, superpollo  wrote:
> hi:
>
> #!/usr/bin/env python
> data = "seq=123"
> name , value = data.split("=")
> print name
> print value
> if not name == "seq":
>      print "DOES NOT PRINT OF COURSE..."
> if name is not "seq":
>      print "WTF! WHY DOES IT PRINT?"
>
> help please.
>
> bye

is and == are different operators.

* A == B means A.__eq__(B)

* A is B means that A and B are the same object, living at
the same physical location in the computer's memory.

HTH

--
Arnaud
-- 
http://mail.python.org/mailman/listinfo/python-list

What is a list compression in Python?

[Kit]

Hello Everyone, I am not sure if I have posted this question in a
correct board. Can anyone please teach me:

What is a list compression in Python?

Would you mind give me some list compression examples?


Thanks & really appreciate that.
Kit
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: not and is not problem

[J]

On Mon, Jan 18, 2010 at 10:43, superpollo  wrote:
> hi:
>
> #!/usr/bin/env python
> data = "seq=123"
> name , value = data.split("=")
> print name
> print value
> if not name == "seq":
>    print "DOES NOT PRINT OF COURSE..."
> if name is not "seq":
>    print "WTF! WHY DOES IT PRINT?"

is will return True if two variables point to the same object, == if
the objects referred to by the variables are equal.

>> if not name == "seq":

says if the object that the variable name points to is NOT the same as
the string "seq" then do the following.
Since the object that "name" points to contains the string "seq" and
the string "seq" is identical in value to the "seq" in your
comparison, the result is TRUE (b = a) and your if statment only
proceeds if the comparison result is FALSE.

>> if name is not "seq"

is and is not relate to pointing to an object, not the object's contents.

Example:

>> name = "foo"
>> name1 = name
>> print name
foo
>> print name1
foo
>> name is name1
True
>> name1 is name
True
>> name = "bar"
>> print name
bar
>> print name1
foo
>> name is name1
False
>> name is not name1
True

Or even better:

>> name = "foo"
>> name1 = name
>> id(name)
11875840
>> id(name1)
11875840
>> name = "bar"
>> id(name)
11875744
>> id(name1)
11875840
>> id("foo")
11875840
>> id("bar")
11875744

shows the location for each object in relation to the name pointing to
that object...

-- 

Joan Crawford  - "I, Joan Crawford, I believe in the dollar.
Everything I earn, I spend." -
http://www.brainyquote.com/quotes/authors/j/joan_crawford.html
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: What is a list compression in Python?

[superpollo]

Kit ha scritto:

Hello Everyone, I am not sure if I have posted this question in a
correct board. Can anyone please teach me:

What is a list compression in Python?

Would you mind give me some list compression examples?


Thanks & really appreciate that.
Kit


i think that's compreHENsion...

http://www.python.org/doc/2.5.4/tut/node7.html#SECTION00714
--
http://mail.python.org/mailman/listinfo/python-list

Re: What is a list compression in Python?

[Simon Brunning]

2010/1/18 Kit :
> Hello Everyone, I am not sure if I have posted this question in a
> correct board. Can anyone please teach me:
>
> What is a list compression in Python?

Perhaps you mean a list comprehension? If so, see
.

-- 
Cheers,
Simon B.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: substitution

[Duncan Booth]

Adi Eyal  wrote:

> Using regular expressions the answer is short (and sweet)
> 
> mapping = {
> "foo" : "bar",
> "baz" : "quux",
> "quuux" : "foo"
> }
> 
> pattern = "(%s)" % "|".join(mapping.keys())
> repl = lambda x : mapping.get(x.group(1), x.group(1))
> s = "fooxxxbazyyyquuux"
> re.subn(pattern, repl, s)

I'd use a def as being IMHO clearer than the lambda but YMMV. More 
importantly I'd also escape the keys in case they contain any characters 
special to regular expressions:

>>> mapping = {
"foo" : "bar",
"baz" : "quux",
"quuux" : "foo"
}
>>> import re
>>> pattern = "(%s)" % "|".join(re.escape(k) for k in mapping)
>>> def repl(x): return mapping[x.group(1)]

>>> s = "fooxxxbazyyyquuux"
>>> re.subn(pattern, repl, s)
('barxxxquuxyyyfoo', 3)
>>> re.sub(pattern, repl, s)
'barxxxquuxyyyfoo'
>>> 


-- 
Duncan Booth http://kupuguy.blogspot.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: not and is not problem

[Steven D'Aprano]

On Mon, 18 Jan 2010 16:43:25 +0100, superpollo wrote:

> hi:
> 
> #!/usr/bin/env python
> data = "seq=123"
> name , value = data.split("=")
> print name
> print value
> if not name == "seq":
>  print "DOES NOT PRINT OF COURSE..."
> if name is not "seq":
>  print "WTF! WHY DOES IT PRINT?"


`is` is not an alternative spelling for `==`. `is` tests for object 
identity, not equality. Unless you care about object identity, always use 
equals.

>>> 'seq 1' is 'seq 1'  # the SAME object is used twice
True
>>> s = "seq 1"  # TWO objects are used
>>> t = "seq 1"
>>> s is t
False


Beware: Python caches strings that look like identifiers. This will 
include "seq". So, purely as an implementation-specific detail, Python 
will sometimes re-use the same string object, and sometimes not.

Do not use `is` when you are testing for equality.

By the way, instead of 

not name == "seq"

you should write

name != "seq"


-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: What is a list compression in Python?

[Stephen Hansen]

On Mon, Jan 18, 2010 at 8:07 AM, Kit  wrote:

> Hello Everyone, I am not sure if I have posted this question in a
> correct board. Can anyone please teach me:
>
> What is a list compression in Python?
>
> Would you mind give me some list compression examples?
>

Do you mean list *comprehension*? If so, its a special syntax for list
construct which can be used for shorter, clearer code (provided one doesn't
abuse it, at which point it becomes quite obtuse). Its never required:
nothing you do with a list comprehension you couldn't do with a standard
loop.

For example:

evens = []
for n in range(100):
if n %2 == 0:
evens.append(n)

verses:

my_list = [n for n in range(100) if n % 2 == 0]

The basic syntax is:

[ for  in ]

The 'if' part at the end is optional. The syntax is converted into a for
loop that builds a list, and then returns it to you.

So, basically it becomes:

temp = []
for  in :
temp.append()

Except the 'temp' variable doesn't really have  a name and is returned from
the comprehension, where you can give it a name. Sometimes you need an 'if'
clause, sometimes you don't. Sometimes your 'expression to be appended' is
very simple, other times you mutate it. For example, an easy way to convert
a list of numbers into a list of strings is:

[str(x) for x in range(10)]

HTH,

--S
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: substitution

[Anthra Norell]

superpollo wrote:

hi.

what is the most pythonic way to substitute substrings?

eg: i want to apply:

foo --> bar
baz --> quux
quuux --> foo

so that:

fooxxxbazyyyquuux --> barxxxquuxyyyfoo

bye





Third attempt. Clearly something doesn't work right. My code gets 
clipped on the way up. I have to send it as an attachment. Here's again 
what it does:


>>> substitutions = (('foo', 'bar'), ('baz', 'quux'), ('quuux', 
'foo'))   # Sequence of doublets

>>> T = Translator (substitutions)   # Compile substitutions -> translator
>>> s = 'fooxxxbazyyyquuux'   # Your source string
>>> d = 'barxxxquuxyyyfoo'# Your destination string
>>> print T (s)
barxxxquuxyyyfoo
>>> print T (s) == d
True


Code attached

Regards

Frederic


class Translator:

r"""
Will translate any number of targets, handling them correctly 
if some overlap.

Making Translator
T = Translator (definitions, [eat = 1])
'definitions' is a sequence of pairs: ((target, 
substitute),(t2, s2), ...)
'eat = True' will make an extraction filter that lets 
only the replaced targets pass.
Definitions example: 
(('a','A'),('b','B'),('ab','ab'),('abc','xyz'),
   ('\x0c', 'page break'), ('\r\n','\n'), ('   ','\t')) 
  # ('ab','ab') see Tricks.
Order doesn't matter.  

Running
translation = T (source)

Tricks 
Deletion:  ('target', '')
Exception: (('\n',''), ('\n\n','\n\n')) # Eat LF 
except paragraph breaks.
Exception: (('\n', '\r\n'), ('\r\n',\r\n')) # Unix to 
DOS, would leave DOS unchanged
Translation cascade: 
# Unwrap paragraphs, Unix or DOS, restoring 
inter-word space if missing,
Mark_LF = Translator 
((('\n','+LF+'),('\r\n','+LF+'),('\n\n','\n\n'),('\r\n\r\n','\r\n\r\n')))
# Pick any positively identifiable mark for end 
of lines in either Unix or MS-DOS.   
Single_Space_Mark = Translator (((' +LF+', ' 
'),('+LF+', ' '),('-+LF+', '')))
no_lf_text = Single_Space_Mark (Mark_LF (text))
Translation cascade: 
# Nested calls
reptiles = T_latin_english (T_german_latin 
(reptilien))

Limitations
1. The number of substitutions and the maximum size of 
input depends on the respective 
capabilities of the Python re module.
2. Regular expressions will not work as such.

Author:
Frederic Rentsch (i...@anthra-norell.ch).
 
"""

def __init__ (self, definitions, eat = 0):

'''
definitions: a sequence of pairs of strings. ((target, 
substitute), (t, s), ...)
eat: False (0) means translate: unaffected data passes 
unaltered.
 True  (1) means extract:   unaffected data doesn't 
pass (gets eaten).
 Extraction filters typically require substitutes 
to end with some separator, 
 else they fuse together. (E.g. ' ', '\t' or '\n') 
'eat' is an attribute that can be switched anytime.

''' 
self.eat = eat
self.compile_sequence_of_pairs (definitions)


def compile_sequence_of_pairs (self, definitions):

'''
Argument 'definitions' is a sequence of pairs:
(('target 1', 'substitute 1'), ('t2', 's2'), ...)
Order doesn't matter. 

'''

import re
self.definitions = definitions
targets, substitutes = zip (*definitions)
re_targets = [re.escape (item) for item in targets]
re_targets.sort (reverse = True)
self.targets_set = set (targets)   
self.table = dict (definitions)
regex_string = '|'.join (re_targets)
self.regex = re.compile (regex_string, re.DOTALL)


def __call__ (self, s):
hits = self.regex.findall (s)
nohits = self.regex.split (s)
valid_hits = set (hits) & self.targets_set  # Ignore targets 
with illegal re modifiers.
if valid_hits:
substitutes = [self.table [item] for item in hits if 
it

Re: substitution

[Steven D'Aprano]

On Mon, 18 Jan 2010 06:23:44 -0800, Iain King wrote:

> On Jan 18, 2:17 pm, Adi Eyal  wrote:
[...]
>> Using regular expressions the answer is short (and sweet)
>>
>> mapping = {
>>         "foo" : "bar",
>>         "baz" : "quux",
>>         "quuux" : "foo"
>>
>> }
>>
>> pattern = "(%s)" % "|".join(mapping.keys())
>> repl = lambda x : mapping.get(x.group(1), x.group(1)) 
>> s = "fooxxxbazyyyquuux"
>> re.subn(pattern, repl, s)
> 
> Winner! :)

What are the rules for being declared "Winner"? For the simple case 
given, calling s.replace three times is much faster: more than twice as 
fast.

But a bigger problem is that the above "winner" may not work correctly if 
there are conflicts between the target strings (e.g. 'a'->'X', 
'aa'->'Y'). The problem is that the result you get depends on the order 
of the searches, BUT as given, that order is non-deterministic. 
dict.keys() returns in an arbitrary order, which means the caller can't 
specify the order except by accident. For example:

>>> repl = lambda x : m[x.group(1)]
>>> m = {'aa': 'Y', 'a': 'X'}
>>> pattern = "(%s)" % "|".join(m.keys())
>>> subn(pattern, repl, 'aaa')  # expecting 'YX'
('XXX', 3)

The result that you get using this method will be consistent but 
arbitrary and unpredictable.




For those who care, here's my timing code:

from timeit import Timer

setup = """
mapping = {"foo" : "bar", "baz" : "quux", "quuux" : "foo"}
pattern = "(%s)" % "|".join(mapping.keys())
repl = lambda x : mapping.get(x.group(1), x.group(1))
repl = lambda x : mapping[x.group(1)]
s = "fooxxxbazyyyquuux"
from re import subn
"""

t1 = Timer("subn(pattern, repl, s)", setup)
t2 = Timer(
"s.replace('foo', 'bar').replace('baz', 'quux').replace('quuux', 'foo')",
"s = 'fooxxxbazyyyquuux'")


And the results on my PC:

>>> min(t1.repeat(number=10))
1.1273870468139648
>>> min(t2.repeat(number=10))
0.49491715431213379



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: What is a list compression in Python?

[Steven D'Aprano]

On Mon, 18 Jan 2010 08:07:41 -0800, Kit wrote:

> Hello Everyone, I am not sure if I have posted this question in a
> correct board. Can anyone please teach me:
> 
> What is a list compression in Python?

Google "python list comprehension".

If Google is broken for you, try Yahoo, or any other search engine.


> Would you mind give me some list compression examples?

Instead of this:

L = []
for x in range(10):
L.append(x**2)


you can write:

L = [x**2 for x in range(10)]


Instead of this example:


L = []
for x in range(10):
if x % 2 == 0:
L.append(x**2)



you can write:

L = [x**2 for x in range(10) if x % 2 == 0]


-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

py.test-1.2.0: junitxml, standalone test scripts, pluginization

[holger krekel]

Hi all, 

i just released some bits related to automated testing with Python: 

  py-1.2.0: py.test core which grew junitxml, standalone-script generation 
  pytest-xdist-1.0: separately installable dist-testing & looponfailing plugin
  pytest-figleaf-1.0: separately installable figleaf-coverage testing plugin

See below or at this URL for the announcement:

http://pylib.org/announce/release-1.2.0.html

If you didn't experience much speed-up or previously had problems with 
distributed testing i recommend you try to install "pytest-xdist" now and see 
if it works better. For me it speeds up some tests runs by 500% on a 4 CPU
machine due to its better internal model and several fixes.  (It's five
times because several tests depend on IO and don't block CPU meanwhile).

Another tip: if you use "pip" (best with a virtualenv) you can do e.g.:

pip install pytest-xdist 
pip uninstall pytest-xdist 

to conveniently activate/deactivate plugins for py.test. easy_install 
works ok as well but has no uninstall, yet remains the only option 
for installing with Python3 at the moment, though.  You need to use
the fine 'distribute' project's easy_install for the latter.

cheers & have fun,
holger


py.test/pylib 1.2.0: junitxml, standalone test scripts, pluginization


py.test is an advanced automated testing tool working with
Python2, Python3 and Jython versions on all major operating
systems.  It has a simple plugin architecture and can run many 
existing common Python test suites without modification.  It offers 
some unique features not found in other testing tools.  
See http://pytest.org for more info.

py.test 1.2.0 brings many bug fixes and interesting new abilities:

* --junitxml=path will create an XML file for use with CI processing 
* --genscript=path creates a standalone py.test-equivalent test-script 
* --ignore=path prevents collection of anything below that path
* --confcutdir=path only lookup conftest.py test configs below that path
* a 'pytest_report_header' hook to add info to the terminal report header 
* a 'pytestconfig' function argument gives direct access to option values
* 'pytest_generate_tests' can now be put into a class as well 
* on CPython py.test additionally installs as "py.test-VERSION", on
  Jython as py.test-jython and on PyPy as py.test-pypy-XYZ

Apart from many bug fixes 1.2.0 also has better pluginization: 
Distributed testing and looponfailing testing now live in the
separately installable 'pytest-xdist' plugin.  The same is true for
'pytest-figleaf' for doing coverage reporting.  Those two plugins
can serve well now as blue prints for doing your own.  

thanks to all who helped and gave feedback,
have fun,

holger krekel, January 2010

Changes between 1.2.0 and 1.1.1
=

- moved dist/looponfailing from py.test core into a new 
  separately released pytest-xdist plugin.

- new junitxml plugin: --junitxml=path will generate a junit style xml file
  which is processable e.g. by the Hudson CI system. 

- new option: --genscript=path will generate a standalone py.test script
  which will not need any libraries installed.  thanks to Ralf Schmitt. 

- new option: --ignore will prevent specified path from collection. 
  Can be specified multiple times. 

- new option: --confcutdir=dir will make py.test only consider conftest 
  files that are relative to the specified dir.  

- new funcarg: "pytestconfig" is the pytest config object for access
  to command line args and can now be easily used in a test. 

- install 'py.test' and `py.which` with a ``-$VERSION`` suffix to
  disambiguate between Python3, python2.X, Jython and PyPy installed versions. 

- new "pytestconfig" funcarg allows access to test config object

- new "pytest_report_header" hook can return additional lines 
  to be displayed at the header of a test run. 

- (experimental) allow "py.test path::name1::name2::..." for pointing
  to a test within a test collection directly.  This might eventually
  evolve as a full substitute to "-k" specifications. 

- streamlined plugin loading: order is now as documented in
  customize.html: setuptools, ENV, commandline, conftest. 
  also setuptools entry point names are turned to canonical namees ("pytest_*")

- automatically skip tests that need 'capfd' but have no os.dup 

- allow pytest_generate_tests to be defined in classes as well 

- deprecate usage of 'disabled' attribute in favour of pytestmark 
- deprecate definition of Directory, Module, Class and Function nodes
  in conftest.py files.  Use pytest collect hooks instead.

- collection/item node specific runtest/collect hooks are only called exactly
  on matching conftest.py files, i.e. ones which are exactly below
  the filesystem path of an item

- change: the first pytest_collect_directory hook to return something
  will now prevent further hooks to be called.

- change: figleaf plugin now requires 

Re: Arrrrgh! Another module broken

[Grant Edwards]

On 2010-01-18, Jive Dadson  wrote:
> I just found another module that broke when I went to 2.6.  Gnuplot. 
> Apparently one of its routines has a parameter named "with."  That used 
> to be okay, and now it's not.

I remember seeing depreicated warnings about that _years_ ago,
and I would have sworn it's been fixed for at least a couple
years.

-- 
Grant
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Is python not good enough?

[Steven D'Aprano]

On Mon, 18 Jan 2010 03:03:26 -0800, Phlip wrote:

> On Jan 12, 7:09 am, ikuta liu  wrote:
> 
>> Go language try to merge low level, hight level and browser language.
> 
> Go uses := for assignment.
> 
> This means, to appease the self-righteous indignation of the math
> professor who would claim = should mean "equality"...
> 
> ...you gotta type a shift and 2 characters for a very common operator.

I doubt it has anything to do with "the math professor". Any maths 
professor will tell you that, in mathematics, = is used for both 
assignment and equality, since in maths they are the same thing.

And besides, equality testing is no less common than assignment. To 
appease the "self-righteous indignation of the C coders", we have to type 
== instead of = for a very common operator. No matter what convention you 
use, you're going to upset some group of people.

Seriously, I programmed in Pascal for many years, and typing := for 
assignment is not a burden.



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Is python not good enough?

[Steven D'Aprano]

On Mon, 18 Jan 2010 07:37:36 -0800, Phlip wrote:

> They fixed the hideous redundancy of Java without the ill-defined scope
> issues of Python

Which ill-defined scope issues are you referring to?


-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Python decorator syntax limitations

[Jonathan S]

Hi all,
The following is what I want to do, but this results in a syntax
error:


@news_page('template.html').lookup(News, 'news_id', 'news')
def view(request, group, news):
pass


What does work is the equivalent old way of doing decorating:


def view(request, group, news):
pass
view = news_page('template.html').lookup(News, 'news_id', 'news')
(view)



Any suggestions? I have my reasons for doing this, (news_page is a
class, and __call__ is used to wrap the template.)
I'm sure this is a limitation in the syntax, but would parenthesis
somewhere help?

Thanks!
Jonathan
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: substitution

[Iain King]

On Jan 18, 4:26 pm, Steven D'Aprano  wrote:
> On Mon, 18 Jan 2010 06:23:44 -0800, Iain King wrote:
> > On Jan 18, 2:17 pm, Adi Eyal  wrote:
> [...]
> >> Using regular expressions the answer is short (and sweet)
>
> >> mapping = {
> >>         "foo" : "bar",
> >>         "baz" : "quux",
> >>         "quuux" : "foo"
>
> >> }
>
> >> pattern = "(%s)" % "|".join(mapping.keys())
> >> repl = lambda x : mapping.get(x.group(1), x.group(1))
> >> s = "fooxxxbazyyyquuux"
> >> re.subn(pattern, repl, s)
>
> > Winner! :)
>
> What are the rules for being declared "Winner"? For the simple case
> given, calling s.replace three times is much faster: more than twice as
> fast.
>
> But a bigger problem is that the above "winner" may not work correctly if
> there are conflicts between the target strings (e.g. 'a'->'X',
> 'aa'->'Y'). The problem is that the result you get depends on the order
> of the searches, BUT as given, that order is non-deterministic.
> dict.keys() returns in an arbitrary order, which means the caller can't
> specify the order except by accident. For example:
>
> >>> repl = lambda x : m[x.group(1)]
> >>> m = {'aa': 'Y', 'a': 'X'}
> >>> pattern = "(%s)" % "|".join(m.keys())
> >>> subn(pattern, repl, 'aaa')  # expecting 'YX'
>
> ('XXX', 3)
>
> The result that you get using this method will be consistent but
> arbitrary and unpredictable.
>
> For those who care, here's my timing code:
>
> from timeit import Timer
>
> setup = """
> mapping = {"foo" : "bar", "baz" : "quux", "quuux" : "foo"}
> pattern = "(%s)" % "|".join(mapping.keys())
> repl = lambda x : mapping.get(x.group(1), x.group(1))
> repl = lambda x : mapping[x.group(1)]
> s = "fooxxxbazyyyquuux"
> from re import subn
> """
>
> t1 = Timer("subn(pattern, repl, s)", setup)
> t2 = Timer(
> "s.replace('foo', 'bar').replace('baz', 'quux').replace('quuux', 'foo')",
> "s = 'fooxxxbazyyyquuux'")
>
> And the results on my PC:
>
> >>> min(t1.repeat(number=10))
> 1.1273870468139648
> >>> min(t2.repeat(number=10))
>
> 0.49491715431213379
>
> --
> Steven

Adi elicited that response from me because his solution was vastly
more succinct than everything else that had appeared up til that point
while still meeting the OP's requirements.  The OP never cared about
overlap between 2 'find' strings, just between the 'find' and
'replace' strings (though I did take it into account in my second post
for the sake of completeness).  His code could have been a little
cleaner, I'd have trimmed it to:

mapping = {"foo": "bar", "baz": "quux", "quuux": "foo"}
pattern = "(%s)" % "|".join(mapping)
repl = lambda x : mapping[x.group(1)]
s = "fooxxxbazyyyquuux"
re.subn(pattern, repl, s)

but apart from that was very pythonic: explicit, succinct, and all the
heavy work is done by the module (i.e. in compiled c code in the
majority case of CPython).  It can be 'upgraded' to cover the find-
find overlap if you really want (I *believe* regexps will match the
leftmost option in a group first):

subs = [("foo", "bar"), ("baz", "quux"), ("quuux", "foo")]
pattern = "(%s)" % "|".join((x[0] for x in subs))
mapping = dict(subs)
repl = lambda x : mapping[x.group(1)]
s = "fooxxxbazyyyquuux"
re.subn(pattern, repl, s)

Anyway, there's no prize for winning, but by all means: if you think
someone else's code and not a variation on this should win for most
pythonic, then make your nomination :)

Iain
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python decorator syntax limitations

[Steve Howell]

On Jan 18, 8:44 am, Jonathan S  wrote:
> Hi all,
> The following is what I want to do, but this results in a syntax
> error:
>
> @news_page('template.html').lookup(News, 'news_id', 'news')
> def view(request, group, news):
>     pass
>
> What does work is the equivalent old way of doing decorating:
>
> def view(request, group, news):
>     pass
> view = news_page('template.html').lookup(News, 'news_id', 'news')
> (view)
>
> Any suggestions? I have my reasons for doing this, (news_page is a
> class, and __call__ is used to wrap the template.)
> I'm sure this is a limitation in the syntax, but would parenthesis
> somewhere help?


You might want to consider doing something like this:

def lookup_view(page_class, template, class_, id_, label):
def decorator(func):
return page_class(template).lookup(class_, id_, label)
(func)
return decorator

@lookup_view(news_page, 'template.html', News, 'news_id', 'news')
def view(...):
pass

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: not and is not problem

[Jean-Michel Pichavant]

Steven D'Aprano wrote:

On Mon, 18 Jan 2010 16:43:25 +0100, superpollo wrote:

  

hi:

#!/usr/bin/env python
data = "seq=123"
name , value = data.split("=")
print name
print value
if not name == "seq":
 print "DOES NOT PRINT OF COURSE..."
if name is not "seq":
 print "WTF! WHY DOES IT PRINT?"


[snip]
Beware: Python caches strings that look like identifiers. This will 
include "seq". So, purely as an implementation-specific detail, Python 
will sometimes re-use the same string object, and sometimes not.
  


This is the most important point to understand. Some people use equality 
where in fact they should test for identity, because of this 'cache' 
effect Steven described that makes you feel that the 'is' operator is 
not reliable.


>>>'seq' is 'seq'
True
>>>name = 'seq'
>>>name is 'seq'
True 
>>>name, _ = 'seq _'.split()

>>>name is 'seq'
False

In your case, what you want is equality, because you don't care if name 
and 'seq' are the same object in memory, there's a good chance they are 
not in fact (see last example).



A simple example to illustrate the difference:

>>>class Foo:
>>>def __eq__(self, any):
>>> """Foo equals everything."
>>> return True

>>>f = Foo()
>>>f == None
True
>>>f is None
False

f equals None, but f is not None.

JM
--
http://mail.python.org/mailman/listinfo/python-list

Re: ctypes: nested structures and pointers

[Gabriele Modena]

On Mon, Jan 18, 2010 at 10:33 AM, Gabriele Modena
 wrote:
>  1. what is the correct (pythonic) way to capture the prototype
> definition of dev_callbacks and the relation between that structure
> and dev_info?
>
>  2. is it correct to wrap "connect", "transceive" and "disconnect" in
> separate structures and reference them within  pyDEV_CALLBACKS?

Solved.
The answer was in the doc:
http://docs.python.org/library/ctypes.html#structured-data-types
(_fields_ & _pack_).
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python decorator syntax limitations

[Lie Ryan]

On 01/19/10 03:44, Jonathan S wrote:
> Any suggestions? I have my reasons for doing this, (news_page is a
> class, and __call__ is used to wrap the template.)
> I'm sure this is a limitation in the syntax, but would parenthesis
> somewhere help?

The restriction[1] is put in there since Guido has a "gut feeling" that
allowing arbitrary expression in decorator syntax severely harms
readability.
http://mail.python.org/pipermail/python-dev/2004-August/046711.html

A workaround is to put an assignment above it:

nplookup = news_page('template.html').lookup
@nplookup(News, 'news_id', 'news')
def view(request, group, news):
pass

If you are sure you can put up a convincing argument for lifting this
restriction, and you are willing to put some time arguing, you are
welcome to start a thread in the python-dev mailing list. Be sure to
read about previous discussions, as repeating arguments wouldn't change
anything.

[1] there is a subtle difference between "restriction" and "limitation"
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Generic Python Benchmark suite?

[Antoine Pitrou]

Le Mon, 18 Jan 2010 11:30:16 +0100, Stefan Behnel a écrit :
> Anand Vaidya, 18.01.2010 10:58:
>> Is there a generic python benchmark suite in active development? [...]
>> PS: I think a benchmark should cover file / network, database  I/O,
>> data structures (dict, list etc), object creation/manipulation,
>> numbers, measure looping inefficiencies, effects of caching (memcache
>> etc) at the minimum
> 
> That doesn't sound generic at all. Maybe you should prefer an
> application based benchmark instead.

Actually, it sounds much more generic that most of the benchmarks we 
usually rely on :-)

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python decorator syntax limitations

[Peter Otten]

Jonathan S wrote:

> Hi all,
> The following is what I want to do, but this results in a syntax
> error:
> 
> 
> @news_page('template.html').lookup(News, 'news_id', 'news')
> def view(request, group, news):
> pass
> 
> 
> What does work is the equivalent old way of doing decorating:
> 
> 
> def view(request, group, news):
> pass
> view = news_page('template.html').lookup(News, 'news_id', 'news')
> (view)
> 
> 
> 
> Any suggestions? I have my reasons for doing this, (news_page is a
> class, and __call__ is used to wrap the template.)
> I'm sure this is a limitation in the syntax, but would parenthesis
> somewhere help?

@apply(lambda:news_page('template.html').lookup(News, 'news_id', 'news'))
def view(request, group, news):
pass

But I'd go with a dedicated helper function.

Peter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Generic Python Benchmark suite?

[Antoine Pitrou]

Le Mon, 18 Jan 2010 01:58:42 -0800, Anand Vaidya a écrit :
> Is there a generic python benchmark suite in active development? I am
> looking forward to comparing some code on various python implementations
> (primarily CPython 2.x, CPython 3.x, UnladenSwallow, Psyco).
> 
> I am happy with something that gives me a relative number eg: ULS is 30%
> faster than CPy2.x etc
> 
> I found pybench which is probably not maintained actively.

pybench is not abandoned. However, it only gets an addition now and then. 
More importantly, it's a set of low-level microbenchmarks designed to 
stress the execution cost of certain primitives. It won't give you the 
answer to any high-level questions.

In the SVN sandbox (*) you'll find a couple of other benchmarks:
- stringbench, stressing string operations
- iobench, stressing common file I/O operations
- ccbench, trying to measure interpreter efficiency in the face of multi-
threaded workloads

(*) http://svn.python.org/view/sandbox/trunk/

Regards

Antoine.


-- 
http://mail.python.org/mailman/listinfo/python-list

using super

[Jean-Michel Pichavant]

class SubClass(Base):
colour = "Red"
def parrot(self):
"""docstring for Subclass"""
return super(Subclass, self).parrot()

I'm not a big fan of super, but I'm still wondering if

>>> return super(self.__class__, self).parrot()

would have made it.

What if Subclass has more than one base class ?


JM

--
http://mail.python.org/mailman/listinfo/python-list

Re: python gui ide under linux..like visual studio ;) ?

[Antoine Pitrou]

Le Mon, 18 Jan 2010 15:32:36 +0100, ted a écrit :
> 
> And, a good library for access to database (mysql, sql server, oracle) ?

If you want something high-level: http://www.sqlalchemy.org/
You won't regret it :)

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: using super

[Duncan Booth]

Jean-Michel Pichavant  wrote:

> 
 class SubClass(Base):
 colour = "Red"
 def parrot(self):
 """docstring for Subclass"""
 return super(Subclass, self).parrot()
> I'm not a big fan of super, but I'm still wondering if
> 
> >>> return super(self.__class__, self).parrot()
> 
> would have made it.

No, it wouldn't.

> What if Subclass has more than one base class ?

super() will work in that case provided you specify the current class. It 
never works correctly if you specify the class of self.

Consider another level of subclasses and it may be clearer:

>>> class Base(object):
def parrot(self):
print "Base.parrot"


>>> class SubClass(Base):
def parrot(self):
print "SubClass.parrot"
return super(SubClass, self).parrot()


>>> class SubSubClass(SubClass):
def parrot(self):
print "SubSubClass.parrot"
return super(SubSubClass, self).parrot()


>>> SubSubClass().parrot()
SubSubClass.parrot
SubClass.parrot
Base.parrot

>>> class SubClass(Base):
def parrot(self):
print "SubClass.parrot"
return super(self.__class__, self).parrot()


>>> class SubSubClass(SubClass):
def parrot(self):
print "SubSubClass.parrot"
return super(self.__class__, self).parrot()

>>> SubSubClass().parrot()
SubClass.parrot
SubClass.parrot
SubClass.parrot
SubClass.parrot
SubClass.parrot
SubClass.parrot
SubClass.parrot
SubClass.parrot
SubClass.parrot
... (you're in an infinite loop now) ...
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Is python not good enough?

[MRAB]

Phlip wrote:

On Jan 12, 7:09 am, ikuta liu  wrote:


Go language try to merge low level, hight level and browser language.


Go uses := for assignment.

This means, to appease the self-righteous indignation of the math
professor who would claim = should mean "equality"...

...you gotta type a shift and 2 characters for a very common operator.

Pass!


If I were going to list what I didn't like about Go, that wouldn't be
one of them!
--
http://mail.python.org/mailman/listinfo/python-list

Re: Python decorator syntax limitations

[Aahz]

In article <4b54998...@dnews.tpgi.com.au>,
Lie Ryan   wrote:
>
>If you are sure you can put up a convincing argument for lifting this
>restriction, and you are willing to put some time arguing, you are
>welcome to start a thread in the python-dev mailing list. Be sure to
>read about previous discussions, as repeating arguments wouldn't change
>anything.

While I haven't been reading python-dev in recent months, I'm pretty sure
the moratorium makes the only appropriate place for this python-ideas,
and even there people are not likely to invest much time:

http://www.python.org/dev/peps/pep-3003/
-- 
Aahz (a...@pythoncraft.com)   <*> http://www.pythoncraft.com/

"If you think it's expensive to hire a professional to do the job, wait
until you hire an amateur."  --Red Adair
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: substitution

[Adi Eyal]

> From: Steven D'Aprano 
> To: python-l...@python.org
> Date: 18 Jan 2010 16:26:48 GMT
> Subject: Re: substitution
> On Mon, 18 Jan 2010 06:23:44 -0800, Iain King wrote:
>
>> On Jan 18, 2:17 pm, Adi Eyal  wrote:
> [...]
>>> Using regular expressions the answer is short (and sweet)
>>>
>>> mapping = {
>>>         "foo" : "bar",
>>>         "baz" : "quux",
>>>         "quuux" : "foo"
>>>
>>> }
>>>
>>> pattern = "(%s)" % "|".join(mapping.keys())
>>> repl = lambda x : mapping.get(x.group(1), x.group(1))
>>> s = "fooxxxbazyyyquuux"
>>> re.subn(pattern, repl, s)
>>
>> Winner! :)
>
> What are the rules for being declared "Winner"? For the simple case
> given, calling s.replace three times is much faster: more than twice as
> fast.

:) - The solution above is short and easy to read and digest. It is
also easy to maintain - you simply need to add additional entries to
the mapping dict if you need to add additional strings.

Calling replace 3 times in a row doesn't work correctly
e.g.

foo => bar
bar => baz

foo.replace("foo", "bar").replace("bar", "baz")


>
> But a bigger problem is that the above "winner" may not work correctly if
> there are conflicts between the target strings (e.g. 'a'->'X',
> 'aa'->'Y'). The problem is that the result you get depends on the order
> of the searches, BUT as given, that order is non-deterministic.
> dict.keys() returns in an arbitrary order, which means the caller can't
> specify the order except by accident. For example:
>
 repl = lambda x : m[x.group(1)]
 m = {'aa': 'Y', 'a': 'X'}
 pattern = "(%s)" % "|".join(m.keys())
 subn(pattern, repl, 'aaa')  # expecting 'YX'
> ('XXX', 3)
>
> The result that you get using this method will be consistent but
> arbitrary and unpredictable.
>

That's a simple fix

import re

mapping = {
"foo" : "bar",
"baz" : "quux",
"quuux" : "foo"
}

keys = [(len(key), key) for key in mapping.keys()]
keys.sort(reverse=True)
keys = [key for (_, key) in keys]

pattern = "(%s)" % "|".join(keys)
repl = lambda x : mapping[x.group(1)]
s = "fooxxxbazyyyquuux"

re.subn(pattern, repl, s)


With regards to timing - I suspect that as the input string grows, the
ratio of the regexp solution to the string solution will grow. If the
map grows, the regexp solution seems faster (see below). Python's
regular expression state machine is very inefficient with lots of
disjunctions since it involves a lot of backtracking - this can be
hand optimised for a much greater speed improvement.

N = 1000
alphabet = "abcdefghijklmnopqrstuvwxyz"
make_token = lambda : "".join(sample(alphabet, 5))
mapping = dict([(make_token(), make_token()) for i in range(N)])

keys = [(len(key), key) for key in mapping.keys()]
keys.sort(reverse=True)
keys = [key for (_, key) in keys]

pattern = "(%s)" % "|".join(keys)
replace_strs = ["replace('%s', '%s')" % (key, mapping[key]) for key in keys]
command = "s." + ".".join(replace_strs)

setup = """
import re

regexp = re.compile("%s")
repl = lambda x : mapping[x.group(1)]
s = "fooxxxbazyyyquuux"
""" % pattern


t1 = Timer("regexp.subn(repl, s)", setup)
t2 = Timer(command, "s = 'fooxxxbazyyyquuux'")

res1 = sum(t1.repeat(number=1000))
res2 = sum(t2.repeat(number=1000))
print res1 / res2

on my machine this comes to
>>> 0.316323109737
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python decorator syntax limitations

[Arnaud Delobelle]

a...@pythoncraft.com (Aahz) writes:

> In article <4b54998...@dnews.tpgi.com.au>,
> Lie Ryan   wrote:
>>
>>If you are sure you can put up a convincing argument for lifting this
>>restriction, and you are willing to put some time arguing, you are
>>welcome to start a thread in the python-dev mailing list. Be sure to
>>read about previous discussions, as repeating arguments wouldn't change
>>anything.
>
> While I haven't been reading python-dev in recent months, I'm pretty sure
> the moratorium makes the only appropriate place for this python-ideas,
> and even there people are not likely to invest much time:
>
> http://www.python.org/dev/peps/pep-3003/

In actual fact there is a recent discussion of this on python-ideas:

http://groups.google.co.uk/group/python-ideas/browse_thread/thread/1eebf486969c39a1/

-- 
Arnaud
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: lightweight encryption of text file

[Aahz]

In article ,
Daniel Fetchinson   wrote:
>
>Well, that's sort of true about learning a complex API :) But it's
>also true that I'm not storing anything really valuable in the file
>but still wouldn't want to leave it lying around in plain text. In
>case I lose the laptop with the file I seriously doubt anybody who
>finds it will go through each and every file and try to find what's in
>it, even though they look like data files and there is no hint what so
>ever that any one of them contains encrypted info. If they see a text
>file, well, that can give them ideas, so let's encrypt a little bit.

One reason I like OSX is that it's dirt-simple to encrypt my home dir.
-- 
Aahz (a...@pythoncraft.com)   <*> http://www.pythoncraft.com/

"If you think it's expensive to hire a professional to do the job, wait
until you hire an amateur."  --Red Adair
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: using super

[Jean-Michel Pichavant]

Duncan Booth wrote:

Jean-Michel Pichavant  wrote:

  

class SubClass(Base):
colour = "Red"
def parrot(self):
"""docstring for Subclass"""
return super(Subclass, self).parrot()
  

I'm not a big fan of super, but I'm still wondering if



return super(self.__class__, self).parrot()
  

would have made it.



No, it wouldn't.

  

What if Subclass has more than one base class ?



super() will work in that case provided you specify the current class. It 
never works correctly if you specify the class of self.


Consider another level of subclasses and it may be clearer:

  

class Base(object):


def parrot(self):
print "Base.parrot"


  

class SubClass(Base):


def parrot(self):
print "SubClass.parrot"
return super(SubClass, self).parrot()


  

class SubSubClass(SubClass):


def parrot(self):
print "SubSubClass.parrot"
return super(SubSubClass, self).parrot()


  

SubSubClass().parrot()


SubSubClass.parrot
SubClass.parrot
Base.parrot

  

class SubClass(Base):


def parrot(self):
print "SubClass.parrot"
return super(self.__class__, self).parrot()


  

class SubSubClass(SubClass):


def parrot(self):
print "SubSubClass.parrot"
return super(self.__class__, self).parrot()

  

SubSubClass().parrot()


SubClass.parrot
SubClass.parrot
SubClass.parrot
SubClass.parrot
SubClass.parrot
SubClass.parrot
SubClass.parrot
SubClass.parrot
SubClass.parrot
... (you're in an infinite loop now) ...
  

I see.

Then is there a reason why

return super(Subclass, self).parrot()
would be prefered over the "classic" 
return Base.parrot(self) 

?

Or is it just a matter of preference ?

JM
--
http://mail.python.org/mailman/listinfo/python-list

Unable to install numpy

[vsoler]

Hi all,

I just download Numpy, and tried to install it using "numpy-1.4.0-
win32-superpack-python2.6.exe"

I get an error:  "Python version 2.6 required, which was not found in
the Registry"

However, I am using Python 2.6 every day. I'm running Windows 7.

What can I do?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Unable to install numpy

[Robert Kern]

On 2010-01-18 14:02 PM, vsoler wrote:

Hi all,

I just download Numpy, and tried to install it using "numpy-1.4.0-
win32-superpack-python2.6.exe"

I get an error:  "Python version 2.6 required, which was not found in
the Registry"

However, I am using Python 2.6 every day. I'm running Windows 7.

What can I do?


Please ask numpy installation questions on the numpy mailing list.

  http://www.scipy.org/Mailing_Lists

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco

--
http://mail.python.org/mailman/listinfo/python-list

Re: using super

[Arnaud Delobelle]

Jean-Michel Pichavant  writes:

[...]
> Then is there a reason why
 return super(Subclass, self).parrot()
> would be prefered over the "classic" 
 return Base.parrot(self) 
> ?
>
> Or is it just a matter of preference ?

Using super() calls the next method in the class's Method Resolution
Order (mro - to see it on class A, use A.mro()).  This guarantees that
ancestor methods won't be called twice when the inheritance graph is not
a tree.  Here is a (minimal?) example.

>>> class A(object):
... def foo(self): print 'A'
... 
>>> class B(A):
... def foo(self):
... super(B, self).foo()
... print 'B'
... 
>>> class C(A):
... def foo(self):
... super(C, self).foo()
... print 'C'
... 
>>> class D(B, C):
... def foo(self):
... super(D, self).foo()
... print 'D'
... 
>>> d = D()
>>> d.foo()
A
C
B
D
>>> D.mro()
[, , , , ]

The reason why super() may be necessary is that if an object is an
instance of say class C, its method resolution order above class C is
not known at compile time.

HTH

-- 
Arnaud
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: force URLencoding script

[João]

On Jan 15, 4:46 pm, r0g  wrote:
> João wrote:
> > On Jan 15, 2:38 pm, r0g  wrote:
> >> João wrote:
> >>> On Jan 14, 5:58 pm, r0g  wrote:
>  João wrote:
> > On Jan 12, 10:07 pm, r0g  wrote:
> >> João wrote:
> > for the following data,
> > authentication = "UID=somestring&"
> > message = 'PROBLEM severity High: OperatorX Plat1(locationY) global
> > Succ. : 94.47%'
> > dest_number = 'XXX'
> > url_values = urlencode({'M':message})
> > enc_data = authentication + url_values + dest_number
> > I'm getting null for
> > full_url = Request(url, enc_data, headers)
> > and thus,
> > response = urlopen(full_url).read()
> > returns,
> > TypeError: 
> > )
>  Are you sure it's returning a null and not just some other unexpected
>  type?
>  I think your problem may be that you are passing a urllib2 class to
>  urllib(1)'s urlopen. Try using urllib2's urlopen instead e.g.
>  import urllib2
>  request_object = urllib2.Request('http://www.example.com')
>  response = urllib2.urlopen(request_object)
>  the_page = response.read()
>  Roger.
> >>> Thanks Roger.
> >>> I think it's a null because i did a print(full_url) right after the
> >>> Request
> >>> I tried
> >>> request_object = urllib2.Request('http://www.example.com')
> >>> print(request_object)
> >>> but when printing I get: 
> >> Hi João,
>
> >> That's exactly what you want, an object that is an instance of the
> >> Request class. That object doesn't do anything by itself, you still need
> >> to a) Connect to the server and request that URL and b) Read the data
> >> from the server.
>
> >> a) To connect to the web server and initialize the request you need to
> >> call urllib2.urlopen() with the Request object you just created and
> >> assign the result to a name e.g.
>
>  response = urllib2.urlopen(request_object)
> >> That will give you an object (response) that you can call the .read()
> >> method of to get the web page data.
>
>  the_page = response.read()
> >> If that doesn't make sense or seem to work for you then please try
> >> reading the following website from top to bottom before taking any
> >> further steps...
>
> >>http://www.voidspace.org.uk/python/articles/urllib2.shtml
>
> >>> I've read about Python 2.4 not playing well with proxies even with no
> >>> proxy activated.
> >>> Any sugestion?
> >> I doubt any language can play well with proxies if there are none so I
> >> doubt it's a factor ;)
>
> >> Good luck,
>
> >> Roger.
>
> > lol.
> > I've expressed myself poorly,
> > I meant I read about some issues when getting the Request + urlopen
> > working when there's a proxy involved (like in my case)
> > even when activating a no_proxy configuration, something like,
>
> > proxy_support = urllib.ProxyHandler({})
> > opener = urllib.build_opener(proxy_support)
> > urllib.install_opener(opener)
>
> > But I don't know how to use it :(
>
> That is how you use it IIRC, this installs the proxy handler into urllib
> and subsequent objects you subclass from urllib will use the custom handler.
>
> From what I can tell, you should be using urllib2 though, not urllib.
>
> Lets take a step back. You had the following line...
>
> request_object = urllib2.Request('http://www.example.com')
>
> ...You printed it and it showed that you had created a Request object
> right. Now what happens when you type...
>
> response = urllib2.urlopen(request_object)
> print response
>
> ?
>
> Roger.

Thanks for the patience Roger.
Your explanation opened my eyes.

I finally got it to work, and it turned out I didn't have to include
any custom proxy handler to avoid our proxy.
It ended on such a small and simple block of code after getting the
pieces together..


#!/usr/bin/env python

import sys
from urllib2 import Request, urlopen
from urllib import urlencode


authentication = "UID=124675&PW=gdberishyb8LcIANtvT89QVQ==&"
url = 'http://10.112.28.221:38080/GwHTTPin/sendtext'
encoded_data = urlencode({'M':sys.argv[2],
'N':sys.argv[1]})

# Was having problem with this one, shouldn't have tried to pass the
authentication as an urlencode parameter because it was breaking the
password string!
sent_data = authentication + encoded_data

full_url = Request(url,sent_data)
response = urlopen(full_url).read()
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: force URLencoding script

[João]

On Jan 15, 4:46 pm, r0g  wrote:
> João wrote:
> > On Jan 15, 2:38 pm, r0g  wrote:
> >> João wrote:
> >>> On Jan 14, 5:58 pm, r0g  wrote:
>  João wrote:
> > On Jan 12, 10:07 pm, r0g  wrote:
> >> João wrote:
> > for the following data,
> > authentication = "UID=somestring&"
> > message = 'PROBLEM severity High: OperatorX Plat1(locationY) global
> > Succ. : 94.47%'
> > dest_number = 'XXX'
> > url_values = urlencode({'M':message})
> > enc_data = authentication + url_values + dest_number
> > I'm getting null for
> > full_url = Request(url, enc_data, headers)
> > and thus,
> > response = urlopen(full_url).read()
> > returns,
> > TypeError: 
> > )
>  Are you sure it's returning a null and not just some other unexpected
>  type?
>  I think your problem may be that you are passing a urllib2 class to
>  urllib(1)'s urlopen. Try using urllib2's urlopen instead e.g.
>  import urllib2
>  request_object = urllib2.Request('http://www.example.com')
>  response = urllib2.urlopen(request_object)
>  the_page = response.read()
>  Roger.
> >>> Thanks Roger.
> >>> I think it's a null because i did a print(full_url) right after the
> >>> Request
> >>> I tried
> >>> request_object = urllib2.Request('http://www.example.com')
> >>> print(request_object)
> >>> but when printing I get: 
> >> Hi João,
>
> >> That's exactly what you want, an object that is an instance of the
> >> Request class. That object doesn't do anything by itself, you still need
> >> to a) Connect to the server and request that URL and b) Read the data
> >> from the server.
>
> >> a) To connect to the web server and initialize the request you need to
> >> call urllib2.urlopen() with the Request object you just created and
> >> assign the result to a name e.g.
>
>  response = urllib2.urlopen(request_object)
> >> That will give you an object (response) that you can call the .read()
> >> method of to get the web page data.
>
>  the_page = response.read()
> >> If that doesn't make sense or seem to work for you then please try
> >> reading the following website from top to bottom before taking any
> >> further steps...
>
> >>http://www.voidspace.org.uk/python/articles/urllib2.shtml
>
> >>> I've read about Python 2.4 not playing well with proxies even with no
> >>> proxy activated.
> >>> Any sugestion?
> >> I doubt any language can play well with proxies if there are none so I
> >> doubt it's a factor ;)
>
> >> Good luck,
>
> >> Roger.
>
> > lol.
> > I've expressed myself poorly,
> > I meant I read about some issues when getting the Request + urlopen
> > working when there's a proxy involved (like in my case)
> > even when activating a no_proxy configuration, something like,
>
> > proxy_support = urllib.ProxyHandler({})
> > opener = urllib.build_opener(proxy_support)
> > urllib.install_opener(opener)
>
> > But I don't know how to use it :(
>
> That is how you use it IIRC, this installs the proxy handler into urllib
> and subsequent objects you subclass from urllib will use the custom handler.
>
> From what I can tell, you should be using urllib2 though, not urllib.
>
> Lets take a step back. You had the following line...
>
> request_object = urllib2.Request('http://www.example.com')
>
> ...You printed it and it showed that you had created a Request object
> right. Now what happens when you type...
>
> response = urllib2.urlopen(request_object)
> print response
>
> ?
>
> Roger.

Thanks for the patience Roger.
Your explanation opened my eyes.

I finally got it to work, and it turned out I didn't have to include
any custom proxy handler to avoid our proxy.
It ended on such a small and simple block of code after getting the
pieces together..


#!/usr/bin/env python

import sys
from urllib2 import Request, urlopen
from urllib import urlencode


authentication = "UID=124675&PW=gdberishyb8LcIANtvT89QVQ==&"
url = 'http://10.112.28.221:38080/GwHTTPin/sendtext'
encoded_data = urlencode({'M':sys.argv[2],
'N':sys.argv[1]})

# Was having problem with this one, shouldn't have tried to pass the
authentication as an urlencode parameter because it was breaking the
password string!
sent_data = authentication + encoded_data

full_url = Request(url,sent_data)
response = urlopen(full_url).read()
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python decorator syntax limitations

[Jonathan S]

Thanks a lot, all of you! This was really helpful. (or at least give
me the inspiration I needed to finish it.)

I'm sure this is a use case where most other options are less readable
than the chain of methods in the decorator.
In this use case, I had a lot of Django views to which access
permissions had to be attached. Chaining the permissions was in my
opinion the easiest way to go but appeared not to work because Guido
had a gut feeling about that.
Using multiple decorators in the usual way (like every following wraps
the previous) does not work, because these decorators need to access
the same authentication class. (little hard to explain, but take from
me that I had a case where it didn't work.)
Placing all possible options in the constructor method of the
decorator like Steve Howell proposed would work but is ugly.

This is what I made. I'll upgrade my news_page decorator to work in
the following way. (Applying the news_page decorator to 'view', will
turn it into a class, but make  it callable buy wrapping the actual
view in __call__. So we can still apply the other methods to the view.

@news_page('template.html')
def view(request, group, news):
...
pass
view.lookup(News, 'news_id', 'news')
view.require_administrator()


That is still rather ugly, because the 'lookup' option is appended
behind the view. I kept playing with the code and came to the
following:


@require_administrator
@do_lookup(News, 'news_id', 'news)
@news_page('template.html')
def view(request, group, news):
...
pass


Where 'do_lookup' and 'require_administrator' passes the method, but
sets some parameters. So, 'do_lookup' can access class members of
news_page. It would look like:

def do_lookup(*lookup_options):
def set_options(news_page_view):
news_page_view.lookup(*lookup_options)
return news_page_view
return set_options

def require_administrator(news_page_view):
news_page_view.require_administrator()
return news_page_view


Maybe, I'll join the discussion later on, when I have time to read all
the conversations and write down good arguments.
But, honestly, I'm satisfied with the last result.
Have a nice day!
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: python gui ide under linux..like visual studio ;) ?

[Mike Driscoll]

On Jan 18, 8:32 am, ted  wrote:
> Hi at all...
> Can someone please give me some advice, about a good IDE with control
> GUI under Linux ?
>
> Actually i know QT Creator by Nokia which i can use with Python (but i
> don't know how).
>
> And, a good library for access to database (mysql, sql server, oracle) ?
>
> Thank you very much !
>
> bye

Check out Dabo for the database stuff: http://dabodev.com/

There is nothing like Visual Studio. The closest I've found are things
like the QT Creator and wxGlade / Boa Constructor / wxFormBuilder. I
know I've heard of another one that was commercial, but I can't recall
the name at the moment.

---
Mike Driscoll

Blog:   http://blog.pythonlibrary.org

PyCon 2010 Atlanta Feb 19-21  http://us.pycon.org/
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: I really need webbrowser.open('file://') to open a web browser

[Timur Tabi]

On Sat, Jan 16, 2010 at 3:43 PM, Paul Boddie  wrote:

> Generally, the desktop-specific tools should know that a browser is
> the appropriate application for an HTML file, and testing with both
> xdg-open, gnome-open and "kfmclient openURL" seems to open browsers on
> HTML files (using file:///...) for me (using KDE, Kubuntu 8.04). Of
> course, this depends on the settings in use on your desktop, but it
> should be noted that using "kfmclient exec" could have the effect you
> describe.

I'm using Gnome, and I have HTML files associated with Firefox.
However, my default web browser is Seamonkey, and when I do
webbrowser.open('http://...'), it opens that URL in Seamonkey, not
Firefox.  So if there is some Gnome association between an .html file
and a text editor, I don't know where it is defined.

> Not that I'm aware of. Sadly, standardisation of applications and
> services - having a command which can open a particular class of
> application (such as "e-mail reader", "Web browser") - seems to be
> absent from the free desktop arena, although I do recall there being a
> preferred applications dialogue in KDE, at least.

I would be sympathetic to this problem if the API were called
desktop.open(...).  But it's called webbrowser.open(), so it has to be
certain that a web browser is being at all times.  IMHO, any other
behavior is a bug.

-- 
Timur Tabi
Linux kernel developer at Freescale
-- 
http://mail.python.org/mailman/listinfo/python-list

form mailer for info + files

[Gregory]

There is code all over on how to create a form mailer.
I need an example that will show how to upload two files and send them
along with form field info.
Is it required to overwrite the python cgi.fieldstorage to do this?
Working with Python 2.4.3.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: searching and storing large quantities of xml!

[dads]

Thanks all, took your advice and have been playing all weekend which
has been great fun. ElementTree is awesome. I created a script that
organises the xml as they're in year blocks and I didn't realise the
required xml is mixed up with other xml. Plus the volumes are much
greater than I realised, I checked as back at work and it was
something like 600,000 files in a year, just over a gig for each
year.

I'm going to add zipping up of the files and getting the required info
and putting it in a db this week hopefully. It's been completely
overhauled, originally I used modified date now it gets the date from
the parsed xml, safer that way. The code is below but word of caution,
it's hobbyist code so it'll probably make your eyes bleed =), thanks
again:

There was one thing that I forgot about - when ElementTree fails to
parse due to an element not being closed why doesn't it close the file
like object. As later on I would raise 'WindowsError: [Error
32] ...file being used by other process' when using shutil.move(). I
got round this by using a 'try except' block.

from __future__ import print_function
import xml.etree.cElementTree as ET
import calendar
import zipfile
import os.path
import shutil
import zlib
import os


class Xmlorg(object):

def __init__(self):

self.cwd = os.getcwd()
self.year = os.path.basename(self.cwd)

def _mkMonthAndDaysDirs(self):

''' creates dirs for every month and day of a of specidifed
year.
Works for leap years as well.

(specified)year/(year)month/day


...2010/201001/01
...2010/201001/02
...2010/201001/03 '''


def addZero(n):

if len(str(n)) < 2:
return '0' + str(n)
else:
return str(n)

dim = [ calendar.monthrange(year,month)[1] for year in \
[int(self.year)] for month in range(1,13) ]

count = 1
for n in dim:
month = addZero(count)
count += 1
ym = os.path.join(self.cwd, self.year + month)
os.mkdir(ym)
for x in range(1,n+1):
x = addZero(x)
os.mkdir(os.path.join(ym, x))


def ParseAndOrg(self):

'''requires dir and zip struct:

.../(year)/(year).zip - example .../2008/2008.zip '''


def movef(fp1,fp2):

'''moves files with exception handling'''

try:
shutil.move(fp1,fp2)
except IOError, e:
print(e)
except WindowsError, e:
print(e)

self._mkMonthAndDaysDirs()
os.mkdir(os.path.join(self.cwd, 'otherFileType'))

# dir struct .../(year)/(year).zip - ex. .../2008/2008.zip
zf = zipfile.ZipFile(os.path.join(self.cwd, self.year +
'.zip'))
zf.extractall()
ld = os.listdir(self.cwd)
for i in ld:
if os.path.isfile(i) and i.endswith('.xml'):
try:
tree = ET.parse(i)
except:
print('%s np' % i) #not parsed
root = tree.getroot()
if root.findtext('Summary/FileType') == 'Order':
date = root.findtext('OrderHeader/OrderDate')[:10]
#dd/mm/
dc = date.split('/')
fp1 = os.path.join(self.cwd, i)
fp2 = os.path.join(self.cwd, dc[2] + dc[1], dc[0])
movef(fp1,fp2)
else:
fp1 = os.path.join(self.cwd, i)
fp2 = os.path.join(self.cwd, 'otherFileType')
movef(fp1,fp2)


if __name__ == '__main__':
os.chdir('c:/sv_zip_test/2010/') #remove
xo = Xmlorg()
xo.ParseAndOrg()
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Generic Python Benchmark suite?

[Terry Reedy]

On 1/18/2010 4:58 AM, Anand Vaidya wrote:

Is there a generic python benchmark suite in active development? I am
looking forward to comparing some code on various python
implementations (primarily CPython 2.x, CPython 3.x, UnladenSwallow,
Psyco).


You might find this interesting if you have not seen it before:
http://code.google.com/p/unladen-swallow/wiki/Benchmarks

--
http://mail.python.org/mailman/listinfo/python-list

Re: Parse a log file

[Tim Chase]

kak...@gmail.com wrote:

I want to parse a log file with the following format for
example:
  TIMESTAMPEOperation FileName
Bytes
12/Jan/2010:16:04:59 +0200   EXISTS   sample3.3gp   37151
12/Jan/2010:16:04:59 +0200  EXISTSsample3.3gp   37151
12/Jan/2010:16:04:59 +0200  EXISTSsample3.3gp   37151
12/Jan/2010:16:04:59 +0200  EXISTSsample3.3gp   37151
12/Jan/2010:16:04:59 +0200  EXISTSsample3.3gp   37151
12/Jan/2010:16:05:05 +0200  DELETE  sample3.3gp   37151

How can i count the operations for a month(e.g total of 40 Operations,
30 exists, 10 delete?)


It can be done pretty easily with a regexp to parse the relevant 
bits:


  import re
  r = re.compile(r'\d+/([^/]+)/(\d+)\S+\s+\S+\s+(\w+)')
  stats = {}
  for line in file('log.txt'):
m = r.match(line)
if m:
  stats[m.groups()] = stats.get(m.groups(), 0) + 1
  print stats

This prints out

  {('Jan', '2010', 'EXISTS'): 5, ('Jan', '2010', 'DELETE'): 1}


With the resulting data structure, you can manipulate it to do 
coarser-grained aggregates such as the total operations, or remap 
month-name abbreviations into integers so they could be sorted 
for output.


-tkc


--
http://mail.python.org/mailman/listinfo/python-list

inner methods and recursion

[Steve Howell]

Hi, I have a style/design question relating to recursion and inner
methods.

I wrote some code recently that implements a recursive algorithm that
takes several parameters from the original caller.  Once the algorithm
starts running and recursing, those parameters remain the same, so I
define an inner method called "recurse" that essentially curries those
parameters.  In particular, the recurse method can get passed to a
callback method that the caller supplies, so that the caller can wrap
the recursive step with their own logic.

The only thing I don't like about the technique that I'm using is that
it seems needlessly repetitive to define mostly the same parameter
list in two different places.  The repetition seems mostly harmless,
but it seems like some kind of a smell that I'm overlooking a simpler
way to write the code.  I just can't put my finger on what it is.

def indent_lines(lines,
branch_method,
leaf_method,
pass_syntax,
flush_left_syntax,
flush_left_empty_line,
indentation_method,
get_block,
):
def recurse(lines):
return indent_lines(
lines,
branch_method,
leaf_method,
pass_syntax,
flush_left_syntax,
flush_left_empty_line,
indentation_method,
get_block,
)
output = []
while lines:
if lines[0].strip() == '':
lines.pop(0)
output.append('')
else:
prefix, i = get_block(lines, indentation_method)
if i == 1:
line = lines.pop(0)[len(prefix):]
if line == pass_syntax:
pass
elif line.startswith(flush_left_syntax):
output.append(line[len(flush_left_syntax):])
elif line.startswith(flush_left_empty_line):
output.append('')
else:
output.append(prefix + leaf_method(line))
else:
block = lines[:i]
lines = lines[i:]
output += branch_method(prefix, block, recurse)
return output

Does anybody have any inspiration here?  I vaguely recall a discussion
some time back about having some kind of @recursive decorator, but I
can't seem to find the thread.



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: r"string" vs R"string

[Terry Reedy]

On 1/18/2010 8:01 AM, Colin W. wrote:

On 17-Jan-10 18:27 PM, Steven D'Aprano wrote:

On Sun, 17 Jan 2010 11:13:48 -0500, Roy Smith wrote:


In article,
"Colin W." wrote:


On 17-Jan-10 02:16 AM, Terry Reedy wrote:

On 1/17/2010 1:55 AM, Brendan Miller wrote:

Is there any difference whatsoever between a raw string beginning
with the captical R or one with the lower case r e.g. r"string" vs
R"string"?


No. Nor is there and difference between the strings created with raw
literals and cooked literals.



"cooked" literal ??


I've never seen it referred to this way in the Python literature, but
"cooked" is a well-known term meaning, "not raw". The usage goes back
decades.


I think the use of "cooked" meaning "not raw" goes back a little bit more
than decades. The verb "to cook" dates from the late 14th century, and
the adjective form "cooked" would follow soon after. The use of "cooked"
to mean manipulated (as in "cooking the books") comes from the 1630s.
Extending it to strings (as in raw versus cooked strings) is an obvious
extension.


Yes, I should have cottoned on, but perhaps "... between the strings,
raw and other." would have conveyed the idea.


I intentionally did not say that because Python only has one type of 
string object (in 3.x), not raw and normal. Python *code* has string 
literals optionally prefixed with 'r' (or 'R', which I did not know 
before) to suppress the normal processing that reduces escape sequences 
to one char.


The term 'cooked' versus 'raw' was used in Unixland at least through the 
1980s. For instance, the input stream from a terminal could be cooked or 
left raw. I presume that is whence came Python's use of 'raw', with much 
the same meaning.


Terry Jan Reedy

--
http://mail.python.org/mailman/listinfo/python-list