Re: python-daemon

2012-12-17 Thread rurpy
On Monday, December 17, 2012 12:33:52 AM UTC-7, Chris Angelico wrote:
> On Mon, Dec 17, 2012 at 6:25 PM,   wrote:
> > No, that's not what you were "just" informing people of...
> > you were also informing us that we are "twits" for finding
> > Google Groups fits our needs better than some other clients.
> 
> I didn't say that. The first twit filter I met was when my dad put
> *his own* address into it, to avoid downloading the posts he'd sent
> (on a modem, that was significant). And that was the most normal thing
> to do, twitting yourself :)

You wrote,

>>> If your post is swallowed by someone's twit filter, that probably
>>> means that you're doing something twittish. 

Perhaps in your part of the world, "twit" means something different 
than it means here, but here the meaning is pretty clear and quite
derogatory.
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Pastebin [was: Trying to make a basic Python score counter in a game... will not count.]

2012-12-17 Thread Chris Angelico
On Mon, Dec 17, 2012 at 5:47 PM, Steven D'Aprano
 wrote:
> On Mon, 17 Dec 2012 07:13:44 +1100, Chris Angelico wrote:
>> I don't understand the idea behind the boycott. Are people worried about
>> the longevity of linked-to content, in the event that pastebin should,
>> as you say, cease to exist tomorrow? Or is it that some won't click a
>> pastebin link in case it's abusive? This isn't the sort of abuse that
>> can compromise your computer.
>
> How do you know? Between javascript and flash, just about any browser
> could be vulnerable to just about any website. You might implicitly trust
> Pastebin, but you can't possibly *know* that the site won't do bad
> things. It wouldn't be the first time that even a reputable website got
> hacked by somebody who used it to deploy malware.

Sure, anything can be compromised. But demonstrations that Pastebin
has been used to disseminate illegal information have nothing to do
with that. However...

> But that's not why I dislike Pastebin. I argue against Pastebin because:
>
> 1) Longevity of the content. Your question is going to be around for
> much, much longer than your pastebin. People searching for help will
> click through to the pastebin and find the code is gone. It is really
> frustrating to (say) search for the solution to a problem, and find that
> the answer is given in an expired pastebin.

... ah, that's a much more serious issue. When do they expire? I
didn't find it on the site. Some of the others expire after a VERY
short time (weeks, or even hours), making them completely
inappropriate for this sort of thing. It really needs to be indefinite
duration.

> 2) When you ask for help via email, you shouldn't assume that the people
> reading have access to the web. Perhaps they have email access, but all
> or part of the web is blocked to them. Perhaps they are reading email on
> a mobile device and don't mind paying to download a couple of KB of
> email, but draw the line at (potentially) hundreds of KB of a web page
> plus associated images, unnecessary javascript, web bugs, advertisements,
> etc. Or maybe they just don't want the context switch:
>
> "I'm reading email right now, I'll click the link later..."
>
> Email is a push technology. A pastebin is a pull technology. Whenever you
> require your audience to actively go and get content, you're cutting your
> audience by some fraction.

Yes, I agree. And I'll go further: I don't like having to download an
attached file. Put your code inline; if it's too long for that, it's
probably too long to be asking about.

There are exceptions, of course, but if I'm going to go fetch
components from elsewhere, I have to have already been drawn into the
thread with strong interest. It's a steep hill to climb.

> "Why should I
> have to go out of my way to find out what your question is? You're asking
> me to do you a favour, and you're making me work to find out what the
> favour is???"

Can't argue with that! I agree, other than that I don't swear. :)

> I'm not saying "never use a paste bin". I think it probably makes lots of
> sense to use one in IRC, where it is inappropriate to paste more than a
> line or two of code at once, and the conversation is already ephemeral.
> But in a Usenet or email forum, I think it is almost always inappropriate
> to use paste bins. If your code is too large to paste directly in the
> body of your email, chances are it is too large to expect people to debug
> for you. But you can try adding it as an attachment (.py, not .doc), and
> only if you can't do that for some reason, then maybe a paste bin is
> appropriate.

Yeah, it's good for a MUD too. We use URL shorteners and such, and
aren't too concerned that tinyurl.com might not exist in a few years
(though it is better to use ones that don't expire URLs). But really,
when you're asking for Python help, you shouldn't need to blat tens of
kay of code at us all. It should be possible to put it in-line.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Unicode

2012-12-17 Thread Anatoli Hristov
> What happens when you do use UTF-8?
This is the result when I encode the string:
" étroits, en utilisant un portable extrêmement puissant—le plus
petit et le plus léger des HP EliteBook pleine puissance—avec un
écran de diagonale 31,75 cm (12,5 pouces), idéal pour le
professionnel ultra-mobile.
"
No accents
>
> What do you mean, "use UTF-8"?

Trying to encode the string
>
>
> To learn about Unicode, start here:
>
> http://www.joelonsoftware.com/articles/Unicode.html
>
> If that helps you solve the problem, excellent. If not, please come back
> with your questions, but first read this:
I will try to understand the logic :)

>
> http://www.sscce.org/
>
> As given, we cannot answer your question easily, or at all, because we
> cannot run your code. It gives indentation errors, you don't tell us what
> modules you're using, and you haven't reduced the example down to the
> critical parts that demonstrate the failure.
I didn't wanted to include all my code as it is 15K. and also I know
my code is crappy and you will start blaming and saying that my code
is crap.- and I know it !

Thanks
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: modify image and save with exif data

2012-12-17 Thread jwe . van . dijk
On Sunday, 16 December 2012 20:43:12 UTC+1, [email protected]  wrote:
> I want to resize an image but retain the exif data
> 
> I now have:
> 
> import Image
> 
> 
> 
> img = Image.open('photo.jpg')
> 
> img.thumbnail((800, 800), Image.ANTIALIAS)
> 
> img.save('photo800.jpg', 'JPEG')
> 
> 
> 
> The saved image photo800.jpg has no exif info anymore.
> 
> I would so much like to have it retained in particular the exposure and gps 
> data.
> 
> 
> 
> I use Python 2.7 with PIL 1.17 on Ubuntu 12.04
> 
> For exif reading/writing I use pyexiv2 0.3.2 but apparently you cannot add 
> tags to an image that has none; only modifying existing ones seems to work.
> 
> 
> 
> Thanks for any helpful suggestions,
> 
> Janwillem

Thanks Vincent,
I tried that but without try-except. I should have done and with also a try 
except in your except it works. It appears that the failure is in a few Nikon 
tags that apparently can be read but not set.

Setting Exif.Nikon3.0x002d failed, 
Setting Exif.Nikon3.0x009d failed, 
Setting Exif.Nikon3.ExposureTuning failed, 
Setting Exif.Nikon3.Preview failed, 

Not important so I am happy with your tip.
Janwillem
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Why Isn't Multiple Inheritance Automatic in Python?

2012-12-17 Thread Ian Kelly
On Sun, Dec 16, 2012 at 9:30 PM, Nick M. Daly  wrote:
> It's very unlikely that multiple inheritance would go horribly wrong, as
> long as classes adopt class-specific argument naming conventions.
> However, ever since bug 1683368 [0] was fixed, it's now impossible to
> cleanly create arbitrary inheritance trees.

No, it isn't.  You just code each class to strip out the particular
arguments that it uses, and by the time you get up to object, either
you've removed all the arguments, or your inheritance tree is buggy.
The fix for bug 1683368 means that this latter case is detected and
raised as an error.

There was a thread not too long ago about the fact that this fix was
recently extended to the __init__ methods of immutable classes, and
while I'm not convinced that this was the correct thing to do, Terry
Reedy pointed out in the issue comments back in 2010 that the proper
way to initialize immutable instances is by overriding __new__ rather
than __init__, the former of which is still perfectly clean to
inherit.

> The only reason I can't
> just take anybody's code and plop it into my inheritance tree is because
> Python demands that each class specifically opts in to MI though
> mechanisms like the following:
>
> 1: class Foo(object):
> 2: def __init__(self, foo_bar, *args, **kwargs):
> 3: if Foo.__mro__[1] != object:
> 4: super().__init__(*args, **kwargs)
> 5:
> 6: self.bar = foo_bar
>
> Lines 3 and 4 are required because Foo might fall at the beginning or
> the middle of in the inheritance tree: we can't know ahead of time.

Of course we can know the full MRO of Foo just by looking at this
code.  Foo derives from object, and nothing else.  The MRO of Foo is
therefore (Foo, object), and the test is always false.  What we can't
know ahead of time is the MRO of *self*, which could be an instance of
a subclass of Foo.  But line 3 is not testing the MRO of self, only of
Foo.  If self happens to be an instance of FooBar, with the MRO
(FooBar, Foo, Bar, object), then the above code will cause bugs,
because Bar.__init__ is never called.

In any case, these lines are not necessary.  The only reason not to
call super() in cooperative MI is if the method does not exist on the
super object.  A better way to test this would be:

if hasattr(super(), '__init__'):
   super().__init__(**kwargs)

However, that test is still silly, since __init__ is a method of
object and *always* exists.  For non-init methods, best practice is to
use a root class (as recommended by Raymond Hettinger [1]).  Anything
that implements the method would inherit from the root class (to
ensure that it will precede the root class in the MRO) and call
super().  The root class serves only to end the chain and does not
call super().


> From my perspective, it'd be lovely if init methods implicitly accepted
> *args and **kwargs while implicitly sending them off to the next class
> in the MRO as the first call.  This would make the previous example
> semantically equivalent to:
>
> 1: class Foo(object):
> 2: def __init__(self, foo_bar):
> 3: self.bar = foo_bar
>
> Granted, that's probably too excessive and implicit for most folks to be
> comfortable with, even though that's obviously the behavior a user
> intends when they write code like:
>
> 1: class Baz(Foo, Bar):
> 2: def __init__(self):
> 3: super().__init__(foo_bar=1, bar_quote="Give me another!")

I don't find that obvious at all.  Does the implicit super() call
happen before or after the body of the method?  There are cases where
the subclass may want to have some code before and some code after.
How do you implicitly call super() in methods that return a value --
what do you implicitly do with the return value of the super() call?
How do you write methods that intentionally do not call super, such as
in the root classes mentioned above, or in methods that are simply
meant to be overridden and not extended?  If the user calls
Baz(foo_bar=42), then does the super() call from Baz still pass
foo_bar=1, or does it implicitly call super() with foo_bar=42 instead,
or does it try to do "super().__init__(foo_bar=1, bar_quote="Give me
another!", foo_bar=42)" and raise a TypeError due to repeated
arguments?
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Unicode

2012-12-17 Thread Benjamin Kaplan
On Mon, Dec 17, 2012 at 12:59 AM, Anatoli Hristov  wrote:
>> What happens when you do use UTF-8?
> This is the result when I encode the string:
> " étroits, en utilisant un portable extrêmement puissant—le plus
> petit et le plus léger des HP EliteBook pleine puissance—avec un
> écran de diagonale 31,75 cm (12,5 pouces), idéal pour le
> professionnel ultra-mobile.
> "
> No accents
>>
>> What do you mean, "use UTF-8"?
>
> Trying to encode the string
>>

What's your terminal's encoding? That looks like you have a CP-1252
terminal trying to output UTF-8 text.
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Unicode

2012-12-17 Thread Anatoli Hristov
> What's your terminal's encoding? That looks like you have a CP-1252
> terminal trying to output UTF-8 text.

Thanks for your answer, I tried  in my terminal and it gives
this as an output:
LANG=en_US
LC_CTYPE="en_US"
LC_NUMERIC="en_US"
LC_TIME="en_US"
LC_COLLATE="en_US"
LC_MONETARY="en_US"
LC_MESSAGES="en_US"
LC_PAPER="en_US"
LC_NAME="en_US"
LC_ADDRESS="en_US"
LC_TELEPHONE="en_US"
LC_MEASUREMENT="en_US"
LC_IDENTIFICATION="en_US"
LC_ALL=
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Unicode

2012-12-17 Thread Anatoli Hristov
> if you only see encoding problems on printing results to your
> terminal, its settings or unicode capability might be the cause,
> however, if you also get badly encoding items in the database, you are
> likely using an inappropriate encoding in some step.

I get badly encoding into my DB

> you seem to be doing something like the following (explicitly or
> partly implicitly, based on your system defaults):
>
 print u"étroits, en utilisant un portable extrêmement 
 puissant".encode("utf-8").decode("windows-1252")
> étroits, en utilisant un portable extrêmement puissant

>
> i.e. encode a text using utf-8 and handling it like windows-1252
> afterwards (or take an already encoded text and decode it with the
> inappropriate ANSI encoding.

Thank you Vlastimil,

I tried to print it as you sholed mr, but I receive an erro:
>>> print u"étroits, en utilisant un portable extrêmement 
>>> puissant".encode("utf-8").decode("windows-1252")
Traceback (most recent call last):
  File "", line 1, in ?
UnicodeEncodeError: 'latin-1' codec can't encode character u'\u0192'
in position 1: ordinal not in range(256)
>>>
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Unicode

2012-12-17 Thread Vlastimil Brom
2012/12/17 Anatoli Hristov :
>> if you only see encoding problems on printing results to your
>> terminal, its settings or unicode capability might be the cause,
>> however, if you also get badly encoding items in the database, you are
>> likely using an inappropriate encoding in some step.
>
> I get badly encoding into my DB
>
>> you seem to be doing something like the following (explicitly or
>> partly implicitly, based on your system defaults):
>>
> print u"étroits, en utilisant un portable extrêmement 
> puissant".encode("utf-8").decode("windows-1252")
>> étroits, en utilisant un portable extrêmement puissant
>
>>
>> i.e. encode a text using utf-8 and handling it like windows-1252
>> afterwards (or take an already encoded text and decode it with the
>> inappropriate ANSI encoding.
>
> Thank you Vlastimil,
>
> I tried to print it as you sholed mr, but I receive an erro:
 print u"étroits, en utilisant un portable extrêmement 
 puissant".encode("utf-8").decode("windows-1252")
> Traceback (most recent call last):
>   File "", line 1, in ?
> UnicodeEncodeError: 'latin-1' codec can't encode character u'\u0192'
> in position 1: ordinal not in range(256)


Hi,
this seems to be an encoding error of your terminal on printing.
You may need to describe (or better post the respective parts of the
source) where the text is coming from (external text file, database
entry, harcoded in the python source ...), how it is stored, retrieved
and possibly manipulated before you insert it to the database.

You may try to print a repr(...) of the string to be inserted to the
database to see, whether it isn't already mangled in some previous
part of the processing.

hth,

vbr
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: python-daemon

2012-12-17 Thread Steven D'Aprano
On Sun, 16 Dec 2012 08:59:46 -0800, rurpy wrote:

> Or you could repost from other than GG if you don't mind being a tool of
> someone else's political agenda.

We're all tools of someone's political agenda.


-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list


Supporting list()

2012-12-17 Thread Skip Montanaro
What method(s) does a class have to support to properly emulate a container
which supports turning it into a list?  For example:

  class Foo:
pass

  f = Foo()
  print list(f)

Is it just __iter__() and next()?  (I'm still using 2.4 and 2.7.)

Thx,

Skip



-- 
http://mail.python.org/mailman/listinfo/python-list


Hot Requirement Business Objects Developer

2012-12-17 Thread Job Alert

Hot Requirement Business Objects Developer Apply here 
http://www.hot-skills.com/display-job/57491/Business-Objects-Developer.html
-- 
http://mail.python.org/mailman/listinfo/python-list


Hot Requirement Business Objects Developer

2012-12-17 Thread Job Alert

Hot Requirement Business Objects Developer Apply here 
http://www.hot-skills.com/display-job/57491/Business-Objects-Developer.html
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Supporting list()

2012-12-17 Thread Dave Angel
On 12/17/2012 09:33 AM, Skip Montanaro wrote:
> What method(s) does a class have to support to properly emulate a container
> which supports turning it into a list?  For example:
>
>   class Foo:
> pass
>
>   f = Foo()
>   print list(f)
>
> Is it just __iter__() and next()?  (I'm still using 2.4 and 2.7.)

I believe the container class needs to define the __iter__() method,
which has to return an iterator object.

That (possibly different) iterator class needs both an __iter__() method
and a next() method.

If the container class is also the iterator class, which is common, then
you just need one __iter__() method, which returns self.



-- 

DaveA

-- 
http://mail.python.org/mailman/listinfo/python-list


Iterating over files of a huge directory

2012-12-17 Thread Gilles Lenfant
Hi,

I have googled but did not find an efficient solution to my problem. My 
customer provides a directory with a hge list of files (flat, potentially 
10+) and I cannot reasonably use os.listdir(this_path) unless creating a 
big memory footprint.

So I'm looking for an iterator that yields the file names of a directory and 
does not make a giant list of what's in.

i.e :

for filename in enumerate_files(some_directory):
# My cooking...

Many thanks by advance.
-- 
Gilles Lenfant
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Supporting list()

2012-12-17 Thread Ethan Furman

Skip Montanaro wrote:

What method(s) does a class have to support to properly emulate a container
which supports turning it into a list?  For example:

  class Foo:
pass

  f = Foo()
  print list(f)

Is it just __iter__() and next()?  (I'm still using 2.4 and 2.7.)



You can either use __iter__ and next to conform to the iterator
protocol, or you can define __getitem__.

If using __getitem__ it needs to work with integers from 0 to len(f)-1,
and raise IndexError for len(f), len(f+1), etc.

~Ethan~
--
http://mail.python.org/mailman/listinfo/python-list


Re: Iterating over files of a huge directory

2012-12-17 Thread Chris Angelico
On Tue, Dec 18, 2012 at 2:28 AM, Gilles Lenfant
 wrote:
> Hi,
>
> I have googled but did not find an efficient solution to my problem. My 
> customer provides a directory with a hge list of files (flat, potentially 
> 10+) and I cannot reasonably use os.listdir(this_path) unless creating a 
> big memory footprint.
>
> So I'm looking for an iterator that yields the file names of a directory and 
> does not make a giant list of what's in.

Sounds like you want os.walk. But... a hundred thousand files? I know
the Zen of Python says that flat is better than nested, but surely
there's some kind of directory structure that would make this
marginally manageable?

http://docs.python.org/3.3/library/os.html#os.walk

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Iterating over files of a huge directory

2012-12-17 Thread Tim Golden
On 17/12/2012 15:41, Chris Angelico wrote:
> On Tue, Dec 18, 2012 at 2:28 AM, Gilles Lenfant 
>  wrote:
>> Hi,
>> 
>> I have googled but did not find an efficient solution to my
>> problem. My customer provides a directory with a hge list of
>> files (flat, potentially 10+) and I cannot reasonably use
>> os.listdir(this_path) unless creating a big memory footprint.
>> 
>> So I'm looking for an iterator that yields the file names of a
>> directory and does not make a giant list of what's in.
> 
> Sounds like you want os.walk. But... a hundred thousand files? I
> know the Zen of Python says that flat is better than nested, but
> surely there's some kind of directory structure that would make this 
> marginally manageable?
> 
> http://docs.python.org/3.3/library/os.html#os.walk

Unfortunately all of the built-in functions (os.walk, glob.glob,
os.listdir) rely on the os.listdir functionality which produces a list
first even if (as in glob.iglob) it later iterates over it.

There are external functions to iterate over large directories in both
Windows & Linux. I *think* the OP is on *nix from his previous posts, in
which case someone else will have to produce the Linux-speak for this.
If it's Windows, you can use the FindFilesIterator in the pywin32 package.

TJG
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Supporting list()

2012-12-17 Thread Skip Montanaro
> If using __getitem__ it needs to work with integers from 0 to len(f)-1,
> and raise IndexError for len(f), len(f+1), etc.

Ah, thanks.  I have a __getitem__ method, but it currently doesn't raise
IndexError.  (I'm indexing into a ring buffer, and the usage of the class
pretty much precludes indexing with an out-of-bounds index.)

S



-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Iterating over files of a huge directory

2012-12-17 Thread Oscar Benjamin
On 17 December 2012 15:28, Gilles Lenfant  wrote:
> I have googled but did not find an efficient solution to my problem. My 
> customer provides a directory with a hge list of files (flat, potentially 
> 10+) and I cannot reasonably use os.listdir(this_path) unless creating a 
> big memory footprint.
>
> So I'm looking for an iterator that yields the file names of a directory and 
> does not make a giant list of what's in.
>
> i.e :
>
> for filename in enumerate_files(some_directory):
> # My cooking...

In the last couple of months there has been a lot of discussion (on
python-list or python-dev - not sure) about creating a library to more
efficiently iterate over the files in a directory. The result so far
is this library on github:
https://github.com/benhoyt/betterwalk

It says there that
"""
Somewhat relatedly, many people have also asked for a version of
os.listdir() that yields filenames as it iterates instead of returning
them as one big list.

So as well as a faster walk(), BetterWalk adds iterdir_stat() and
iterdir(). They're pretty easy to use, but see below for the full API
docs.
"""

Does that code work for you? If so, I imagine the author would be
interested to get some feedback on how well it works.

Alternatively, perhaps consider calling an external utility.


Oscar
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Supporting list()

2012-12-17 Thread Ethan Furman

Dave Angel wrote:

On 12/17/2012 09:33 AM, Skip Montanaro wrote:

What method(s) does a class have to support to properly emulate a container
which supports turning it into a list?  For example:

  class Foo:
pass

  f = Foo()
  print list(f)

Is it just __iter__() and next()?  (I'm still using 2.4 and 2.7.)


I believe the container class needs to define the __iter__() method,
which has to return an iterator object.

That (possibly different) iterator class needs both an __iter__() method
and a next() method.

If the container class is also the iterator class, which is common, then
you just need one __iter__() method, which returns self.


The `next()` method is also needed, as `__iter__()` and `next()` are the two methods that make up 
the iterator protocol (`__next__` in python 3k).


~Ethan~
--
http://mail.python.org/mailman/listinfo/python-list


Re: Iterating over files of a huge directory

2012-12-17 Thread marduk


On Mon, Dec 17, 2012, at 10:28 AM, Gilles Lenfant wrote:
> Hi,
> 
> I have googled but did not find an efficient solution to my problem. My
> customer provides a directory with a hge list of files (flat,
> potentially 10+) and I cannot reasonably use os.listdir(this_path)
> unless creating a big memory footprint.
> 
> So I'm looking for an iterator that yields the file names of a directory
> and does not make a giant list of what's in.
> 
> i.e :
> 
> for filename in enumerate_files(some_directory):
> # My cooking...
> 


You could try using opendir[1] which is a binding to the posix call.  I
believe that it returns an iterator (file-like) of the entries in the
directory.

[1] http://pypi.python.org/pypi/opendir/
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Iterating over files of a huge directory

2012-12-17 Thread Gilles Lenfant
Le lundi 17 décembre 2012 16:52:19 UTC+1, Oscar Benjamin a écrit :
> On 17 December 2012 15:28, Gilles Lenfant <...> wrote:
> 
> 
> In the last couple of months there has been a lot of discussion (on
> 
> python-list or python-dev - not sure) about creating a library to more
> 
> efficiently iterate over the files in a directory. The result so far
> 
> is this library on github:
> 
> https://github.com/benhoyt/betterwalk
> 
> 
> 
> It says there that
> 
> """
> 
> Somewhat relatedly, many people have also asked for a version of
> 
> os.listdir() that yields filenames as it iterates instead of returning
> 
> them as one big list.
> 
> 
> 
> So as well as a faster walk(), BetterWalk adds iterdir_stat() and
> 
> iterdir(). They're pretty easy to use, but see below for the full API
> 
> docs.
> 
> """
> 
> 
> 
> Does that code work for you? If so, I imagine the author would be
> 
> interested to get some feedback on how well it works.
> 
> 
> 
> Alternatively, perhaps consider calling an external utility.
> 

Many thanks for this pointer Oscar.

"betterwalk" is exactly what I was looking for. More particularly iterdir(...) 
and iterdir_stat(...)
I'll get a deeper look at betterwalk and provide (hopefully successful) 
feedback.

Cheers
-- 
Gilles Lenfant
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Supporting list()

2012-12-17 Thread Dave Angel
On 12/17/2012 06:27 PM, Ethan Furman wrote:
> Dave Angel wrote:
>> On 12/17/2012 09:33 AM, Skip Montanaro wrote:
>>> What method(s) does a class have to support to properly emulate a
>>> container
>>> which supports turning it into a list?  For example:
>>>
>>>   class Foo:
>>> pass
>>>
>>>   f = Foo()
>>>   print list(f)
>>>
>>> Is it just __iter__() and next()?  (I'm still using 2.4 and 2.7.)
>>
>> I believe the container class needs to define the __iter__() method,
>> which has to return an iterator object.
>>
>> That (possibly different) iterator class needs both an __iter__() method
>> and a next() method.
>>
>> If the container class is also the iterator class, which is common, then
>> you just need one __iter__() method, which returns self.
> 
> The `next()` method is also needed, as `__iter__()` and `next()` are the
> two methods that make up the iterator protocol (`__next__` in python 3k).
> 
> ~Ethan~
> 
> 

Didn't I say that?  The next() method need not be in the container
class; it needs to be in the iterator class returned by the __iter__()
method.

class MyIter():
def __init__(self, value):
self.internal = value
def __iter__(self):
return self
def next(self):
self.internal += 1
if self.internal > 100:
raise StopIteration
return self.internal


class Container:
def __iter__(self):
return MyIter(42)


for item in Container():
print item

print list(Container())

(tested in Python 2.7)


AHH, upon rereading, I see you misinterpreted what I meant.  I was
trying to say that if there was only one class serving as both container
and iterator, you only needed one of the __iter__() methods instead of
two.  In other words, you need two methods, not three.

-- 

DaveA
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: os.system and subprocess odd behavior

2012-12-17 Thread py_genetic
Thanks! I am using .txt extensions.  Sorry for being a little vague.
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: os.system and subprocess odd behavior

2012-12-17 Thread py_genetic
Thanks for verifying this for me Steven.  I'm glad you are seeing it work.  
It's really the strangest thing.

The issue seems to be with the " > outfile.txt" portion of the command.

The actual command is running a query on a verticalDB and dumping the result.  
The EXACT command run from the command line runs just fine.  

Even if I use the simple cat command to and out file as just a simple test 
case...  The file is created with zero bytes (see below)... but its as if 
python moves on or gets an 0 exit code after the first part of the cmd is 
executed  and no data is written.

-rw-r--r-- 1 root root0 Dec 14 15:33 QUAD_12142012203251.TXT

Any thoughts as to why on my end this may happen?

Thanks again!



-- 
http://mail.python.org/mailman/listinfo/python-list


Re: os.system and subprocess odd behavior

2012-12-17 Thread Oscar Benjamin
On 17 December 2012 16:39, py_genetic  wrote:
> Thanks for verifying this for me Steven.  I'm glad you are seeing it work.  
> It's really the strangest thing.
>
> The issue seems to be with the " > outfile.txt" portion of the command.
>
> The actual command is running a query on a verticalDB and dumping the result. 
>  The EXACT command run from the command line runs just fine.
>
> Even if I use the simple cat command to and out file as just a simple test 
> case...  The file is created with zero bytes (see below)... but its as if 
> python moves on or gets an 0 exit code after the first part of the cmd is 
> executed  and no data is written.
>
> -rw-r--r-- 1 root root0 Dec 14 15:33 QUAD_12142012203251.TXT
>
> Any thoughts as to why on my end this may happen?

Because of the root permissions on the file? What happens if you write
to a file that doesn't need privileged access?

Instead of running the "exact command", run the cat commands you
posted (that Steven has confirmed as working) and run them somewhere
in your user directory without root permissions.

Also you may want to use subprocess.check_call as this raises a Python
error if the command returns an error code.


Oscar
-- 
http://mail.python.org/mailman/listinfo/python-list


How to exactly set style in tkinter.ttk

2012-12-17 Thread Netrick
So, we have now in python 3 the tile 
module http://tktable.sourceforge.net/tile/screenshots/unix.html integrated as 
tkinter.ttk. However, in the python and tk docs there is a lot about that tkk, 
but only how to set your own style for specific widgets. There is nothing on 
how to use that built-in theme (the one that interests me is the "Revitalized" 
one, see the link I wrote) as a global style for all widgets. How can I do it? 
I just want my python 3.3 app to use that built-in style, but I can't find 
anything in docs about it.
 
Or we have better way for tkinter on linux to look more native and less ugly? 
(for example IDLE for python 3.3 written in tkinter looks like windows 95 app 
on linux and pretty good on windows xp).
 
Thanks!-- 
http://mail.python.org/mailman/listinfo/python-list


Delete dict and subdict items of some name

2012-12-17 Thread Gnarlodious
Hello. What I want to do is delete every dictionary key/value of the name 
'Favicon' regardless of depth in subdicts, of which there are many. What is the 
best way to do it?

-- Gnarlie
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Iterating over files of a huge directory

2012-12-17 Thread Paul Rudin
Chris Angelico  writes:

> On Tue, Dec 18, 2012 at 2:28 AM, Gilles Lenfant
>  wrote:
>> Hi,
>>
>> I have googled but did not find an efficient solution to my
>> problem. My customer provides a directory with a hge list of
>> files (flat, potentially 10+) and I cannot reasonably use
>> os.listdir(this_path) unless creating a big memory footprint.
>>
>> So I'm looking for an iterator that yields the file names of a
>> directory and does not make a giant list of what's in.
>
> Sounds like you want os.walk. 

But doesn't os.walk call listdir() and that creates a list of the
contents of a directory, which is exactly the initial problem?

> But... a hundred thousand files? I know the Zen of Python says that
> flat is better than nested, but surely there's some kind of directory
> structure that would make this marginally manageable?
>

Sometimes you have to deal with things other people have designed, so
the directory structure is not something you can control. I've run up
against exactly the same problem and made something in C that
implemented an iterator.

It would probably be better if listdir() made an iterator rather than a
list.
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Delete dict and subdict items of some name

2012-12-17 Thread Oscar Benjamin
On 17 December 2012 17:27, Gnarlodious  wrote:
> Hello. What I want to do is delete every dictionary key/value of the name 
> 'Favicon' regardless of depth in subdicts, of which there are many. What is 
> the best way to do it?

You might need to be a bit clearer about what you mean by subdicts. I
don't really know what you mean.

Could you perhaps post some short code that creates the kind of data
structure you are referring to?

e.g. Do you mean something like this?

d = {
'a': {'b': 'Favicon'},
'b': {'c': 'Favicon'},
}


Oscar
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Unicode

2012-12-17 Thread Anatoli Hristov
> Hi,
> I don't know, what the product ID would look like, for this page, but
> assuming, the catalog pages are also utf-8 encoded as well as the
> error page I get, it should work ok; cf.:
You are right, I get it work on Windows too, but not in Linux. I
changed the codec of linux, but still I don't get it

Here is what I get from Linux:

>>> import urllib
>>> opener = urllib.FancyURLopener({})
>>> ffr = 
>>> opener.open("http://prf.icecat.biz/index.cgi?product_id=%s;mi=start;smi=product;shopname=openICEcat-url;lang=fr";
>>>  % (14688538))
>>> src = ffr.read()
>>> print src.decode("utf-8")
Traceback (most recent call last):
  File "", line 1, in ?
UnicodeEncodeError: 'latin-1' codec can't encode character u'\u2122'
in position 17167: ordinal not in range(256)
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Delete dict and subdict items of some name

2012-12-17 Thread Mitya Sirenef

On 12/17/2012 12:27 PM, Gnarlodious wrote:

Hello. What I want to do is delete every dictionary key/value of the name 
'Favicon' regardless of depth in subdicts, of which there are many. What is the 
best way to do it?

-- Gnarlie


Something like this should work:

def delkey(d, key):
if isinstance(d, dict):
if key in d: del d[key]
for val in d.values():
delkey(val, key)


--
Lark's Tongue Guide to Python: http://lightbird.net/larks/

--
http://mail.python.org/mailman/listinfo/python-list


Re: Delete dict and subdict items of some name

2012-12-17 Thread Paul Rubin
Gnarlodious  writes:
> Hello. What I want to do is delete every dictionary key/value of the
> name 'Favicon' regardless of depth in subdicts, of which there are
> many. What is the best way to do it?

Untested:

def unfav(x):
  if type(x) != dict: return x
  return dict((k,unfav(v)) for k,v in x.iteritems() if k != 'favicon')
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Delete dict and subdict items of some name

2012-12-17 Thread Dave Angel
On 12/17/2012 12:27 PM, Gnarlodious wrote:
> Hello. What I want to do is delete every dictionary key/value of the name 
> 'Favicon' regardless of depth in subdicts, of which there are many. What is 
> the best way to do it?
>
> -- Gnarlie
I would write a recursive function that accepts a dict.

In that function, if a key "Favicon" exists, then remove it. Then loop
through the dictionary key/value pairs, and for any value that's an
instance of dict, call yourself recursively.

Give it a try, and if it won't work, supply us with a bit more
information, starting with a sample dict, and the python version you're
aiming at.




-- 

DaveA

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Unicode

2012-12-17 Thread Dave Angel
On 12/17/2012 12:43 PM, Anatoli Hristov wrote:
>> Hi,
>> I don't know, what the product ID would look like, for this page, but
>> assuming, the catalog pages are also utf-8 encoded as well as the
>> error page I get, it should work ok; cf.:
> You are right, I get it work on Windows too, but not in Linux. I
> changed the codec of linux, but still I don't get it
>
> Here is what I get from Linux:
>
 import urllib
 opener = urllib.FancyURLopener({})
 ffr = 
 opener.open("http://prf.icecat.biz/index.cgi?product_id=%s;mi=start;smi=product;shopname=openICEcat-url;lang=fr";
  % (14688538))
 src = ffr.read()
 print src.decode("utf-8")
> Traceback (most recent call last):
>   File "", line 1, in ?
> UnicodeEncodeError: 'latin-1' codec can't encode character u'\u2122'
> in position 17167: ordinal not in range(256)

I can tell you what's happening, but maybe not how to fix it.

src.decode() is creating a unicode string.  The error is not happening
there.  But when print is used with a unicode string, it has to encode
the data.  And for whatever reason, yours is using latin-1, and you have
a character in there which is not in the latin-1 encoding.

My python 2.7 uses utf-8 everywhere (on Linux Ubuntu 11.04).


-- 

DaveA

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Delete dict and subdict items of some name

2012-12-17 Thread Tim Chase
On 12/17/12 11:43, Mitya Sirenef wrote:
> On 12/17/2012 12:27 PM, Gnarlodious wrote:
>> Hello. What I want to do is delete every dictionary key/value
>> of the name 'Favicon' regardless of depth in subdicts, of which
>> there are many. What is the best way to do it?
> 
> Something like this should work:
> 
> def delkey(d, key):
>  if isinstance(d, dict):
>  if key in d: del d[key]
>  for val in d.values():
>  delkey(val, key)

Unless you have something hatefully recursive like

  d = {}
  d["hello"] = d

:-)

However, assuming the initial structure is tree-ish (acyclic),
Mitya's function should do the trick

-tkc





-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Iterating over files of a huge directory

2012-12-17 Thread MRAB

On 2012-12-17 17:27, Paul Rudin wrote:

Chris Angelico  writes:


On Tue, Dec 18, 2012 at 2:28 AM, Gilles Lenfant
 wrote:

Hi,

I have googled but did not find an efficient solution to my
problem. My customer provides a directory with a hge list of
files (flat, potentially 10+) and I cannot reasonably use
os.listdir(this_path) unless creating a big memory footprint.

So I'm looking for an iterator that yields the file names of a
directory and does not make a giant list of what's in.


Sounds like you want os.walk.


But doesn't os.walk call listdir() and that creates a list of the
contents of a directory, which is exactly the initial problem?


But... a hundred thousand files? I know the Zen of Python says that
flat is better than nested, but surely there's some kind of directory
structure that would make this marginally manageable?



Sometimes you have to deal with things other people have designed, so
the directory structure is not something you can control. I've run up
against exactly the same problem and made something in C that
implemented an iterator.



Years ago I had to deal with an in-house application that was written
using a certain database package. The package stored each predefined
query in a separate file in the same directory.

I found that if I packed all the predefined queries into a single file
and then called an external utility to extract the desired query from
the file every time it was needed into a file for the package to use,
not only did it save a significant amount of disk space (hard disks
were a lot smaller then), I also got a significant speed-up!

It wasn't as bad as 10 in one directory, but it was certainly too
many...


It would probably be better if listdir() made an iterator rather than a
list.



--
http://mail.python.org/mailman/listinfo/python-list


Re: Unicode

2012-12-17 Thread Anatoli Hristov
> src.decode() is creating a unicode string.  The error is not happening
> there.  But when print is used with a unicode string, it has to encode
> the data.  And for whatever reason, yours is using latin-1, and you have
> a character in there which is not in the latin-1 encoding.
I fixed the print, I changed the setting of the terminal and also on
the sshconfig, so now when I print I'm able to print out without
problems, but when I tried to run the script I've made it gives me
again the same error :
""Unexpected error: exceptions.UnicodeEncodeError
"""
Maybe I will try to update to 2.7
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Re: Iterating over files of a huge directory

2012-12-17 Thread Evan Driscoll
On 12/17/2012 09:52 AM, Oscar Benjamin wrote:
> In the last couple of months there has been a lot of discussion (on
> python-list or python-dev - not sure) about creating a library to more
> efficiently iterate over the files in a directory. The result so far
> is this library on github:
> https://github.com/benhoyt/betterwalk

This is very useful to know about; thanks.

I actually wrote something very similar on my own (I wanted to get
information about whether each directory entry was a file, directory,
symlink, etc. without separate stat() calls). I'm guessing that the
library you linked is more mature than mine (I only have a Linux
implementation at present, for instance) so I'm happy to see that I
could probably switch to something better... and even happier that it
sounds like it's aiming for inclusion in the standard library.


(Also just for the record and anyone looking for other posts, I'd guess
said discussion was on Python-dev. I don't look at even remotely
everything on python-list (there's just too much), but I do skim most
subject lines and I haven't noticed any discussion on it before now.)

Evan





signature.asc
Description: OpenPGP digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: How to exactly set style in tkinter.ttk

2012-12-17 Thread Vlastimil Brom
2012/12/17 Netrick :
> So, we have now in python 3 the tile module
> http://tktable.sourceforge.net/tile/screenshots/unix.html integrated as
> tkinter.ttk. However, in the python and tk docs there is a lot about that
> tkk, but only how to set your own style for specific widgets. There is
> nothing on how to use that built-in theme (the one that interests me is the
> "Revitalized" one, see the link I wrote) as a global style for all widgets.
> How can I do it? I just want my python 3.3 app to use that built-in style,
> but I can't find anything in docs about it.
>
> Or we have better way for tkinter on linux to look more native and less
> ugly? (for example IDLE for python 3.3 written in tkinter looks like windows
> 95 app on linux and pretty good on windows xp).
>
> Thanks!
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>

Hi,
I don't use ttk myself, but it seems, that the original demo from
Guilherme Polo might contain the functionality you are looking for:
http://code.google.com/p/python-ttk/downloads/list
pyttk-samples-0.1.7.tar.gz
theming.py

in the (top-right) section "Style":  "Themes" (although I see partly
different themes on Windows7 - it's possibly platform dependent).

You may be able to find out the appropriate usage from the source code.

hth,
  vbr
-- 
http://mail.python.org/mailman/listinfo/python-list


Quepy is a framework to transform questions in natural language into queries in a database language.

2012-12-17 Thread quepyproject
We are sharing an open source framework that we made here at
Machinalis: Quepy https://github.com/machinalis/quepy

Quepy is a framework to transform questions in natural language into
queries in a database language.
It can be easily adapted to different types of questions in natural
language, so that with little code you can build your own interface to
a database in natural language.

Currently, Quepy supports only the SPARQL query language, but in
future versions and with the collaboration of the community we are
planning to extend it to other database query languages.

You are invited to participate and collaborate with the project.

We leave here links to the documentation [0], the source code [1], and
also a Pypi package [2].

Also, as an example, we have an online instance of Quepy the interacts
with DBpedia available [3].

Source code for this example instance is available within the Quepy
package so you can kickstart your project from an existing, working
example.

If you like it, or if you have suggestions: Tell us about it! We're
just an email away [4].

Cheers!

[0] https://github.com/machinalis/quepy
[1] http://quepy.readthedocs.org/
[2] http://pypi.python.org/pypi/quepy/
[3] quepy.machinalis.com (Don't expect a QA system, it's an example)
[4] quepyproject[(at)]machinalis.com
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Re: Iterating over files of a huge directory

2012-12-17 Thread Oscar Benjamin
On 17 December 2012 18:40, Evan Driscoll  wrote:
> On 12/17/2012 09:52 AM, Oscar Benjamin wrote:
>> https://github.com/benhoyt/betterwalk
>
> This is very useful to know about; thanks.
>
> I actually wrote something very similar on my own (I wanted to get
> information about whether each directory entry was a file, directory,
> symlink, etc. without separate stat() calls).

The initial goal of betterwalk seemed to be the ability to do os.walk
with fewer stat calls. I think the information you want is part of
what betterwalk finds "for free" from the underlying OS iteration
(without the need to call stat()) but I'm not sure.

> (Also just for the record and anyone looking for other posts, I'd guess
> said discussion was on Python-dev. I don't look at even remotely
> everything on python-list (there's just too much), but I do skim most
> subject lines and I haven't noticed any discussion on it before now.)

Actually, it was python-ideas:
http://thread.gmane.org/gmane.comp.python.ideas/17932
http://thread.gmane.org/gmane.comp.python.ideas/17757
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Unicode

2012-12-17 Thread Anatoli Hristov
> I fixed the print, I changed the setting of the terminal and also on
> the sshconfig, so now when I print I'm able to print out without
> problems, but when I tried to run the script I've made it gives me
> again the same error :
> ""Unexpected error: exceptions.UnicodeEncodeError
> """
> Maybe I will try to update to 2.7

Upgraded to python 27 and still it gives Unexpected error:
exceptions.UnicodeEncodeError. Damn encoders I don'y know what to
do...
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Iterating over files of a huge directory

2012-12-17 Thread Evan Driscoll
On 12/17/2012 01:50 PM, Oscar Benjamin wrote:
> On 17 December 2012 18:40, Evan Driscoll  wrote:
>> On 12/17/2012 09:52 AM, Oscar Benjamin wrote:
>>> https://github.com/benhoyt/betterwalk
>>
>> This is very useful to know about; thanks.
>>
>> I actually wrote something very similar on my own (I wanted to get
>> information about whether each directory entry was a file, directory,
>> symlink, etc. without separate stat() calls).
> 
> The initial goal of betterwalk seemed to be the ability to do os.walk
> with fewer stat calls. I think the information you want is part of
> what betterwalk finds "for free" from the underlying OS iteration
> (without the need to call stat()) but I'm not sure.

Yes, that's my impression as well.


>> (Also just for the record and anyone looking for other posts, I'd guess
>> said discussion was on Python-dev. I don't look at even remotely
>> everything on python-list (there's just too much), but I do skim most
>> subject lines and I haven't noticed any discussion on it before now.)
> 
> Actually, it was python-ideas:
> http://thread.gmane.org/gmane.comp.python.ideas/17932
> http://thread.gmane.org/gmane.comp.python.ideas/17757

Thanks again for the pointers; I'll have to go through that thread. It's
possible I can contribute something; it sounds like at least at one
point the implementation was ctypes-based and is sometimes slower, and I
have both a (now-defunct) C implementation and my current Cython module.
Ironically I haven't actually benchmarked mine. :-)

Evan



signature.asc
Description: OpenPGP digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: os.system and subprocess odd behavior

2012-12-17 Thread py_genetic
Oscar, seems you may be correct.  I need to run this program as a superuser.  
However, after some more tests with simple commands...  I seem to be working 
correctly from any permission level in python Except for the output write 
command from the database to a file.  Which runs fine if I paste it into the 
cmd line.  Also, subprocess.call_check() returns clean.  However, nothing is 
written to the output file when called from python.

so this cmd runs great from the cmd line (sudo or no) however the output file 
in this case is owned by the sysadmin either way... not root?

/usr/local/Calpont/mysql/bin/mysql 
--defaults-file=/usr/local/Calpont/mysql/my.cnf -u root myDB < 
/home/myusr/jobs/APP_JOBS/JOB_XXX.SQL > /home/myusr/jobs/APP_JOBS/JOB_XXX.TXT


When run from sudo python (other files are also created and owned by root 
correctly)  however no output is written from the db command zero byte file 
only (owned by root)... returns to python with no errors.

I'm sorta at a loss.  I'd rather still avoid having python connect to the db 
directly or reading the data from stdout, is a waste or mem and time for what I 
need.

Thanks for any more thoughts.

> 
> Because of the root permissions on the file? What happens if you write
> 
> to a file that doesn't need privileged access?
> 
> 
> 
> Instead of running the "exact command", run the cat commands you
> 
> posted (that Steven has confirmed as working) and run them somewhere
> 
> in your user directory without root permissions.
> 
> 
> 
> Also you may want to use subprocess.check_call as this raises a Python
> 
> error if the command returns an error code.
> 
> 
> 
> 
> 
> Oscar

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Unicode

2012-12-17 Thread Dave Angel
On 12/17/2012 03:00 PM, Anatoli Hristov wrote:
>> I fixed the print, I changed the setting of the terminal and also on
>> the sshconfig, so now when I print I'm able to print out without
>> problems, but when I tried to run the script I've made it gives me
>> again the same error :
>> ""Unexpected error: exceptions.UnicodeEncodeError
>> """
That's not the whole error message. What encoding does it report in the
error?

Maybe I will try to update to 2.7

> Upgraded to python 27 and still it gives Unexpected error:
> exceptions.UnicodeEncodeError. Damn encoders I don'y know what to
> do...

I doubted that 2.7 would make any difference.

1. What does your "terminal' expect. (For all I know you're using
TeraTermPro as a terminal, which doesn't support utf-8.)
Have you looked at the terminal encoding to see what your copy of
Terminal is expecting? On my Ubuntu Linux, I open the terminal with
Ctrl-Alt-t, then in the menu bar, I select
Terminal->SetCharacterEncoding->utf-8

2. What does your environment tell Linux to support? At a bash prompt, try
echo $LANG (there are two other environment variables I've seen
reference to, so this aspect is nuts)

Mine says
en_US.UTF-8

3. What does Python think it was told?
import sys
print sys.stdout.encoding

Mine says
UTF-8


I can force a similar error as follows:


import urllib
opener = urllib.FancyURLopener({})
ffr =
opener.open("http://prf.icecat.biz/index.cgi?product_id=%s;mi=start;smi=product;shopname=openICEcat-url;lang=fr";
% (14688538))
src = ffr.read()

out = src.decode("utf-8").encode("latin-1")

Traceback (most recent call last):
File "anatoli3.py", line 9, in 
src.decode("utf-8").encode("latin-1")
UnicodeEncodeError: 'latin-1' codec can't encode character u'\u2122' in
position 17167: ordinal not in range(256)


And from that it's quite clear that for that particular data, I cannot
use a latin-1 encoder.

So I did a bit of hunting, and I find the offending character is the one
after the word 'Core" in the following quote:

processeurs Intel® Core™ de 3ème génération


The symbol is a trademark symbol and is not part of latin-1. If you're
really stuck with a latin-1 terminal, then you could do something like:

print src.decode("utf-8").encode("latin-1", "ignore")

That says to decode it using utf-8 (because the html declared a utf-8
encoding), and encode it back to latin-1 (because your terminal is stuck
there), then print.


Just realize that once you start using 'ignore' you're going to also
ignore discrepancies that are real. For example, maybe your terminal is
actual something other than either latin-1 or utf-8.


For others that just want to play with a minimal subset:


test = u'processeurs Intel\xae Core\u2122 de 3\xe8me g\xe9n\xe9ration av'
print test
print test.encode("latin-1", "ignore")
print test.encode("latin-1")

produces :

processeurs Intel® Core™ de 3ème génération av
processeurs Intel� Core de 3�me g�n�ration av
Traceback (most recent call last):
File "anatoli3.py", line 22, in 
print test.encode("latin-1")
UnicodeEncodeError: 'latin-1' codec can't encode character u'\u2122' in
position 23: ordinal not in range(256)




-- 

DaveA

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Iterating over files of a huge directory

2012-12-17 Thread Chris Angelico
On Tue, Dec 18, 2012 at 5:29 AM, MRAB  wrote:
> 
> Years ago I had to deal with an in-house application that was written
> using a certain database package. The package stored each predefined
> query in a separate file in the same directory.
>
> I found that if I packed all the predefined queries into a single file
> and then called an external utility to extract the desired query from
> the file every time it was needed into a file for the package to use,
> not only did it save a significant amount of disk space (hard disks
> were a lot smaller then), I also got a significant speed-up!
>
> It wasn't as bad as 10 in one directory, but it was certainly too
> many...
> 

Smart Cache, a web cache that we used to use on our network a while
ago, could potentially make a ridiculous number of subdirectories (one
for each domain you go to). Its solution: Hash the domain, then put it
into partitioning directories - by default, 4x4 of them, meaning that
there were four directories /0/ /1/ /2/ /3/ and the same inside each
of them, so the "real content" was divided sixteen ways. I don't know
if PC file systems are better at it now than they were back in the
mid-90s, but definitely back then, storing too much in one directory
would give a pretty serious performance penalty.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Iterating over files of a huge directory

2012-12-17 Thread Terry Reedy

On 12/17/2012 10:28 AM, Gilles Lenfant wrote:

Hi,

I have googled but did not find an efficient solution to my problem.
My customer provides a directory with a hge list of files (flat,
potentially 10+) and I cannot reasonably use
os.listdir(this_path) unless creating a big memory footprint.


Is is really big enough to be a real problem? See below.


So I'm looking for an iterator that yields the file names of a
directory and does not make a giant list of what's in.

i.e :

for filename in enumerate_files(some_directory): # My cooking...


See http://bugs.python.org/issue11406
As I said there, I personally think (and still do) that listdir should 
have been changed in 3.0 to return an iterator rather than a list. 
Developers who count more than me disagree on the basis that no 
application has the millions of directory entries needed to make space a 
real issue. They also claim that time is a wash either way.


As for space, 10 entries x 100 bytes/entry (generous guess at 
average) = 10,000,000 bytes, no big deal with gigabyte memories. So the 
logic goes. A smaller example from my machine with 3.3.


from sys import getsizeof

def seqsize(seq):
"Get size of flat sequence and contents"
return sum((getsizeof(item) for item in seq), getsizeof(seq))

import os
d = os.listdir()
print(seqsize([1,2,3]), len(d), seqsize(d))
#
172 45 3128

The size per entry is relatively short because the two-level directory 
prefix for each path is only about 15 bytes. By using 3.3 rather than 
3.0-3.2, the all-ascii-char unicode paths only take 1 byte per char 
rather than 2 or 4.


If you disagree with the responses on the issue, after reading them, 
post one yourself with real numbers.


--
Terry Jan Reedy

--
http://mail.python.org/mailman/listinfo/python-list


Re: Delete dict and subdict items of some name

2012-12-17 Thread Mitya Sirenef

On 12/17/2012 01:30 PM, Tim Chase wrote:

On 12/17/12 11:43, Mitya Sirenef wrote:

On 12/17/2012 12:27 PM, Gnarlodious wrote:

Hello. What I want to do is delete every dictionary key/value
of the name 'Favicon' regardless of depth in subdicts, of which
there are many. What is the best way to do it?

Something like this should work:

def delkey(d, key):
  if isinstance(d, dict):
  if key in d: del d[key]
  for val in d.values():
  delkey(val, key)

Unless you have something hatefully recursive like

   d = {}
   d["hello"] = d

:-)


True -- didn't think of that..!

I guess then adding a check 'if val is not d: delkey(val, key)'
would take care of it?

 -m


--
Lark's Tongue Guide to Python: http://lightbird.net/larks/

--
http://mail.python.org/mailman/listinfo/python-list


Re: Delete dict and subdict items of some name

2012-12-17 Thread Chris Angelico
On Tue, Dec 18, 2012 at 8:33 AM, Mitya Sirenef  wrote:
> On 12/17/2012 01:30 PM, Tim Chase wrote:
>>
>> On 12/17/12 11:43, Mitya Sirenef wrote:
>>>
>>> On 12/17/2012 12:27 PM, Gnarlodious wrote:

 Hello. What I want to do is delete every dictionary key/value
 of the name 'Favicon' regardless of depth in subdicts, of which
 there are many. What is the best way to do it?
>>>
>>> Something like this should work:
>>>
>>> def delkey(d, key):
>>>   if isinstance(d, dict):
>>>   if key in d: del d[key]
>>>   for val in d.values():
>>>   delkey(val, key)
>>
>> Unless you have something hatefully recursive like
>>
>>d = {}
>>d["hello"] = d
>>
>> :-)
>
>
> True -- didn't think of that..!
>
> I guess then adding a check 'if val is not d: delkey(val, key)'
> would take care of it?

Nope, recursion could occur anywhere. You'd have to maintain a set of
"visited nodes" (or their id()s, same diff), and skip any that are in
it.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Delete dict and subdict items of some name

2012-12-17 Thread Dave Angel
On 12/17/2012 04:33 PM, Mitya Sirenef wrote:
> On 12/17/2012 01:30 PM, Tim Chase wrote:
>> On 12/17/12 11:43, Mitya Sirenef wrote:
>>> On 12/17/2012 12:27 PM, Gnarlodious wrote:
 Hello. What I want to do is delete every dictionary key/value
 of the name 'Favicon' regardless of depth in subdicts, of which
 there are many. What is the best way to do it?
>>> Something like this should work:
>>>
>>> def delkey(d, key):
>>>   if isinstance(d, dict):
>>>   if key in d: del d[key]
>>>   for val in d.values():
>>>   delkey(val, key)
>> Unless you have something hatefully recursive like
>>
>>d = {}
>>d["hello"] = d
>>
>> :-)
>
> True -- didn't think of that..!
>
> I guess then adding a check 'if val is not d: delkey(val, key)'
> would take care of it?
>
No, that would only cover the self-recursive case.  If there's a dict
which contains another one, which contains the first, then the recursion
is indirect, and much harder to check for.

Checking reliably for arbitrary recursion patterns is tricky, but
do-able.  Most people degenerate into just setting an arbitrary max
depth.  But I can describe two approaches to this kind of problem.

1) build a list of the recursion path at present, and compare against
the whole path, rather than just the tail.  If there are any matches, quit.

2) make the iterator an object, and instantiate two of them.  Then each
recursive level, iterate the main one once, and the secondary one
twice.  If the two ever match, you have a loop.  Deciding what to do at
that point is tricky because you may have processed some nodes multiple
times already.  But at least it'll terminate, and it doesn't use linear
memory to do so.  I call this one the lollypop algorithm.


-- 

DaveA

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Unicode

2012-12-17 Thread Terry Reedy

On 12/17/2012 3:00 PM, Anatoli Hristov wrote:

I fixed the print, I changed the setting of the terminal and also on
the sshconfig, so now when I print I'm able to print out without
problems, but when I tried to run the script I've made it gives me
again the same error :
""Unexpected error: exceptions.UnicodeEncodeError
"""
Maybe I will try to update to 2.7


Upgraded to python 27 and still it gives Unexpected error:
exceptions.UnicodeEncodeError. Damn encoders I don'y know what to
do...


If you are working with unicode, and you can upgrade to 3.3, you will 
probably we happier if you do. This does not solve all problems, but the 
python side is definitely better. (IE, there are unicode bugs in 2.7 
whose fix *is* to upgrade to 3.3.)


That said, retrieving

http://prf.icecat.biz/index.cgi?product_id=14688538;mi=start;smi=product;shopname=openICEcat-url;lang=fr

with Firefox on Win 7 returns a page containing



so I presume the http encoding is also utf-8

Also: printing to the screen in IDLE may work better than with the 
standard interactive console (especially the awful Windows version). I 
have the font set to Lucida Sans Unicode (this may be windows specific) 
which seems to works for all BMP (Basic Multilingual Plane) chars.


--
Terry Jan Reedy

--
http://mail.python.org/mailman/listinfo/python-list


Re: Unicode

2012-12-17 Thread Hans Mulder
On 17/12/12 22:09:04, Dave Angel wrote:
> print src.decode("utf-8").encode("latin-1", "ignore")
> 
> That says to decode it using utf-8 (because the html declared a utf-8
> encoding), and encode it back to latin-1 (because your terminal is stuck
> there), then print.
> 
> 
> Just realize that once you start using 'ignore' you're going to also
> ignore discrepancies that are real. For example, maybe your terminal is
> actual something other than either latin-1 or utf-8.

If you need to see such discrepancies, you can do

print src.decode("utf-8").encode("latin-1", ""xmlcharrefreplace")


That would produce something like:

processeurs Intel® Core™ de 3ème génération av

that is, the problem characters are displayed in &#...; notation.
That is ugly, but sometimes it's the only way to see what character
you really have.

Notice that the number you get is in decimal, where the \u
notation uses hex:

>>> ord(u"\u2122")
8482
>>>


Hope this helps,

-- HansM

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Unicode

2012-12-17 Thread Anatoli Hristov
> I doubted that 2.7 would make any difference.

Yeah this complicated my life even more, all my import functions was
gone - took me 2h to fix all :)
and it does not solved my issue:)
>
> 1. What does your "terminal' expect. (For all I know you're using
> TeraTermPro as a terminal, which doesn't support utf-8.)
> Have you looked at the terminal encoding to see what your copy of
> Terminal is expecting? On my Ubuntu Linux, I open the terminal with
> Ctrl-Alt-t, then in the menu bar, I select
> Terminal->SetCharacterEncoding->utf-8

I'm using putty for windows and I changed the putty to UTF-8 and this
is what solved the problem - ihuu :p
There is no logic, but it solved the issue !

> 2. What does your environment tell Linux to support? At a bash prompt, try
> echo $LANG (there are two other environment variables I've seen
> reference to, so this aspect is nuts)
> Mine says
> en_US.UTF-8

Mine too US.UTF-8, but the putty was in latin1

> 3. What does Python think it was told?
> import sys
> print sys.stdout.encoding
>
> Mine says
> UTF-8
Mine too :p

Thank you Dave you always come with a solution :)
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Unicode

2012-12-17 Thread Anatoli Hristov
>> Just realize that once you start using 'ignore' you're going to also
>> ignore discrepancies that are real. For example, maybe your terminal is
>> actual something other than either latin-1 or utf-8.
>
> If you need to see such discrepancies, you can do
>
> print src.decode("utf-8").encode("latin-1", ""xmlcharrefreplace")
>
>
> That would produce something like:
>
> processeurs Intel® Core™ de 3ème génération av
>
> that is, the problem characters are displayed in &#...; notation.
> That is ugly, but sometimes it's the only way to see what character
> you really have.
>
> Notice that the number you get is in decimal, where the \u
> notation uses hex:

Thanks guys my issue is now solved - the problem came from my Putty
client, it was on latin1 by default and changing it to utf-8, now
works...
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Why Isn't Multiple Inheritance Automatic in Python?

2012-12-17 Thread Terry Reedy

On 12/17/2012 4:14 AM, Ian Kelly wrote:

On Sun, Dec 16, 2012 at 9:30 PM, Nick M. Daly  wrote:

It's very unlikely that multiple inheritance would go horribly wrong, as
long as classes adopt class-specific argument naming conventions.
However, ever since bug 1683368 [0] was fixed, it's now impossible to
cleanly create arbitrary inheritance trees.


No, it isn't.  You just code each class to strip out the particular
arguments that it uses, and by the time you get up to object, either
you've removed all the arguments, or your inheritance tree is buggy.
The fix for bug 1683368 means that this latter case is detected and
raised as an error.


In other words, one could say that the fix forces a clean inheritance tree.


There was a thread not too long ago about the fact that this fix was
recently extended to the __init__ methods of immutable classes,


Since 3.3.0?
>>> object.__init__(object(), 1)
Traceback (most recent call last):
  File "", line 1, in 
object.__init__(object(), 1)
TypeError: object.__init__() takes no parameters
>>> tuple.__init__(tuple(), 1)
>>>

Or were you referring to something else? Or done since?


while I'm not convinced that this was the correct thing to do, Terry
Reedy pointed out in the issue comments back in 2010 that the proper
way to initialize immutable instances is by overriding __new__ rather
than __init__, the former of which is still perfectly clean to
inherit.


It seems to me that tuple, etc., should just inherit __init__ from object.

--
Terry Jan Reedy

--
http://mail.python.org/mailman/listinfo/python-list


Re: Trying to make a basic Python score counter in a game... will not count.

2012-12-17 Thread darnold
On Dec 16, 12:38 pm, tbg  wrote:
> Nice, will have to try it out...

if you're interested in learning Python and/or game programming in
Python, you might want to take a look at http://inventwithpython.com/
.

HTH,
Don
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Delete dict and subdict items of some name

2012-12-17 Thread MRAB

On 2012-12-17 22:00, Dave Angel wrote:

On 12/17/2012 04:33 PM, Mitya Sirenef wrote:

On 12/17/2012 01:30 PM, Tim Chase wrote:

On 12/17/12 11:43, Mitya Sirenef wrote:

On 12/17/2012 12:27 PM, Gnarlodious wrote:

Hello. What I want to do is delete every dictionary key/value
of the name 'Favicon' regardless of depth in subdicts, of which
there are many. What is the best way to do it?

Something like this should work:

def delkey(d, key):
  if isinstance(d, dict):
  if key in d: del d[key]
  for val in d.values():
  delkey(val, key)

Unless you have something hatefully recursive like

   d = {}
   d["hello"] = d

:-)


True -- didn't think of that..!

I guess then adding a check 'if val is not d: delkey(val, key)'
would take care of it?


No, that would only cover the self-recursive case.  If there's a dict
which contains another one, which contains the first, then the recursion
is indirect, and much harder to check for.

Checking reliably for arbitrary recursion patterns is tricky, but
do-able.  Most people degenerate into just setting an arbitrary max
depth.  But I can describe two approaches to this kind of problem.


Wouldn't a set of the id of the visited objects work?
--
http://mail.python.org/mailman/listinfo/python-list


python ldap bind error

2012-12-17 Thread Jorge Alberto Diaz Orozco

hi there.
I'm working with python ldap and I need to authenticate my user.
this is the code I'm using.

import ldap
ldap.set_option(ldap.OPT_REFERRALS,0)
ldap.protocol_version = 3
conn = ldap.initialize("ldap://ldap.domain.cu";)
conn.simple_bind_s("[email protected]","password")

every time I do this it gives me the next error:
ldap.INVALID_DN_SYNTAX: {'info': 'invalid DN', 'desc': 'Invalid DN syntax'}

can someone help me???
I don't know what I'm doing wrong

10mo. ANIVERSARIO DE LA CREACION DE LA UNIVERSIDAD DE LAS CIENCIAS 
INFORMATICAS...
CONECTADOS AL FUTURO, CONECTADOS A LA REVOLUCION

http://www.uci.cu
http://www.facebook.com/universidad.uci
http://www.flickr.com/photos/universidad_uci
--
http://mail.python.org/mailman/listinfo/python-list


Re: Unicode

2012-12-17 Thread Steven D'Aprano
On Mon, 17 Dec 2012 19:36:01 +0100, Anatoli Hristov wrote:

>> src.decode() is creating a unicode string.  The error is not happening
>> there.  But when print is used with a unicode string, it has to encode
>> the data.  And for whatever reason, yours is using latin-1, and you
>> have a character in there which is not in the latin-1 encoding.
> I fixed the print, I changed the setting of the terminal and also on the
> sshconfig, so now when I print I'm able to print out without problems,
> but when I tried to run the script I've made it gives me again the same
> error :
> ""Unexpected error: exceptions.UnicodeEncodeError """

That is not a full Python traceback. Python gives you lots of debugging 
information, in the form of a complete traceback. Use those tracebacks, 
don't ignore them.

Trying to debug code without the full traceback is like trying to read a 
book by reading only every third page.


-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Delete dict and subdict items of some name

2012-12-17 Thread Steven D'Aprano
On Mon, 17 Dec 2012 09:27:48 -0800, Gnarlodious wrote:

> Hello. What I want to do is delete every dictionary key/value of the
> name 'Favicon' regardless of depth in subdicts, of which there are many.
> What is the best way to do it?

Firstly, you should assume we know what you are talking about, rather 
than explain in clear terms. Never show us an example of your data 
structure. That way we can have the entertainment of guessing what the 
structure of the data structure is, and coding for data that you don't 
care about.

Secondly, never tell us what (if anything) you have already tried, so 
that we can share in the same dead-ends and failed attempts. Share and 
share alike.

Thirdly, make sure we don't have a clear idea of what you consider 
"best", e.g. fastest to write, fastest to execute, most readable, most 
easily maintainable, most memory efficient, short enough to be used as a 
one-liner, or something else. For bonus points, this shouldn't be clear 
in your own mind either.

Fourth, never be clear about the functional requirements. For example, do 
you want to mutate the data in place? Or should the original data remain 
untouched, and a modified copy be made? By leaving this unstated or 
subtly implied, you can ensure that our answers have a 50% chance of 
getting it wrong.

If you keep these points in mind, you too can ask open-ended, poor 
questions that generate much discussion but don't solve your actual 
problem.

Oh wait, I see you already know this! Sorry for the noise.


*wink*


-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Delete dict and subdict items of some name

2012-12-17 Thread Oscar Benjamin
On 17 December 2012 23:44, Oscar Benjamin  wrote:
> On 17 December 2012 23:08, MRAB  wrote:
>> Wouldn't a set of the id of the visited objects work?
>
> Of course it would. This is just a tree search.
>
> Here's a depth-first-search function:
>
> def dfs(root, childfunc, func):
> '''depth first search on a tree
> calls func(node) once for each node'''
> visited = set()
> visiting = OrderedDict()
> visiting[id(root)] = it = iter([root])
>
> while True:
> try:
> node = next(it)
> except StopIteration:
> try:
> node, it = visiting.popitem()
> except KeyError:
> return
> key = id(node)
> if isinstance(node, dict) and key not in visited:
> func(node)
> visiting[key] = it = iter(childfunc(node))
> visited.add(key)
>
> Now you can do:
>
> dfs(my_dict_tree, lambda x: x.pop('Favicon', None))

Slight correction:
dfs(g, lambda n: n.values(), lambda x: x.pop('Favicon', None))


Oscar
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: os.system and subprocess odd behavior

2012-12-17 Thread Oscar Benjamin
On 17 December 2012 20:56, py_genetic  wrote:
> Oscar, seems you may be correct.  I need to run this program as a superuser.  
> However, after some more tests with simple commands...  I seem to be working 
> correctly from any permission level in python Except for the output write 
> command from the database to a file.  Which runs fine if I paste it into the 
> cmd line.  Also, subprocess.call_check() returns clean.  However, nothing is 
> written to the output file when called from python.
>
> so this cmd runs great from the cmd line (sudo or no) however the output file 
> in this case is owned by the sysadmin either way... not root?
>
> /usr/local/Calpont/mysql/bin/mysql 
> --defaults-file=/usr/local/Calpont/mysql/my.cnf -u root myDB < 
> /home/myusr/jobs/APP_JOBS/JOB_XXX.SQL > /home/myusr/jobs/APP_JOBS/JOB_XXX.TXT
>
>
> When run from sudo python (other files are also created and owned by root 
> correctly)  however no output is written from the db command zero byte file 
> only (owned by root)... returns to python with no errors.
>
> I'm sorta at a loss.  I'd rather still avoid having python connect to the db 
> directly or reading the data from stdout, is a waste or mem and time for what 
> I need.

Follow through the bash session below

$ cd /usr
$ ls
bin  games  include  lib  local  sbin  share  src
$ touch file
touch: cannot touch `file': Permission denied
$ sudo touch file
[sudo] password for oscar:
$ ls
bin  file  games  include  lib  local  sbin  share  src
$ cat < file > file2
bash: file2: Permission denied
$ sudo cat < file > file2
bash: file2: Permission denied
$ sudo cat < file > file2
bash: file2: Permission denied
$ sudo cat < file | tee file2
tee: file2: Permission denied
$ sudo cat < file | sudo tee file2
$ ls
bin  file  file2  games  include  lib  local  sbin  share  src

The problem is that when you do

  $ sudo cmd > file2

it is sort of like doing

  $ sudo cmd | this_bash_session > file2

so the permissions used to write to file2 are the same as the bash
session rather than the command cmd which has root permissions. By
piping my output into "sudo tee file2" I can get file2 to be written
by a process that has root permissions.

I suspect you have the same problem although it all complicated by the
fact that everything is a subprocess of Python. Is it possibly the
case that the main Python process does not have root permissions but
you are using it to run a command with sudo that then does have root
permissions?

Does piping through something like "sudo tee" help?


Oscar
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Delete dict and subdict items of some name

2012-12-17 Thread Mitya Sirenef

On 12/17/2012 05:00 PM, Dave Angel wrote:

On 12/17/2012 04:33 PM, Mitya Sirenef wrote:

On 12/17/2012 01:30 PM, Tim Chase wrote:

On 12/17/12 11:43, Mitya Sirenef wrote:

On 12/17/2012 12:27 PM, Gnarlodious wrote:

Hello. What I want to do is delete every dictionary key/value
of the name 'Favicon' regardless of depth in subdicts, of which
there are many. What is the best way to do it?

Something like this should work:

def delkey(d, key):
   if isinstance(d, dict):
   if key in d: del d[key]
   for val in d.values():
   delkey(val, key)

Unless you have something hatefully recursive like

d = {}
d["hello"] = d

:-)

True -- didn't think of that..!

I guess then adding a check 'if val is not d: delkey(val, key)'
would take care of it?


No, that would only cover the self-recursive case.  If there's a dict
which contains another one, which contains the first, then the recursion
is indirect, and much harder to check for.

Checking reliably for arbitrary recursion patterns is tricky, but
do-able.  Most people degenerate into just setting an arbitrary max
depth.  But I can describe two approaches to this kind of problem.

1) build a list of the recursion path at present, and compare against
the whole path, rather than just the tail.  If there are any matches, quit.

2) make the iterator an object, and instantiate two of them.  Then each
recursive level, iterate the main one once, and the secondary one
twice.  If the two ever match, you have a loop.  Deciding what to do at
that point is tricky because you may have processed some nodes multiple
times already.  But at least it'll terminate, and it doesn't use linear
memory to do so.  I call this one the lollypop algorithm.




Thanks, this is quite interesting..


--
Lark's Tongue Guide to Python: http://lightbird.net/larks/

--
http://mail.python.org/mailman/listinfo/python-list


Re: Delete dict and subdict items of some name

2012-12-17 Thread Oscar Benjamin
On 17 December 2012 23:08, MRAB  wrote:
> On 2012-12-17 22:00, Dave Angel wrote:
>> On 12/17/2012 04:33 PM, Mitya Sirenef wrote:
>>> On 12/17/2012 01:30 PM, Tim Chase wrote:
 On 12/17/12 11:43, Mitya Sirenef wrote:
> On 12/17/2012 12:27 PM, Gnarlodious wrote:
>>
>> Hello. What I want to do is delete every dictionary key/value
>> of the name 'Favicon' regardless of depth in subdicts, of which
>> there are many. What is the best way to do it?
>
> Something like this should work:
>
> def delkey(d, key):
>   if isinstance(d, dict):
>   if key in d: del d[key]
>   for val in d.values():
>   delkey(val, key)

 Unless you have something hatefully recursive like

d = {}
d["hello"] = d

 :-)
>>>
>>>
>>> True -- didn't think of that..!
>>>
>>> I guess then adding a check 'if val is not d: delkey(val, key)'
>>> would take care of it?
>>>
>> No, that would only cover the self-recursive case.  If there's a dict
>> which contains another one, which contains the first, then the recursion
>> is indirect, and much harder to check for.
>>
>> Checking reliably for arbitrary recursion patterns is tricky, but
>> do-able.  Most people degenerate into just setting an arbitrary max
>> depth.  But I can describe two approaches to this kind of problem.
>>
> Wouldn't a set of the id of the visited objects work?

Of course it would. This is just a tree search.

Here's a depth-first-search function:

def dfs(root, childfunc, func):
'''depth first search on a tree
calls func(node) once for each node'''
visited = set()
visiting = OrderedDict()
visiting[id(root)] = it = iter([root])

while True:
try:
node = next(it)
except StopIteration:
try:
node, it = visiting.popitem()
except KeyError:
return
key = id(node)
if isinstance(node, dict) and key not in visited:
func(node)
visiting[key] = it = iter(childfunc(node))
visited.add(key)

Now you can do:

dfs(my_dict_tree, lambda x: x.pop('Favicon', None))


Although I wouldn't bother with the above unless I had some reason to
expect possible cycles.


Oscar
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Delete dict and subdict items of some name

2012-12-17 Thread Dave Angel
On 12/17/2012 06:08 PM, MRAB wrote:
> On 2012-12-17 22:00, Dave Angel wrote:
>> On 12/17/2012 04:33 PM, Mitya Sirenef wrote:
>>> On 12/17/2012 01:30 PM, Tim Chase wrote:
 On 12/17/12 11:43, Mitya Sirenef wrote:
> On 12/17/2012 12:27 PM, Gnarlodious wrote:
>> Hello. What I want to do is delete every dictionary key/value
>> of the name 'Favicon' regardless of depth in subdicts, of which
>> there are many. What is the best way to do it?
> Something like this should work:
>
> def delkey(d, key):
>   if isinstance(d, dict):
>   if key in d: del d[key]
>   for val in d.values():
>   delkey(val, key)
 Unless you have something hatefully recursive like

d = {}
d["hello"] = d

 :-)
>>>
>>> True -- didn't think of that..!
>>>
>>> I guess then adding a check 'if val is not d: delkey(val, key)'
>>> would take care of it?
>>>
>> No, that would only cover the self-recursive case.  If there's a dict
>> which contains another one, which contains the first, then the recursion
>> is indirect, and much harder to check for.
>>
>> Checking reliably for arbitrary recursion patterns is tricky, but
>> do-able.  Most people degenerate into just setting an arbitrary max
>> depth.  But I can describe two approaches to this kind of problem.
>>
> Wouldn't a set of the id of the visited objects work?

Sure.  But the set will get lots larger than a list, which is limited to
the depth of max recursion.  It also locks a lot more objects in memory,
where the list only locks one per level.

Now, maybe if the search is depth-first, and if you prune the set on the
way back up, then it'll be space efficient.



-- 

DaveA

-- 
http://mail.python.org/mailman/listinfo/python-list


Install python with custom tk-tcl installation

2012-12-17 Thread Derek Thomas
Hello, 

I seem to have a problem because it seems Tkinter assumes relative paths for 
TCL_LIBRARY and TK_LIBRARY.

I am working with the homebrew group to get python27 to install nicely with a 
custom installation of tk and tcl 
(https://github.com/mxcl/homebrew/pull/16626).  However, this breaks because 
homebrew installs tk and tcl in separate directories and then soft-links them 
to /usr/local/lib. When I install python with this method, it breaks tkinter 
with the following error:

```
>>> import Tkinter; root = Tkinter.Tk()
Traceback (most recent call last):
  File "", line 1, in 
  File
"/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk/Tkinter.py",
line 1685, in __init__
self.tk = _tkinter.create(screenName, baseName, className, interactive,
wantobjects, useTk, sync, use)
_tkinter.TclError: Can't find a usable tk.tcl in the following directories:
/usr/local/Cellar/tcl/8.5.9/lib/tcl8.5/tk8.5
/usr/local/Cellar/tcl/8.5.9/lib/tcl8.5/tk8.5/Resources/Scripts
/usr/local/Cellar/tcl/8.5.9/lib/tk8.5
/usr/local/Cellar/tcl/8.5.9/lib/tk8.5/Resources/Scripts
/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/lib/tk8.5
/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/lib/tk8.5/Resources/Scripts
/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/lib/tk8.5
/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/library

This probably means that tk wasn't installed properly.
```

It is looking in the wrong library directory for the tk.tcl file, which is  in 
/usr/local/Cellar/tk/8.5.9/lib/tk8.5/tk.tcl . In fact, it is assuming that the 
tk8.5 lib-directory is installed relative to the tcl8.5 lib-directory which is 
incorrect. It fails because it cannot find the file in the relative directory.

I can fix this by setting the environment variable TCL_LIBRARY to a path that 
is relative to a soft-linked tk-library (/usr/local/lib/tcl8.5), but is there 
someway to set tkinter to search the correct path during installation without 
having to set the environment after installation? I looked in _tkinter.c and 
tkappinit.c but could not find where it was determining the tcl library. 

If you have any advice or thoughts, I would be very appreciative. Thank you in 
advance.
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: os.system and subprocess odd behavior

2012-12-17 Thread photonymous
I hope I understand the question... but shouldn't you wait for the process to 
complete before exiting?

Something like:

pid = subprocess.Popen(...)
pid.wait()

Otherwise, it'll exit before the background process is done. 
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Trying to make a basic Python score counter in a game... will not count.

2012-12-17 Thread Jason Friedman
> if you're interested in learning Python and/or game programming in
> Python, you might want to take a look at http://inventwithpython.com/

And https://www.coursera.org/course/interactivepython.
-- 
http://mail.python.org/mailman/listinfo/python-list


where to view open() function's C implementation source code ?

2012-12-17 Thread iMath
where to view  open() function's C implementation source code ?
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Delete dict and subdict items of some name

2012-12-17 Thread Gnarlodious
This problem is solved, I am so proud of myself for figuring it out! After 
reading some of these ideas I discovered the plist is really lists underneath 
any "Children" key:


from plistlib import readPlist

def explicate(listDicts):
for dict in listDicts:
if 'FavIcon' in dict:
del dict['FavIcon']
if 'Children' in dict:
dict['Children']=explicate(dict['Children'])
return listDicts

listDicts=readPlist(TARGET_FILE)['Children']
explicate(listDicts)
print(listDicts)


This plist is used by the Mac browser iCab for bookmarks. Removing the Favicon 
data shrinks the file by about 99% and speeds uploading.

I am glad everyone had a nice discussion about my question, but it wasn't 
really accurate. Sorry bout that!

-- Gnarlie
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: where to view open() function's C implementation source code ��

2012-12-17 Thread Roy Smith
In article ,
 iMath  wrote:

> where to view  open() function's C implementation source code ?

http://www.python.org/download/releases/

Download the source for the version you're interested in.
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: where to view open() function's C implementation source code �

2012-12-17 Thread iMath
在 2012年12月18日星期二UTC+8下午1时35分58秒,Roy Smith写道:
> In article ,
> 
>  iMath  wrote:
> 
> 
> 
> > where to view  open() function's C implementation source code ?
> 
> 
> 
> http://www.python.org/download/releases/
> 
> 
> 
> Download the source for the version you're interested in.

but which python module is  open() in ?
-- 
http://mail.python.org/mailman/listinfo/python-list


py2exe is on Sourceforge list of top growth projects

2012-12-17 Thread Frank Millman

This is from Sourceforge's monthly update -



Top Growth Projects

We're always on the lookout for projects that might be doing interesting 
things, and a surge in downloads is one of many metrics that we look at 
to identify them. Here's the projects that had the greatest growth in 
the last month.


[...]

py2exe: A distutils extension to create standalone Windows programs from 
python scripts.




It is 19th on a list of 19, but still, it is nice to see. I wonder if 
there was any particular reason for that?


Frank Millman

--
http://mail.python.org/mailman/listinfo/python-list