Re: [Python-Dev] how bugfixes are handled?

[Martin v. Löwis]

Arkadiusz Miskiewicz wrote:
> I wasn't, thanks for information.
> 
> Still few questions... one of developers/commiters reviews patch and commit
> it? Few developers has to review single patch?

As Neal says, a single committer can review and commit. However,
non-committers can also review; this is the point of asking for
patch reviews. In many cases, the initial patch will not be "good
enough": it will lack documentation and test cases, it will contain
bugs, not follow the code formatting guidelines, and it will make
changes irrelevant to the issue being addressed ("gratuitous changes").

A reviewer is supposed to sort these all out, and then end up with
a final recommendation ("accept" or "reject"). Of course, if it is
going to be "reject", there is little point in making the submitter
comply with formal criteria.

Ideally, a committer then will only have to read the entire review
process, agree with it step-by-step, and commit the proposed change.

As a historical note: people doing a lot of reviews eventually end
up as committers, just because it is easier for the other committers
if they also do the final step.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] bytes type discussion

[Stephen J. Turnbull]

> "Guido" == Guido van Rossum <[EMAIL PROTECTED]> writes:

Guido> I think that the implementation of encoding-guessing or
Guido> auto-encoding-upgrade techniques should be left out of the
Guido> standard library design for now.

As far as I can see, little new design is needed.  There's no reason
why an encoding-guesser couldn't be written as a codec that detects
the coding, then dispatches to the appropriate codec.  The only real
issue I know of is that if you ask such a codec "who are you?", there
are two plausible answers: "autoguess" and the codec actually being
used to translate the stream.  If there's no API to ask for both of
those, the API might want generalization.

Guido> As far as searching bytes objects, that shouldn't be a
Guido> problem as long as the search 'string' is also specified as
Guido> a bytes object.

You do need to be a little careful in implementation, as (for example)
"case insensitive" should be meaningless for searching bytes objects.
This would be especially important if searching and collation become
more Unicode conformant.

-- 
School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp
University of TsukubaTennodai 1-1-1 Tsukuba 305-8573 JAPAN
   Ask not how you can "do" free software business;
  ask what your business can "do for" free software.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] bytes.from_hex() [Was: PEP 332 revival in coordination with pep 349?]

[Martin v. Löwis]

Josiah Carlson wrote:
> I would agree that zip is questionable, but 'uu', 'rot13', perhaps 'hex',
> and likely a few others that the two of you may be arguing against
> should stay as encodings, because strictly speaking, they are defined as
> encodings of data.  They may not be encodings of _unicode_ data, but
> that doesn't mean that they aren't useful encodings for other kinds of
> data, some text, some binary, ...

To support them, the bytes type would have to gain a .encode method,
and I'm -1 on supporting bytes.encode, or string.decode.

Why is

s.encode("uu")

any better than

binascii.b2a_uu(s)

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposal: defaultdict

[Raymond Hettinger]

>> Over lunch with Alex Martelli, he proposed that a subclass of dict
>> with this behavior (but implemented in C) would be a good addition to
>> the language

I would like to add something like this to the collections module, but a PEP is 
probably needed to deal with issues like:

* implications of a __getitem__ succeeding while get(value, x) returns x 
(possibly different from the overall default)
* implications of a __getitem__ succeeding while __contains__ would fail
* whether to add this to the collections module (I would say yes)
* whether to allow default functions as well as default values (so you could 
instantiate a new default list)
* comparing all the existing recipes and third-party modules that have already 
done this
* evaluating its fitness for common use cases (i.e. bags and dict of lists).
* lay out a few examples:

# bag like behavior
dd = collections.default_dict()
dd.default(0)
for elem in collection:
dd[elem] += 1

# setdefault-like behavior
dd = collections.default_dict()
dd.default(list)# instantiate a new list for 
empty cells
for page_number, page in enumerate(book):
for word in page.split():
dd[word].append(word)


Raymond 
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Does eval() leak?

[Martin v. Löwis]

John Marshall wrote:
> Should I expect the virtual memory allocation
> to go up if I do the following?

python-dev is a list for discussing development of Python,
not the development with Python. Please post this question
to [EMAIL PROTECTED]

For python-dev, a message explaining where the memory leak
is and how to correct it would be more appropriate. Most
likely, there is no memory leak in eval.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Test failures in test_timeout

[Dmitry Vasiliev]

Steve Holden wrote:
> Thomas Wouters wrote:
>> I'm thinking that it could probably try to connect to a less reliable
>> website, but that's just moving the problem around (and possibly harassing
>> an unsuspecting website, particularly around release-time.) Perhaps the test
>> should try to connect to a known unconnecting site, like a firewalled port
>> on www.python.org? Not something that refuses connections, just something
>> that times out.
>>
> Couldn't the test use subprocess to start a reliably slow server on 
> localhost? It might even be possible to retrieve the ephemeral port 
> number used by the server, to avoid conflicts with already-used ports on 
> the testing machine.

About 3 years ago I submitted the patch for test_timeout which had fixed some 
of these issues:

http://sourceforge.net/tracker/index.php?func=detail&aid=728815&group_id=5470&atid=305470

Now I think the patch need more review and need to be updated for the current 
Python version and maybe some new ideas.

-- 
Dmitry Vasiliev (dima at hlabs.spb.ru)
 http://hlabs.spb.ru
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] bdist_* to stdlib?

[Stephen J. Turnbull]

> "Bob" == Bob Ippolito <[EMAIL PROTECTED]> writes:

Bob> Huh?  What does that have to do with anything?  I've never
Bob> seen a system where /usr/include, /usr/lib, /usr/bin,
Bob> etc. are not all on the same mount.  It's not really any
Bob> different with OS X either.

/usr/share often is on a different mount; that's the whole rationale
for /usr/share.


-- 
School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp
University of TsukubaTennodai 1-1-1 Tsukuba 305-8573 JAPAN
   Ask not how you can "do" free software business;
  ask what your business can "do for" free software.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] http://www.python.org/dev/doc/devel still available

[Bob Ippolito]

On Feb 16, 2006, at 11:35 AM, Benji York wrote:

> Alexander Schremmer wrote:
>> In fact, PHP does it like php.net/functionname which is even  
>> shorter, i.e.
>> they fallback to the documentation if that path does not exist  
>> otherwise.
>
> Like many things PHP, that seems a bit too magical for my tastes.

Not only does it fall back to documentation, it falls back to a  
search for documentation if there isn't a function of that name.

It's a convenient feature, I'm sure people would use it if it was  
there... even if it was something like http://python.org/doc/name

-bob

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposal: defaultdict

[Georg Brandl]

Guido van Rossum wrote:

>   d = DefaultDict([])
> 
> can be written as simply
> 
>   d[key].append(value)

> Feedback?

Probably a good idea, has been proposed multiple times on clpy.
One good thing would be to be able to specify either a default value
or a factory function.

While at it, other interesting dict subclasses could be:
* sorteddict, practically reinvented by every larger project
* keytransformdict, such as d = keytransformdict(str.lower).

Georg

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposal: defaultdict

[Martin v. Löwis]

Guido van Rossum wrote:
> Feedback?

I would like this to be part of the standard dictionary type,
rather than being a subtype.

d.setdefault([]) (one argument) should install a default value,
and d.cleardefault() should remove that setting; d.default
should be read-only. Alternatively, d.default could be assignable
and del-able.

Also, I think has_key/in should return True if there is a default.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Does eval() leak?

[John Marshall]

Hi,

Should I expect the virtual memory allocation
to go up if I do the following?
-
raw = open("data").read()
while True:
d = eval(raw)
-

I would have expected the memory allocated to the
object referenced by d to be deallocated, garbage
collected, and reallocated for the new eval(raw)
results, assigned to d.

The file contains a large, SIMPLE (no self refs; all
native python types/objects) dictionary (>300K).

While doing 'd = eval(raw)' in the python interpreter
I am monitoring the VIRT column of top and it keeps
increasing until I run out of memory.

When I use a safe_eval() from:
 http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/364469
I have no memory problems.

I see this under python 2.3.5 (fast and obvious).

Thanks,
John

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposal: defaultdict

[Walter Dörwald]

Guido van Rossum wrote:

> A bunch of Googlers were discussing the best way of doing the
> following (a common idiom when maintaining a dict of lists of values
> relating to a key, sometimes called a multimap):
> 
>   if key not in d: d[key] = []
>   d[key].append(value)
> 
> An alternative way to spell this uses setdefault(), but it's not very 
> readable:
> 
>   d.setdefault(key, []).append(value)
> 
> and it also suffers from creating an unnecessary list instance.
> (Timings were inconclusive; the approaches are within 5-10% of each
> other in speed.)
> 
> My conclusion is that setdefault() is a failure -- it was a
> well-intentioned construct, but doesn't actually create more readable
> code.
> 
> Google has an internal data type called a DefaultDict which gets
> passed a default value upon construction. Its __getitem__ method,
> instead of raising KeyError, inserts a shallow copy (!) of the given
> default value into the dict when the value is not found. So the above
> code, after
> 
>   d = DefaultDict([])
> 
> can be written as simply
> 
>   d[key].append(value)

Using a shallow copy of the default seems a bit too magical to me. How 
would this be done? Via copy.copy?

And passing [] to the constructor of dict has a different meaning already.

Fetching the default via a static/class method would solve both problems:

class default_dict(dict):
def __getitem__(self, key):
   if key in self:
  return dict.__getitem__(self, key)
   else:
  default = self.getdefault()
  self[key] = default
  return default

class multi_map(default_dict):
@staticmethod
def getdefault(self):
   return []

class counting_dict(default_dict):
@staticmethod
def getdefault(self):
   return 0

> [...]

Bye,
Walter Dörwald
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposal: defaultdict

[Thomas Heller]

> Guido van Rossum wrote:
> 
>>   d = DefaultDict([])
>>
>> can be written as simply
>>
>>   d[key].append(value)
> 
>> Feedback?
> 

Ok, setdefault is a horrible name.  Would it make sense to come up with a 
better name?

Georg Brandl wrote:

> Probably a good idea, has been proposed multiple times on clpy.
> One good thing would be to be able to specify either a default value
> or a factory function.
> 
> While at it, other interesting dict subclasses could be:
> * sorteddict, practically reinvented by every larger project

You mean ordereddict, not sorteddict, I hope.

> * keytransformdict, such as d = keytransformdict(str.lower).

Not sure what you mean by that.

What *I* would like is probably more ambitious:  I want a dict that allows 
case-insensitive
lookup of string keys, plus ideally I want to use it as class or instance 
dictionary.
Use case: COM wrappers.

Thomas

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] bytes.from_hex() [Was: PEP 332 revival in coordination with pep 349?]

[Jason Orendorff]

On 2/15/06, Guido van Rossum <[EMAIL PROTECTED]> wrote:
>  Actually users trying to figure out Unicode would probably be better served> if bytes.encode() and text.decode() did not exist.[...]It would be better if the signature of text.encode() always returned a
bytes object. But why deny the bytes object a decode() method if textobjects have an encode() method?
I agree, text.encode() and bytes.decode() are both swell.  It's the
other two that bother me.
I'd say there are two "symmetric" API flavors possible (t and b are
text and bytes objects, respectively, where text is a string type,either str or unicode; enc is an encoding name):- b.decode(enc) -> t; t.encode(enc) -> b- b = bytes(t, enc); t = text(b, enc)
I'm not sure why one flavor would be preferred over the other,although having both would probably be a mistake.
I prefer constructor flavor; the word "bytes" feels more concrete than
"encode".  But I worry about constructors being too overloaded.

>>> text(b, enc)  # decode
>>> text(mydict)  # repr
>>> text(b)   # uh... decode with default encoding?

-j

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] bytes.from_hex() [Was: PEP 332 revival in coordination with pep 349?]

[Bob Ippolito]

On Feb 16, 2006, at 9:20 PM, Josiah Carlson wrote:

>
> Greg Ewing <[EMAIL PROTECTED]> wrote:
>>
>> Josiah Carlson wrote:
>>
>>> They may not be encodings of _unicode_ data,
>>
>> But if they're not encodings of unicode data, what
>> business do they have being available through
>> someunicodestring.encode(...)?
>
> I had always presumed that bytes objects are going to be able to be a
> source for encode AND decode, like current non-unicode strings are  
> able
> to be today.  In that sense, if I have a bytes object which is an
> encoding of rot13, hex, uu, etc., or I have a bytes object which I  
> would
> like to be in one of those encodings, I should be able to do  
> b.encode(...)
> or b.decode(...), given that 'b' is a bytes object.
>
> Are 'encodings' going to become a mechanism to encode and decode
> _unicode_ strings, rather than a mechanism to encode and decode _text
> and data_ strings?  That would seem like a backwards step to me, as  
> the
> email package would need to package their own base-64 encode/decode  
> API
> and implementation, and similarly for any other package which uses any
> one of the encodings already available.

It would be VERY useful to separate the two concepts.  bytes<->bytes  
transforms should be one function pair, and bytes<->text transforms  
should be another.  The current situation is totally insane:

str.decode(codec) -> str or unicode or UnicodeDecodeError or  
ZlibError or TypeError.. who knows what else
str.encode(codec) -> str or unicode or UnicodeDecodeError or  
TypeError... probably other exceptions

Granted, unicode.encode(codec) and unicode.decode(codec) are actually  
somewhat sane in that the return type is always a str and the  
exceptions are either UnicodeEncodeError or UnicodeDecodeError.

I think that rot13 is the only conceptually text<->text transform  
(though the current implementation is really bytes<->bytes),  
everything else is either bytes<->text or bytes<->bytes.

-bob

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] http://www.python.org/dev/doc/devel still available

[Georg Brandl]

Bob Ippolito wrote:
> On Feb 16, 2006, at 11:35 AM, Benji York wrote:
> 
>> Alexander Schremmer wrote:
>>> In fact, PHP does it like php.net/functionname which is even  
>>> shorter, i.e.
>>> they fallback to the documentation if that path does not exist  
>>> otherwise.
>>
>> Like many things PHP, that seems a bit too magical for my tastes.
> 
> Not only does it fall back to documentation, it falls back to a  
> search for documentation if there isn't a function of that name.
> 
> It's a convenient feature, I'm sure people would use it if it was  
> there... even if it was something like http://python.org/doc/name

Yes. Either that or docs.python.org/... would be nice.

(alongside with the "custom markers" I proposed one time so that there
can be "speaking" URLs like

docs.python.org/mutable-default-arguments

)

Georg

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposal: defaultdict

[Steve Holden]

Thomas Wouters wrote:
> On Thu, Feb 16, 2006 at 01:11:49PM -0800, Guido van Rossum wrote:
> 
> 
>>Over lunch with Alex Martelli, he proposed that a subclass of dict
>>with this behavior (but implemented in C) would be a good addition to
>>the language. It looks like it wouldn't be hard to implement. It could
>>be a builtin named defaultdict. The first, required, argument to the
>>constructor should be the default value. Remaining arguments (even
>>keyword args) are passed unchanged to the dict constructor.
> 
> 
> Should a dict subclass really change the constructor/initializer signature
> in an incompatible way?
> 
Dict is a particularly difficult type to subclass anyway, given that it 
can take an arbitrary number of arbitrarily-named keyword arguments 
(among many other argument styles).

The proposed behavior is exactly how Icon tables behaved, and it was 
indeed useful in that language. Guido is right about setdefault being a 
busted flush.

If there's no way to resolve the signature issue (which there may not 
be, given that

dict({'one': 2, 'two': 3})
dict({'one': 2, 'two': 3}.items())
dict({'one': 2, 'two': 3}.iteritems())
dict(zip(('one', 'two'), (2, 3)))
dict([['two', 3], ['one', 2]])
dict(one=2, two=3)
dict([(['one', 'two'][i-2], i) for i in (2, 3)])

are all valid calls to the type) then a factory function would be a very 
acceptable substitute, no? (The function could make use of a subclass - 
there's surely no necessity to provide the default as an initializer 
argument: it could be provided as an argument to a method present only 
in the subclass).

wishing-i-could-have-lunch-with-alex-ly y'rs  - steve
-- 
Steve Holden   +44 150 684 7255  +1 800 494 3119
Holden Web LLC www.holdenweb.com
PyCon TX 2006  www.python.org/pycon/

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] bytes.from_hex()

[Stephen J. Turnbull]

> "Guido" == Guido van Rossum <[EMAIL PROTECTED]> writes:

Guido> I'd say there are two "symmetric" API flavors possible (t
Guido> and b are text and bytes objects, respectively, where text
Guido> is a string type, either str or unicode; enc is an encoding
Guido> name):

Guido> - b.decode(enc) -> t; t.encode(enc) -> b

-0  When taking a binary file and attaching it to the text of a mail
message using BASE64, the tendency to say you're "encoding the file in
BASE64" is very strong.  I just don't see how such usages can be
avoided in discussion, which makes the types of decode and encode hard
to remember, and easy to mistake in some contexts.

Guido> - b = bytes(t, enc); t = text(b, enc)

+1  The coding conversion operation has always felt like a constructor
to me, and in this particular usage that's exactly what it is.  I
prefer the nomenclature to reflect that.


-- 
School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp
University of TsukubaTennodai 1-1-1 Tsukuba 305-8573 JAPAN
   Ask not how you can "do" free software business;
  ask what your business can "do for" free software.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Please comment on PEP 357 -- adding nb_index slot to PyNumberMethods

[Georg Brandl]

Bernhard Herzog wrote:
> "Travis E. Oliphant" <[EMAIL PROTECTED]> writes:
> 
>> 2) The __index__ special method will have the signature
>>
>>def __index__(self):
>>return obj
>>
>>Where obj must be either an int or a long or another object
>>that has the __index__ special method (but not self).
> 
> So int objects will not have an __index__ method (assuming that ints
> won't return a different but equal int object).  However:
> 
>> 4) A new operator.index(obj) function will be added that calls
>>equivalent of obj.__index__() and raises an error if obj does not
>>implement the special method.
> 
> So operator.index(1) will raise an exception.  I would expect
> operator.index to be implemented using PyNumber_index.

I'd expect that __index__ won't be called on an int in the first place.

Georg

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] bdist_* to stdlib?

[Jan Claeys]

Op wo, 15-02-2006 te 11:23 -0800, schreef Bob Ippolito:
> On Feb 15, 2006, at 4:49 AM, Jan Claeys wrote:
> 
> > Op wo, 15-02-2006 te 14:00 +1300, schreef Greg Ewing:
> >> I'm disappointed that the various Linux distributions
> >> still don't seem to have caught onto the very simple
> >> idea of *not* scattering files all over the place when
> >> installing something.

> > Those directories might be mounted on entirely different hardware  
> > (even over a network), often with different characteristics (access speed,
> > writeability, etc.).
> 
> Huh?  What does that have to do with anything?  I've never seen a  
> system where /usr/include, /usr/lib, /usr/bin, etc. are not all on  
> the same mount.  It's not really any different with OS X either.

Paths like /etc, /var, /srv, /usr/include and /usr/share are good
candidates to be on another mount than the bin & lib directories...


BTW, Mac-style packages do exist for Linux too, if you prefer that.
Look e.g. at Klik: 


-- 
Jan Claeys

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Test failures in test_timeout

[Steve Holden]

Thomas Wouters wrote:
> I'm seeing spurious test failures in test_timeout, on my own workstation and
> on macteagle.python.org (now that it crashes less; Apple sent over some new
> memory.) The problem is pretty simple: both macteagle and my workstation
> live too closely, network-wise, to www.python.org:
> 
> class TimeoutTestCase(unittest.TestCase):
> [...]
> def setUp(self):
> [...]
> self.addr_remote = ('www.python.org', 80)
> [...]
> def testConnectTimeout(self):
> # Test connect() timeout
> _timeout = 0.001
> self.sock.settimeout(_timeout)
> 
> _t1 = time.time()
> self.failUnlessRaises(socket.error, self.sock.connect,
> self.addr_remote)
> 
> In other words, the test fails because www.python.org responds too quickly.
> 
> The test on my workstation only fails occasionally, but I do expect
> macteagle's failure to be more common (since it's connected to
> www.python.org through (literally) a pair of gigabit switches, whereas my
> workstation has to pass through a few more switches, two Junipers and some
> dark fiber.) Lowering the timeout has no effect, as far as I can tell, which
> is probably a granularity issue.
> 
> I'm thinking that it could probably try to connect to a less reliable
> website, but that's just moving the problem around (and possibly harassing
> an unsuspecting website, particularly around release-time.) Perhaps the test
> should try to connect to a known unconnecting site, like a firewalled port
> on www.python.org? Not something that refuses connections, just something
> that times out.
> 
Couldn't the test use subprocess to start a reliably slow server on 
localhost? It might even be possible to retrieve the ephemeral port 
number used by the server, to avoid conflicts with already-used ports on 
the testing machine.

regards
  Steve
-- 
Steve Holden   +44 150 684 7255  +1 800 494 3119
Holden Web LLC www.holdenweb.com
PyCon TX 2006  www.python.org/pycon/

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Please comment on PEP 357 -- adding nb_index slot to PyNumberMethods

[Bernhard Herzog]

"Travis E. Oliphant" <[EMAIL PROTECTED]> writes:

> 2) The __index__ special method will have the signature
>
>def __index__(self):
>return obj
>
>Where obj must be either an int or a long or another object
>that has the __index__ special method (but not self).

So int objects will not have an __index__ method (assuming that ints
won't return a different but equal int object).  However:

> 4) A new operator.index(obj) function will be added that calls
>equivalent of obj.__index__() and raises an error if obj does not
>implement the special method.

So operator.index(1) will raise an exception.  I would expect
operator.index to be implemented using PyNumber_index.

   Bernhard

-- 
Intevation GmbH http://intevation.de/
Skencil   http://skencil.org/
Thuban  http://thuban.intevation.org/
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 338 issue finalisation (was Re: 2.5 PEP)

[Guido van Rossum]

On 2/16/06, Paul Moore <[EMAIL PROTECTED]> wrote:
> On 2/16/06, Guido van Rossum <[EMAIL PROTECTED]> wrote:
> > On 2/16/06, Nick Coghlan <[EMAIL PROTECTED]> wrote:
>
> > > The PEP itself requests that a string be returned from get_data(), but 
> > > doesn't
> > > require that the file be opened in text mode. Perhaps the PEP 302 
> > > emulation
> > > should use binary mode here? Otherwise there could be strange data 
> > > corruption
> > > bugs on Windows.
> >
> > But PEP 302 shows as its only example reading from a file with a .txt
> > extension. Adding spurious \r characters is also data corruption. We
> > should probably post to python-dev a request for clarification of PEP
> > 302, but in the mean time I vote for text mode.
>
> FWIW, the .txt example was just a toy example. I'd say that binary
> mode makes sense, as I can imagine using the get_data interface to
> load image files, for example. It makes getting text files a bit
> harder (you have to munge CRLF manually) but at least you have the
> *option* of getting binary files.
>
> On reflection, get_data should probably have been given a mode
> argument. But given that it didn't, binary seems safest.
>
> OTOH, I don't know who actually *uses* get_data for real (PJE, for
> eggs? py2exe?). Their opinions are likely to be of more importance.
>
> On the third hand, doing whatever the zipimport module does is likely
> to be right, as that's the key bit of prior art.

It doesn't do any CRLF -> LF translation so this supports the binary theory.

> Regardless, the PEP should be clarified. I'll make the change once
> agreement is reached.

Thanks. Based on the zipimport precedent I propose to make it binary.
The example could be changed to read a GIF image.

--
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Please comment on PEP 357 -- adding nb_index slot to PyNumberMethods

[Nick Coghlan]

Georg Brandl wrote:
> Bernhard Herzog wrote:
>> "Travis E. Oliphant" <[EMAIL PROTECTED]> writes:
>>
>>> 2) The __index__ special method will have the signature
>>>
>>>def __index__(self):
>>>return obj
>>>
>>>Where obj must be either an int or a long or another object
>>>that has the __index__ special method (but not self).
>> So int objects will not have an __index__ method (assuming that ints
>> won't return a different but equal int object).  However:
>>
>>> 4) A new operator.index(obj) function will be added that calls
>>>equivalent of obj.__index__() and raises an error if obj does not
>>>implement the special method.
>> So operator.index(1) will raise an exception.  I would expect
>> operator.index to be implemented using PyNumber_index.
> 
> I'd expect that __index__ won't be called on an int in the first place.

The PEP has been updated to cover adding the __index__ slot to int/long so 
that "one check finds all". The slot will just get bypassed for ints and longs 
by a lot of the C code in the interpreter.

Cheers,
Nick.

-- 
Nick Coghlan   |   [EMAIL PROTECTED]   |   Brisbane, Australia
---
 http://www.boredomandlaziness.org
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] str.translate vs unicode.translate (was: Re: str object going in Py3K)

[Bengt Richter]

If str becomes unicode for PY 3000, and we then have bytes as out 
coding-agnostic
byte data, then I think bytes should have the str translation method, with a 
tweak
that I would hope could also be done to str now.

BTW, str.translate will presumably become unicode.translate, so
perhaps unicode.translate should grow a compatible deletechars parameter.

But that's not the tweak. The tweak is to eliminate unavoidable pre-conversion 
to unicode
in str(something).translate(u'...', delchars) (and preemptively 
bytes(something).translate(u'...', delchars))

E.g. suppose you now want to write:

s_str.translate(table, delch).encode('utf-8')

Note that s_str has no encoding information, and translate is conceptually just 
a 1:1 substitution
minus characters in delch. But if we want to do one-chr:one-unichr substitution 
by specifying a
256-long table of unicode characters, we cannot. It would be simple to allow 
it, and that's the
tweak I would like. It would allow easy custom decodes.

At the moment, if you want to write the above, you have to introduce a phony 
latin-1 decoding
and write it as (not typo-proof)

s_str.translate(table, delch).decode('latin-1').encode('utf-8') # use 
str.translate
or
s_str.decode('latin-1').translate(mapping).encode('utf-8')  # use 
unicode.translate also for delch

to avoid exceptions if you have non-ascii in your s_str (even if delch would 
have removed them!!)

It seems s_str.translate(table, delchars) wants to convert the s_str to unicode
if table is unicode, and then use unicode.translate (which bombs on delchars!)
instead of just effectively defining str.translate as

def translate(self, table, deletechars=None):
return ''.join((table or isinstance(table,unicode) and uidentity or 
sidentity)[ord(x)] for x in self
   if not deletechars or x not in deletechars)

# For convenience in just pruning with deletechars, s_str.translate('', 
deletechars) deletes without translating,
# and s_str.translate(u'', deletechars)  does the same and then maps to 
same-ord unicode characters
# given
# sidentity = ''.join(chr(i) for i in xrange(256))
# and
# uidentity = u''.join(unichr(i) for i in xrrange(256)).

IMO, if you want unicode.translate, then it doesn't hurt to write 
unicode(s_str).translate and use that.

Let str.translate just use the str ords, so simple custom decodes can be 
written without
the annoyance of e.g.,

UnicodeDecodeError: 'ascii' codec can't decode byte 0xf6 in position 3: 
ordinal not in range(128)

Can we change this for bytes? And why couldn't we change this for str.translate 
now?
Or what am I missing? I certainly would like to miss the above message for 
str.translate :-(

BTW This would also allow taking advantage of features of both translates if 
desired, e.g. by
s_str.translate(unichartable256, 
strdelchrs).translate(uniord_to_ustr_or_uniord_mapping).
(e.g., the latter permits single to multiple-character substitution)

I think at least a tweaked translate method for bytes would be good for py3k,
and I hope we can do it for str.translate now.
It it is just too handy a high speed conversion goodie to forgo IMO.

Regards,
Bengt Richter

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] bytes type discussion

[Steve Holden]

Fredrik Lundh wrote:
> Barry Warsaw wrote:
> 
> 
>>We know at least there will never be a 2.10, so I think we still have
>>time.
> 
> 
> because there's no way to count to 10 if you only have one digit?
> 
> we used to think that back when the gas price was just below 10 SEK/L,
> but they found a way...
> 
IIRC Guido is on record as saying "There will be no Python 2.10 because 
I hate the ambiguity of double-digit minor release numbers", or words to 
that effect.

regards
  Steve
-- 
Steve Holden   +44 150 684 7255  +1 800 494 3119
Holden Web LLC www.holdenweb.com
PyCon TX 2006  www.python.org/pycon/

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposal: defaultdict

[Bengt Richter]

On Thu, 16 Feb 2006 13:11:49 -0800, Guido van Rossum <[EMAIL PROTECTED]> wrote:

>A bunch of Googlers were discussing the best way of doing the
>following (a common idiom when maintaining a dict of lists of values
>relating to a key, sometimes called a multimap):
>
>  if key not in d: d[key] = []
>  d[key].append(value)
>
>An alternative way to spell this uses setdefault(), but it's not very readable:
>
>  d.setdefault(key, []).append(value)
>
>and it also suffers from creating an unnecessary list instance.
>(Timings were inconclusive; the approaches are within 5-10% of each
>other in speed.)
>
>My conclusion is that setdefault() is a failure -- it was a
>well-intentioned construct, but doesn't actually create more readable
>code.
>
>Google has an internal data type called a DefaultDict which gets
>passed a default value upon construction. Its __getitem__ method,
>instead of raising KeyError, inserts a shallow copy (!) of the given
>default value into the dict when the value is not found. So the above
>code, after
>
>  d = DefaultDict([])
>
>can be written as simply
>
>  d[key].append(value)
>
Wouldn't it be more generally powerful to pass type or factory function
to use to instantiate a default object when a missing key is encountered, e.g.

   d = DefaultDict(list)

then

   d[key].append(value)

but then you can also do

   d = DefaultDict(dict)
   d[key].update(a=123)

or

   class Foo(object): pass
   d = DefaultDict(Foo)
   d[key].phone = '415-555-1212'

etc. No worries about generalizing shallow copying either ;-)

  
>Note that of all the possible semantics for __getitem__ that could
>have produced similar results (e.g. not inserting the default in the
>underlying dict, or not copying the default value), the chosen
>semantics are the only ones that makes this example work.
>
>Over lunch with Alex Martelli, he proposed that a subclass of dict
>with this behavior (but implemented in C) would be a good addition to
>the language. It looks like it wouldn't be hard to implement. It could
>be a builtin named defaultdict. The first, required, argument to the
>constructor should be the default value. Remaining arguments (even
>keyword args) are passed unchanged to the dict constructor.
>
>Some more design subtleties:
>
>- "key in d" still returns False if the key isn't there
>- "d.get(key)" still returns None if the key isn't there
>- "d.default" should be a read-only attribute giving the default value
>
>Feedback?
>
See above.

Regards,
Bengt Richter

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] r42396 - peps/trunk/pep-0011.txt

[Neal Norwitz]

[Moving to python-dev]

I don't have a strong opinion.  Any one else have an opinion about
removing --with-wctype-functions from configure?

n
--

On 2/16/06, M.-A. Lemburg <[EMAIL PROTECTED]> wrote:
> neal.norwitz wrote:
> > Author: neal.norwitz
> > Date: Thu Feb 16 06:25:37 2006
> > New Revision: 42396
> >
> > Modified:
> >peps/trunk/pep-0011.txt
> > Log:
> > MAL says this option should go away in bug report 874534:
> >
> > The reason for the removal is that the option causes
> > semantical problems and makes Unicode work in non-standard
> > ways on platforms that use locale-aware extensions to the
> > wc-type functions.
> >
> > Since it wasn't previously announced, we can keep the option until 2.6
> > unless someone feels strong enough to rip it out.
>
> I've been wanting to rip this out for some time now, but
> you're right: I forgot to add this to PEP 11, so let's
> wait for another release.
>
> OTOH, this normally only affects system builders, so perhaps
> we could do this a little faster, e.g. add a warning in the
> first alpha and then rip it out with one of the last betas ?!
>
> > Modified: peps/trunk/pep-0011.txt
> >
> > +Name: Systems using --with-wctype-functions
> > +Unsupported in:   Python 2.6
> > +Code removed in:  Python 2.6
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposal: defaultdict

[Georg Brandl]

Thomas Heller wrote:

>> Probably a good idea, has been proposed multiple times on clpy.
>> One good thing would be to be able to specify either a default value
>> or a factory function.
>> 
>> While at it, other interesting dict subclasses could be:
>> * sorteddict, practically reinvented by every larger project
> 
> You mean ordereddict, not sorteddict, I hope.

Well, yes.

>> * keytransformdict, such as d = keytransformdict(str.lower).
> 
> Not sure what you mean by that.
> 
> What *I* would like is probably more ambitious:  I want a dict that allows 
> case-insensitive
> lookup of string keys

This is exactly what this would do. All keys are transformed to lowercase when
setting and looking up.

> plus ideally I want to use it as class or instance dictionary.
> Use case: COM wrappers.

regards,
Georg

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Rename str/unicode to text

[Michael Hudson]

Guido van Rossum <[EMAIL PROTECTED]> writes:

> OTOH, even if we didn't rename str/unicode to text, opentext would
> still be a good name for the function that opens a text file.

Hnnrgh, not really.  You're not opening a 'text', nor are you
constructing something that might reasonably be called an 'opentext'.
textfile() seems better.

Cheers,
mwh

-- 
  Q: What are 1000 lawyers at the bottom of the ocean?
  A: A good start.
  (A lawyer told me this joke.)
  -- Michael Ströder, comp.lang.python
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] str object going in Py3K

[Walter Dörwald]

Bengt Richter wrote:

> On Wed, 15 Feb 2006 18:57:26 -0800, Guido van Rossum <[EMAIL PROTECTED]> 
> wrote:
> 
>> [...]
>> My expectation is that the Py3k standard I/O library will do all of
>> its own conversions on top of binary files anyway -- if you missed it,
>> I'd like to get rid of any ties to C's stdio.
>>
> Would the standard I/O module have low level utility stream-processing 
> generators
> to do things like linesep normalization in text or splitlines etc? I.e., 
> primitives
> that could be composed for unforseen usefulness, like unix pipeable stuff?
> 
> Maybe they could even be composable with '|' for unixy left->right piping, 
> e.g., on windows
> 
> for line in (os.open('somepath') | linechunker | decoder('latin-1')): ...
> 
> where os.open('path').__or__(linechunker) returns 
> linechunker(os.open('path')),
> which in turn has an __or__ to do similarly. Just had this bf, but ISTM it 
> reads ok.
> The equivalent nested generator expression with same assumed primitives would 
> I guess be
> 
> for line in decoder('latin-1')(linechunker(binaryfile('path'))): ...
> 
> which doesn't have the same natural left to right reading order to match 
> processing order.

I'm currently implementing something like this, which might go into 
IPython. See http://styx.livinglogic.de/~walter/IPython/ipipe.py for 
code. (This requires the current IPython svn trunk)

Examples:

for f in ils("/usr/lib/python2.3/") | ifilter("name.endswith('.py')"):
print f.name, f.size

for p in ipwd | ifilter("shell=='/bin/false'") | isort("uid") | \
ieval('"%s (%s)" % (_.name, _.gecos)'):
print p

The other part of the project is a curses based browser for the output 
of these pipelines. See 
http://styx.livinglogic.de/~walter/IPython/newdir.gif for a screenshot 
of the result of ils("/usr/lib/python2.3/") | 
ifilter("name.endswith('.py')")

Bye,
Walter Dörwald

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposal: defaultdict

[Paul Moore]

On 2/17/06, Raymond Hettinger <[EMAIL PROTECTED]> wrote:
> >> Over lunch with Alex Martelli, he proposed that a subclass of dict
> >> with this behavior (but implemented in C) would be a good addition to
> >> the language
>
> I would like to add something like this to the collections module,

+1

> but a PEP is probably needed to deal with issues like:

+0 (You're probably right, but I fear there's no "perfect answer", so
discussions could go round in circles...)

Paul.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Off-topic: www.python.org

[Fredrik Lundh]

Aahz wrote:

> In all fairness to Tim (and despite the fact that emotionally I agree
> with you), the fact is that there had been essentially no forward motion
> on www.python.org redesign until he went to work.  Even if we end up
> chucking out all his work in favor of something else, I'll consider the
> PSF's money well-spent for bringing the community energy into it.

the problem isn't the work that has already been done, the problem is that
things change, and choices that were made years ago are not necessarily
true today.

more on this in another forum, at some other time.  I'll concentrate on the
library reference for now...





___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposal: defaultdict

[Fuzzyman]

Martin v. Löwis wrote:

  Guido van Rossum wrote:
  
  
Feedback?

  
  
I would like this to be part of the standard dictionary type,
rather than being a subtype.

d.setdefault([]) (one argument) should install a default value,
and d.cleardefault() should remove that setting; d.default
should be read-only. Alternatively, d.default could be assignable
and del-able.

Also, I think has_key/in should return True if there is a default.

  

And exactly what use would it then be ?

Michael Foord


  Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk

  




___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposal: defaultdict

[Fredrik Lundh]

Guido van Rossum wrote:

> A bunch of Googlers were discussing the best way of doing the
> following (a common idiom when maintaining a dict of lists of values
> relating to a key, sometimes called a multimap):
>
>   if key not in d: d[key] = []
>   d[key].append(value)

/.../

> Feedback?

+1.  check it in, already (as collections.defaultdict, perhaps?)

alternatively, you could specialize even further: collections.multimap,
which deals with list values only (that shallow copy thing feels a bit
questionable, but all alternatives feel slightly overgeneralized...)





___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Pre-PEP: The "bytes" object

[Bengt Richter]

On Thu, 16 Feb 2006 12:47:22 -0800, Guido van Rossum <[EMAIL PROTECTED]> wrote:

>On 2/15/06, Neil Schemenauer <[EMAIL PROTECTED]> wrote:
>> This could be a replacement for PEP 332.  At least I hope it can
>> serve to summarize the previous discussion and help focus on the
>> currently undecided issues.
>>
>> I'm too tired to dig up the rules for assigning it a PEP number.
>> Also, there are probably silly typos, etc.   Sorry.
>
>I may check it in for you, although right now it would be good if we
>had some more feedback.
>
>I noticed one behavior in your pseudo-code constructor that seems
>questionable: while in the Q&A section you explain why the encoding is
>ignored when the argument is a str instance, in fact you require an
>encoding (and one that's not "ascii") if the str instance contains any
>non-ASCII bytes. So bytes("\xff") would fail, but bytes("\xff",
>"blah") would succeed. I think that's a bit strange -- if you ignore
>the encoding, you should always ignore it. So IMO bytes("\xff") and
>bytes("\xff", "ascii") should both return the same as bytes([255]).
>Also, there's a code path where the initializer is a unicode instance
>and its encode() method is called with None as the argument. I think
>both could be fixed by setting the encoding to
>sys.getdefaultencoding() if it is None and the argument is a unicode
>instance:
>
>def bytes(initialiser=[], encoding=None):
>if isinstance(initialiser, basestring):
>if isinstance(initialiser, unicode):
>if encoding is None:
>encoding = sys.getdefaultencoding()
>initialiser = initialiser.encode(encoding)
>initialiser = [ord(c) for c in initialiser]
>elif encoding is not None:
>raise TypeError("explicit encoding invalid for non-string "
>"initialiser")
>create bytes object and fill with integers from initialiser
>return bytes object

Two things:
[1]

As the above shows, str is encoding-agnostic and passes through
unmodified to bytes (except by ord).

I am wondering what it would hurt to allow the same for unicode ords,
since unicode is also encoding-agnostic. Please read [2] before
deciding that you have already decided this ;-)

The beauty of a unicode literal IMO is that it launders away
the source encoding into a coding-agnostic character sequence
that has stable ords across the universe, so why not use them?
It also solves a lot of ecaping grief. But see [2]

After all, in either case, an encoding can be specified if so desired. Thus

 def bytes(initialiser=[], encoding=None):
 if isinstance(initialiser, basestring):
 if encoding:
 initialiser = initialiser.encode(encoding) # XXX for str ?? 
see [2]
 initialiser = [ord(c) for c in initialiser]
 elif encoding is not None:
 raise TypeError("explicit encoding invalid for non-string "
 "initialiser")
 create bytes object and fill with integers from initialiser
 return bytes object

[2]---

One thing I wonder is where sys.getdefaultencoding() gets its info, and whether
a module_encoding is also necessary for str arguments with encoding.

E.g. if the source encoding is utf-8, and you want sys.getdefaultencoding()
finally, don't you first have to do decode from the source encoding, rather than
let the default decoding assumption for that be ascii? E.g. for utf-8 source,

initialiser.decode('utf-8').encode(sys.getdefaultencodeing()) ?

works, but

initialiser.encode(sys.getdefaultencodeing())  ?

bombs, because it tries to do .decode('ascii') in place of .decode('utf-8')

Notice where the following fails (where utf-8 source is written to tutf8.py
by tutf.py and using latin-1 as standin for sys.getdefaultencoding())

< tutf.py >---
def test():
latin_1_src = """\
# -*- coding: utf-8 -*-
print '\\nfrom tutf8 import:'
print map(hex,map(ord, 'abc\xf6'))
print map(hex,map(ord,'abc\xf6'.decode('utf-8').encode('latin-1')))
print map(hex,map(ord,repr('abc\xf6'.encode('latin-1'
"""
open('tutf8.py','wb').write(latin_1_src.decode('latin-1').encode('utf-8'))

if __name__ == '__main__':
test()
print '\ntutf8.py utf-8 binary line reprs:'
print '\n'.join(repr(L) for L in open('tutf8.py','rb').read().splitlines())
import tutf8
--
The result:

[20:17] C:\pywk\pydev\pep0332>py24 tutf.py

tutf8.py utf-8 binary line reprs:
'# -*- coding: utf-8 -*-'
"print '\\nfrom tutf8 import:'"
"print map(hex,map(ord, 'abc\xc3\xb6'))"
"print map(hex,map(ord,'abc\xc3\xb6'.decode('utf-8').encode('latin-1')))"
"print map(hex,map(ord,repr('abc\xc3\xb6'.encode('latin-1'"

from tutf8 import:
['0x61', '0x62', '0x63', '0xc3', '0xb6']
['0x61', '0x62', '0x63', '0xf6']
Traceback (most recent call last):
  File "tutf.py", line 15, in ?
import 

Re: [Python-Dev] str object going in Py3K

[Guido van Rossum]

On 2/15/06, Bengt Richter <[EMAIL PROTECTED]> wrote:
> On Wed, 15 Feb 2006 18:57:26 -0800, Guido van Rossum <[EMAIL PROTECTED]> 
> wrote:
> >My expectation is that the Py3k standard I/O library will do all of
> >its own conversions on top of binary files anyway -- if you missed it,
> >I'd like to get rid of any ties to C's stdio.
> >
> Would the standard I/O module have low level utility stream-processing 
> generators
> to do things like linesep normalization in text or splitlines etc? I.e., 
> primitives
> that could be composed for unforseen usefulness, like unix pipeable stuff?

Yes. To get a (very limited) idea of what I'm talking about, see the
sio package in the sandbox:
http://svn.python.org/view/sandbox/trunk/sio/

--
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] http://www.python.org/dev/doc/devel still available

[Alexander Schremmer]

On Wed, 15 Feb 2006 21:13:14 +0100, Georg Brandl wrote:

> If something like Fredrik's new doc system is adopted, it would be extremely
> convenient to refer someone to just
> 
> docs.python.org/os.path.join

In fact, PHP does it like php.net/functionname which is even shorter, i.e.
they fallback to the documentation if that path does not exist otherwise.

Kind regards,
Alexander

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] http://www.python.org/dev/doc/devel still available

[Benji York]

Alexander Schremmer wrote:
> In fact, PHP does it like php.net/functionname which is even shorter, i.e.
> they fallback to the documentation if that path does not exist otherwise.

Like many things PHP, that seems a bit too magical for my tastes.
--
Benji York

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Pre-PEP: The "bytes" object

[Guido van Rossum]

On 2/15/06, Neil Schemenauer <[EMAIL PROTECTED]> wrote:
> This could be a replacement for PEP 332.  At least I hope it can
> serve to summarize the previous discussion and help focus on the
> currently undecided issues.
>
> I'm too tired to dig up the rules for assigning it a PEP number.
> Also, there are probably silly typos, etc.   Sorry.

I may check it in for you, although right now it would be good if we
had some more feedback.

I noticed one behavior in your pseudo-code constructor that seems
questionable: while in the Q&A section you explain why the encoding is
ignored when the argument is a str instance, in fact you require an
encoding (and one that's not "ascii") if the str instance contains any
non-ASCII bytes. So bytes("\xff") would fail, but bytes("\xff",
"blah") would succeed. I think that's a bit strange -- if you ignore
the encoding, you should always ignore it. So IMO bytes("\xff") and
bytes("\xff", "ascii") should both return the same as bytes([255]).
Also, there's a code path where the initializer is a unicode instance
and its encode() method is called with None as the argument. I think
both could be fixed by setting the encoding to
sys.getdefaultencoding() if it is None and the argument is a unicode
instance:

def bytes(initialiser=[], encoding=None):
if isinstance(initialiser, basestring):
if isinstance(initialiser, unicode):
if encoding is None:
encoding = sys.getdefaultencoding()
initialiser = initialiser.encode(encoding)
initialiser = [ord(c) for c in initialiser]
elif encoding is not None:
raise TypeError("explicit encoding invalid for non-string "
"initialiser")
create bytes object and fill with integers from initialiser
return bytes object

BTW, for folks who want to experiment, it's quite simple to create a
working bytes implementation by inheriting from array.array. Here's a
quick draft (which only takes str instance arguments):

from array import array
class bytes(array):
def __new__(cls, data=None):
b = array.__new__(cls, "B")
if data is not None:
b.fromstring(data)
return b
def __str__(self):
return self.tostring()
def __repr__(self):
return "bytes(%s)" % repr(list(self))
def __add__(self, other):
if isinstance(other, array):
return bytes(super(bytes, self).__add__(other))
return NotImplemented

--
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposal: defaultdict

[Thomas Wouters]

On Thu, Feb 16, 2006 at 01:11:49PM -0800, Guido van Rossum wrote:

> Over lunch with Alex Martelli, he proposed that a subclass of dict
> with this behavior (but implemented in C) would be a good addition to
> the language. It looks like it wouldn't be hard to implement. It could
> be a builtin named defaultdict. The first, required, argument to the
> constructor should be the default value. Remaining arguments (even
> keyword args) are passed unchanged to the dict constructor.

Should a dict subclass really change the constructor/initializer signature
in an incompatible way?

-- 
Thomas Wouters <[EMAIL PROTECTED]>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Rename str/unicode to text

[Fredrik Lundh]

Michael Hudson wrote:

>  > OTOH, even if we didn't rename str/unicode to text, opentext would
> > still be a good name for the function that opens a text file.
>
> Hnnrgh, not really.  You're not opening a 'text', nor are you
> constructing something that might reasonably be called an 'opentext'.
> textfile() seems better.

except that in Python, file is a type, and open is an action.

but I agree that textfile reads better (haven't we been through this
a couple of times already, btw?  iirc, my original textfile proposal was
posted in 1846, or so)





___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposal: defaultdict

[Fredrik Lundh]

Martin v. Löwis wrote:

> Also, I think has_key/in should return True if there is a default.

and keys should return all possible key values!





___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Deprecate ``multifile``?

[Georg Brandl]

Hi,

as Jim Jewett noted, multifile is supplanted by email as much as mimify etc.
but it is not marked as deprecated. Should it be deprecated in 2.5?

Georg

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Pre-PEP: The "bytes" object

[M.-A. Lemburg]

Guido van Rossum wrote:
> On 2/15/06, Neil Schemenauer <[EMAIL PROTECTED]> wrote:
>> This could be a replacement for PEP 332.  At least I hope it can
>> serve to summarize the previous discussion and help focus on the
>> currently undecided issues.
>>
>> I'm too tired to dig up the rules for assigning it a PEP number.
>> Also, there are probably silly typos, etc.   Sorry.
> 
> I may check it in for you, although right now it would be good if we
> had some more feedback.
> 
> I noticed one behavior in your pseudo-code constructor that seems
> questionable: while in the Q&A section you explain why the encoding is
> ignored when the argument is a str instance, in fact you require an
> encoding (and one that's not "ascii") if the str instance contains any
> non-ASCII bytes. So bytes("\xff") would fail, but bytes("\xff",
> "blah") would succeed. I think that's a bit strange -- if you ignore
> the encoding, you should always ignore it. So IMO bytes("\xff") and
> bytes("\xff", "ascii") should both return the same as bytes([255]).
> Also, there's a code path where the initializer is a unicode instance
> and its encode() method is called with None as the argument. I think
> both could be fixed by setting the encoding to
> sys.getdefaultencoding() if it is None and the argument is a unicode
> instance:
> 
> def bytes(initialiser=[], encoding=None):
> if isinstance(initialiser, basestring):
> if isinstance(initialiser, unicode):
> if encoding is None:
> encoding = sys.getdefaultencoding()
> initialiser = initialiser.encode(encoding)
> initialiser = [ord(c) for c in initialiser]
> elif encoding is not None:
> raise TypeError("explicit encoding invalid for non-string "
> "initialiser")
> create bytes object and fill with integers from initialiser
> return bytes object
> 
> BTW, for folks who want to experiment, it's quite simple to create a
> working bytes implementation by inheriting from array.array. Here's a
> quick draft (which only takes str instance arguments):
> 
> from array import array
> class bytes(array):
> def __new__(cls, data=None):
> b = array.__new__(cls, "B")
> if data is not None:
> b.fromstring(data)
> return b
> def __str__(self):
> return self.tostring()
> def __repr__(self):
> return "bytes(%s)" % repr(list(self))
> def __add__(self, other):
> if isinstance(other, array):
> return bytes(super(bytes, self).__add__(other))
> return NotImplemented

Another hint:

If you want to play around with the migration
to all Unicode in Py3k, start Python with the -U switch and
monkey-patch the builtin str to be an alias for unicode.

Ideally, the bytes type should work under both the Py3k conditions
and the Py2.x default ones.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Feb 17 2006)
>>> Python/Zope Consulting and Support ...http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/


::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! 
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposal: defaultdict

[Raymond Hettinger]

> My conclusion is that setdefault() is a failure -- it was a
> well-intentioned construct, but doesn't actually create more readable
> code.

It was an across the board failure:  naming, clarity, efficiency.
Can we agree to slate dict.setdefault() to disappear in Py3.0?


Raymond
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] str object going in Py3K

[Walter Dörwald]

Guido van Rossum wrote:

> On 2/16/06, M.-A. Lemburg <[EMAIL PROTECTED]> wrote:
>> What will be the explicit way to open a file in bytes mode
>> and in text mode (I for one would like to move away from
>> open() completely as well) ?
>>
>> Will we have a single file type with two different modes
>> or two different types ?
> 
> I'm currently thinking of an I/O stack somewhat like Java's. At the
> bottom there's a class that lets you do raw unbuffered reads and
> writes (and seek/tell) on binary files using bytes arrays. We can
> layer onto this buffering, text encoding/decoding, and more.  (Windows
> CRLF<->LF conversion is also an encoding of sorts).
> 
> Years ago I wrote a prototype; checkout sandbox/sio/.

However sio.DecodingInputFilter and sio.EncodingOutputFilter don't work 
for encodings that need state (e.g. when reading/writing UTF-16). 
Switching to stateful encoders/decoders isn't so easy, because the 
stateful codecs require a stream-API, which brings in a whole bunch of 
other functionality (readline() etc.), which we'd probably like to keep 
separate. I have a patch (http://bugs.python.org/1101097) that should 
fix this problem (at least for all codecs derived from 
codecs.StreamReader/codecs.StreamWriter). Additionally it would make 
stateful codecs more useful in the context for iterators/generators.

I'd like this patch to go into 2.5.

Bye,
Walter Dörwald

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposal: defaultdict

[Phillip J. Eby]

At 10:10 AM 02/17/2006 +0100, Georg Brandl wrote:
>Guido van Rossum wrote:
>
> >   d = DefaultDict([])
> >
> > can be written as simply
> >
> >   d[key].append(value)
>
> > Feedback?
>
>Probably a good idea, has been proposed multiple times on clpy.
>One good thing would be to be able to specify either a default value
>or a factory function.

+1 on factory function, e.g. "DefaultDict(list)".  A default value isn't 
very useful, because for immutable defaults, setdefault() works well 
enough.  If what you want is a copy of some starting object, you can always 
do something like DefaultDict({1:2,3:4}.copy).

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Deprecate ``multifile``?

[Fredrik Lundh]

Georg Brandl wrote:

> as Jim Jewett noted, multifile is supplanted by email as much as mimify etc.
> but it is not marked as deprecated. Should it be deprecated in 2.5?

-0.5 (gratuitous breakage).

I think the current "see also/supersedes" link is good enough.

 



___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] str object going in Py3K

[M.-A. Lemburg]

Guido van Rossum wrote:
> On 2/16/06, M.-A. Lemburg <[EMAIL PROTECTED]> wrote:
>> What will be the explicit way to open a file in bytes mode
>> and in text mode (I for one would like to move away from
>> open() completely as well) ?
>>
>> Will we have a single file type with two different modes
>> or two different types ?
> 
> I'm currently thinking of an I/O stack somewhat like Java's. At the
> bottom there's a class that lets you do raw unbuffered reads and
> writes (and seek/tell) on binary files using bytes arrays. We can
> layer onto this buffering, text encoding/decoding, and more. (Windows
> CRLF<->LF conversion is also an encoding of sorts).

Sounds like the stackable StreamWriters and -Readers would
nicely integrate into this design.

> Years ago I wrote a prototype; checkout sandbox/sio/.

Thanks. Maybe one of these days I'll get around to having
a look - unlike many of the pydev folks, I don't work for
Google and can't spend 20% or 50% of my time on
Python core development :-)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Feb 17 2006)
>>> Python/Zope Consulting and Support ...http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/


::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! 
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Deprecate ``multifile``?

[Georg Brandl]

Fredrik Lundh wrote:
> Georg Brandl wrote:
> 
>> as Jim Jewett noted, multifile is supplanted by email as much as mimify etc.
>> but it is not marked as deprecated. Should it be deprecated in 2.5?
> 
> -0.5 (gratuitous breakage).
> 
> I think the current "see also/supersedes" link is good enough.

Well, it would be deprecated like the other email modules, that is, only
a note is added to the docs and it is added to PEP 4. There would be no
warning.

Georg

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] str object going in Py3K

[M.-A. Lemburg]

Walter Dörwald wrote:
> Guido van Rossum wrote:
> 
>> On 2/16/06, M.-A. Lemburg <[EMAIL PROTECTED]> wrote:
>>> What will be the explicit way to open a file in bytes mode
>>> and in text mode (I for one would like to move away from
>>> open() completely as well) ?
>>>
>>> Will we have a single file type with two different modes
>>> or two different types ?
>>
>> I'm currently thinking of an I/O stack somewhat like Java's. At the
>> bottom there's a class that lets you do raw unbuffered reads and
>> writes (and seek/tell) on binary files using bytes arrays. We can
>> layer onto this buffering, text encoding/decoding, and more.  (Windows
>> CRLF<->LF conversion is also an encoding of sorts).
>>
>> Years ago I wrote a prototype; checkout sandbox/sio/.
> 
> However sio.DecodingInputFilter and sio.EncodingOutputFilter don't work
> for encodings that need state (e.g. when reading/writing UTF-16).
> Switching to stateful encoders/decoders isn't so easy, because the
> stateful codecs require a stream-API, which brings in a whole bunch of
> other functionality (readline() etc.), which we'd probably like to keep
> separate. I have a patch (http://bugs.python.org/1101097) that should
> fix this problem (at least for all codecs derived from
> codecs.StreamReader/codecs.StreamWriter). Additionally it would make
> stateful codecs more useful in the context for iterators/generators.
> 
> I'd like this patch to go into 2.5.

The patch as-is won't go into 2.5. It's simply the wrong approach:
StreamReaders and -Writers work on streams (hence the name). It
doesn't make sense adding functionality to side-step this behavior,
since it undermines the design.

Like I suggested in the patch discussion, such functionality could
be factored out of the implementations of StreamReaders/Writers
and put into new StatefulEncoder/Decoder classes, the objects of
which then get used by StreamReader/Writer.

In addition to that we could extend the codec registry to also
maintain slots for the stateful encoders and decoders, if needed.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Feb 17 2006)
>>> Python/Zope Consulting and Support ...http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/


::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! 
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Rename str/unicode to text

[Michael Hudson]

"Fredrik Lundh" <[EMAIL PROTECTED]> writes:

> Michael Hudson wrote:
>
>>  > OTOH, even if we didn't rename str/unicode to text, opentext would
>> > still be a good name for the function that opens a text file.
>>
>> Hnnrgh, not really.  You're not opening a 'text', nor are you
>> constructing something that might reasonably be called an 'opentext'.
>> textfile() seems better.
>
> except that in Python, file is a type, and open is an action.

Well, yeah, but you can interpret each name in a sane way and try to
ignore the fact that they refer to the same object...

> but I agree that textfile reads better (haven't we been through this
> a couple of times already, btw?  iirc, my original textfile proposal was
> posted in 1846, or so)

Yes, that sounds about right.

Cheers,
mwh

-- 
  I'm sorry, was my bias showing again? :-)
  -- William Tanksley, 13 May 2000
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposal: defaultdict

[Fredrik Lundh]

Raymond Hettinger wrote:

> I would like to add something like this to the collections module, but a PEP 
> is
> probably needed to deal with issues like:

frankly, now that Guido is working 50% on Python, do we really have to use
the full PEP process also for simple things like this?

I'd say we let the BDFL roam free.

(if he adds something really lousy, it can always be tweaked/removed before
the next final release.  not every checkin needs to be final...).

 



___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] str.translate vs unicode.translate

[M.-A. Lemburg]

Bengt Richter wrote:
> If str becomes unicode for PY 3000, and we then have bytes as out 
> coding-agnostic
> byte data, then I think bytes should have the str translation method, with a 
> tweak
> that I would hope could also be done to str now.
> 
> BTW, str.translate will presumably become unicode.translate, so
> perhaps unicode.translate should grow a compatible deletechars parameter.

I'd much rather like to see .translate() method deprecated.

Writing a code for the task is much more effective - the
builtin charmap codec will do all the mapping for you,
if you have a need to go from bytes to Unicode and vice-
versa.

We could also have a bytemap codec for doing bytes to bytes
conversions.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Feb 17 2006)
>>> Python/Zope Consulting and Support ...http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/


::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! 
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposal: defaultdict

[Georg Brandl]

Fredrik Lundh wrote:
> Raymond Hettinger wrote:
> 
>> I would like to add something like this to the collections module, but a PEP 
>> is
>> probably needed to deal with issues like:
> 
> frankly, now that Guido is working 50% on Python, do we really have to use
> the full PEP process also for simple things like this?
> 
> I'd say we let the BDFL roam free.
> 
> (if he adds something really lousy, it can always be tweaked/removed before
> the next final release.  not every checkin needs to be final...).

+1.

Georg

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] bytes.from_hex() [Was: PEP 332 revival in coordination with pep 349?]

[M.-A. Lemburg]

Martin v. Löwis wrote:
> Josiah Carlson wrote:
>> I would agree that zip is questionable, but 'uu', 'rot13', perhaps 'hex',
>> and likely a few others that the two of you may be arguing against
>> should stay as encodings, because strictly speaking, they are defined as
>> encodings of data.  They may not be encodings of _unicode_ data, but
>> that doesn't mean that they aren't useful encodings for other kinds of
>> data, some text, some binary, ...
> 
> To support them, the bytes type would have to gain a .encode method,
> and I'm -1 on supporting bytes.encode, or string.decode.
> 
> Why is
> 
> s.encode("uu")
> 
> any better than
> 
> binascii.b2a_uu(s)

The .encode() and .decode() methods are merely convenience
interfaces to the registered codecs (with some extra logic to
make sure that only a pre-defined set of return types are allowed).
It's up to the user to use them for e.g. UU-encoding or not.

The reason we have codecs for UU, zip and the others is that
you can use their StreamWriters/Readers in stackable streams.

Just because some codecs don't fit into the string.decode()
or bytes.encode() scenario doesn't mean that these codecs are
useless or that the methods should be banned.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Feb 17 2006)
>>> Python/Zope Consulting and Support ...http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/


::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! 
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] r42396 - peps/trunk/pep-0011.txt

[M.-A. Lemburg]

Neal Norwitz wrote:
> [Moving to python-dev]
> 
> I don't have a strong opinion.  Any one else have an opinion about
> removing --with-wctype-functions from configure?

FWIW, I announced this plan in Dec 2004:

http://mail.python.org/pipermail/python-dev/2004-December/050193.html

I didn't get any replies back then, so assumed that no-one
would object, but forgot to add this to the PEP 11.

The reason I'd like to get this removed early rather than
later is that some Linux distros happen to use the config
switch causing the Python Unicode implementation on those
distros to behave inconsistent with regular Python
builds.

After all we've put a lot of effort into making sure that
the Unicode implementation does work independently of
the platform, even on platforms that don't have Unicode
support at all.

Another candidate for removal is the --disable-unicode
switch.

We should probably add a deprecation warning for that in
Py 2.5 and then remove the hundreds of
#idef Py_USING_UNICODE
from the source code in time for Py 2.6.

> n
> --
> 
> On 2/16/06, M.-A. Lemburg <[EMAIL PROTECTED]> wrote:
>> neal.norwitz wrote:
>>> Author: neal.norwitz
>>> Date: Thu Feb 16 06:25:37 2006
>>> New Revision: 42396
>>>
>>> Modified:
>>>peps/trunk/pep-0011.txt
>>> Log:
>>> MAL says this option should go away in bug report 874534:
>>>
>>> The reason for the removal is that the option causes
>>> semantical problems and makes Unicode work in non-standard
>>> ways on platforms that use locale-aware extensions to the
>>> wc-type functions.
>>>
>>> Since it wasn't previously announced, we can keep the option until 2.6
>>> unless someone feels strong enough to rip it out.
>> I've been wanting to rip this out for some time now, but
>> you're right: I forgot to add this to PEP 11, so let's
>> wait for another release.
>>
>> OTOH, this normally only affects system builders, so perhaps
>> we could do this a little faster, e.g. add a warning in the
>> first alpha and then rip it out with one of the last betas ?!
>>
>>> Modified: peps/trunk/pep-0011.txt
>>>
>>> +Name: Systems using --with-wctype-functions
>>> +Unsupported in:   Python 2.6
>>> +Code removed in:  Python 2.6
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/mal%40egenix.com

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Feb 17 2006)
>>> Python/Zope Consulting and Support ...http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/


::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! 
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 338 issue finalisation (was Re: 2.5 PEP)

[Nick Coghlan]

Guido van Rossum wrote:
> [Hey, I thought I sent that just to you. Is python-dev really
> interested in this?]

Force of habit on my part - I saw the python-dev header and automatically 
dropped "pyd" into the To: field of the reply.

Given Paul's contribution on the get_data front, it turned out to be a 
fortuitous accident :)


[init_globals argument]
>> I just realised that anything that's a legal argument to "dict.update" will
>> work. I'll fix the function description in the PEP (and the docs patch as 
>> well).
> 
> I'm not sure that's a good idea -- you'll never be able to switch to a
> different implementation then.

Good point - I'll change the wording so that (officially, at least) it has to 
be a dictionary.


[_read_compiled_file() error handling]
> Also, *perhaps* it makes more sense to return None instead of raising
> ValueError? Since you're always catching it? (Or are you?)

I've changed this in my local copy. That provides a means for dealing with 
marshal, too - catching any Exception from marshal.load and convert it to 
returning None.

This approach loses some details on what exactly was wrong with the file, but 
that doesn't seem like a big issue (and it cleans up some of the other code).


[run_module() error handling]
> OK. But a loader could return None from get_code() -- do you check for
> that? (I don't have the source handy here.)

The current version on SF doesn't check it, but I just updated my local copy 
to fix that.


[run_module() interaction with import]
> What happens when you execute "foo.py" as __main__ and then (perhaps
> indirectly) something does "import foo"? Does a second copy of foo.py
> get loaded by the regular loader?

Yes - this is the same as if foo.py was run directly from the command line via 
its filename.


[YAGNI and 6 public functions where only 1 has a demonstrated use case]
> I do wonder if runpy.py isn't getting a bit over-engineered -- it
> seems a lot of the functionality isn't actually necessary to implement
> -m foo.bar, and the usefulness in other circumstances is as yet
> unproven. What do you think of taking a dose of YAGNI here?
> (Especially since I notice that most of the public APIs are very thin
> layers over exec or execfile -- people can just use those directly.)

I had a look at pdb and profile, and the runpy functions really wouldn't help 
with either of those. Since I don't have any convincing use cases, I'll demote 
run_code and run_module_code to be private helper functions and remove the 
three run*file methods (I might throw those three up on ASPN as a cookbook 
recipe instead).

That leaves the public API containing only run_module, which is all -m really 
needs.

[thread safety and the import lock]
>> Another problem that occurred to me is that the module isn't thread safe at
>> the moment. The PEP 302 emulation isn't protected by the import lock, and the
>> changes to sys.argv in run_module_code will be visible across threads (and 
>> may
>> clobber each other or the original if multiple threads invoke the function).
> 
> Another reason to consider cutting it down to only what's needed by
> -m; -m doesn't need thread-safety (I think).

Yeah, thread-safety is only an issue if invoking runpy.run_module from 
threaded Python code. However, I think this is one of those nasty threading 
problems where it will work for 99.9% of cases and produce intractable bugs 
for the remaining 0.1%. If an extra try-finally block can categorically rule 
out those kinds of problems, then I think it's nicer to include it.

Cheers,
Nick.

-- 
Nick Coghlan   |   [EMAIL PROTECTED]   |   Brisbane, Australia
---
 http://www.boredomandlaziness.org
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The decorator(s) module

[Georg Brandl]

Georg Brandl wrote:
> Hi,
> 
> it has been proposed before, but there was no conclusive answer last time:
> is there any chance for 2.5 to include commonly used decorators in a module?

No interest at all?

Georg

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposal: defaultdict

[Adam Olsen]

On 2/16/06, Guido van Rossum <[EMAIL PROTECTED]> wrote:
> A bunch of Googlers were discussing the best way of doing the
> following (a common idiom when maintaining a dict of lists of values
> relating to a key, sometimes called a multimap):
>
>   if key not in d: d[key] = []
>   d[key].append(value)
>
> An alternative way to spell this uses setdefault(), but it's not very 
> readable:
>
>   d.setdefault(key, []).append(value)

I'd like to see it done passing a factory function (and with a better name):

d.getorset(key, list).append(value)

The name is slightly odd but it is effective.  Plus it avoids creating
a new class when a slight tweak to an existing one will do.


> Over lunch with Alex Martelli, he proposed that a subclass of dict
> with this behavior (but implemented in C) would be a good addition to
> the language. It looks like it wouldn't be hard to implement. It could
> be a builtin named defaultdict. The first, required, argument to the
> constructor should be the default value. Remaining arguments (even
> keyword args) are passed unchanged to the dict constructor.

-1 (atleast until you can explain why that's better than .getorset())

--
Adam Olsen, aka Rhamphoryncus
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposal: defaultdict

[Nick Coghlan]

Phillip J. Eby wrote:
> At 10:10 AM 02/17/2006 +0100, Georg Brandl wrote:
>> Guido van Rossum wrote:
>>
>>>   d = DefaultDict([])
>>>
>>> can be written as simply
>>>
>>>   d[key].append(value)
>>> Feedback?
>> Probably a good idea, has been proposed multiple times on clpy.
>> One good thing would be to be able to specify either a default value
>> or a factory function.
> 
> +1 on factory function, e.g. "DefaultDict(list)".  A default value isn't 
> very useful, because for immutable defaults, setdefault() works well 
> enough.  If what you want is a copy of some starting object, you can always 
> do something like DefaultDict({1:2,3:4}.copy).

+1 here, too (for permitting a factory function only).

This doesn't really limit usage, as you can still supply 
DefaultDict(partial(copy, x)) or DefaultDict(partial(deepcopy, x)), or (heaven 
forbid) a lambda expression. . .

As others have mentioned, the basic types are all easy, since the typename can 
be used directly.

+1 on supplying that factory function to the constructor, too (the default 
value is a fundamental part of the defaultdict). That is, I'd prefer:

   d = defaultdict(func)
   # The defaultdict is fully defined, but not yet populated
   d.update(init_values)

over:

   d = defaultdict(init_values)
   # The defaultdict is partially populated, but not yet fully defined!
   d.default(func)

That is, something that is the same the normal dict except for:

 def __init__(self, default):
 self.default = default

 def __getitem__(self, key):
 return self.get(key, self.default())

Considering some of Raymond's questions in light of the above
> * implications of a __getitem__ succeeding while get(value, x) returns x 
> (possibly different from the overall default)
> * implications of a __getitem__ succeeding while __contains__ would fail

These behaviours seem reasonable for a default dictionary - "containment" is 
based on whether or not the key actually exists in the dictionary as it 
currently stands, and the default is really a "default default" that can be 
overridden using 'get'.

> * whether to add this to the collections module (I would say yes)
> * whether to allow default functions as well as default values (so you could 
> instantiate a new default list)

My preference is for factory functions only, to eliminate ambiguity.

# bag like behavior
dd = collections.default_dict(int)
for elem in collection:
 dd[elem] += 1

# setdefault-like behavior
dd = collections.default_dict(list)
for page_number, page in enumerate(book):
 for word in page.split():
 dd[word].append(word)



-- 
Nick Coghlan   |   [EMAIL PROTECTED]   |   Brisbane, Australia
---
 http://www.boredomandlaziness.org
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Stateful codecs [Was: str object going in Py3K]

[Walter Dörwald]

M.-A. Lemburg wrote:

> Walter Dörwald wrote:
>> Guido van Rossum wrote:
>>
>>> [...]
>>> Years ago I wrote a prototype; checkout sandbox/sio/.
>> However sio.DecodingInputFilter and sio.EncodingOutputFilter don't work
>> for encodings that need state (e.g. when reading/writing UTF-16).
>> Switching to stateful encoders/decoders isn't so easy, because the
>> stateful codecs require a stream-API, which brings in a whole bunch of
>> other functionality (readline() etc.), which we'd probably like to keep
>> separate. I have a patch (http://bugs.python.org/1101097) that should
>> fix this problem (at least for all codecs derived from
>> codecs.StreamReader/codecs.StreamWriter). Additionally it would make
>> stateful codecs more useful in the context for iterators/generators.
>>
>> I'd like this patch to go into 2.5.
> 
> The patch as-is won't go into 2.5. It's simply the wrong approach:
> StreamReaders and -Writers work on streams (hence the name). It
> doesn't make sense adding functionality to side-step this behavior,
> since it undermines the design.

I agree that using a StreamWriter without a stream somehow feels wrong.

> Like I suggested in the patch discussion, such functionality could
> be factored out of the implementations of StreamReaders/Writers
> and put into new StatefulEncoder/Decoder classes, the objects of
> which then get used by StreamReader/Writer.
> 
> In addition to that we could extend the codec registry to also
> maintain slots for the stateful encoders and decoders, if needed.

We *have* to do it like this otherwise there would be no way to get a 
StatefulEncoder/Decoder from an encoding name.

Does this mean that codecs.lookup() would have to return a 6-tuple? But 
this would break if someone uses codecs.lookup("foo")[-1]. So maybe 
codecs.lookup() should return an instance of a subclass of tuple which 
has the StatefulEncoder/Decoder as attributes. But then codecs.lookup() 
must be able to handle old 4-tuples returned by old search functions and 
update those to the new 6-tuples. (But we could drop this again after 
several releases, once all third party codecs are updated).

Bye,
Walter Dörwald

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposal: defaultdict

[Nick Coghlan]

Adam Olsen wrote:
>> Over lunch with Alex Martelli, he proposed that a subclass of dict
>> with this behavior (but implemented in C) would be a good addition to
>> the language. It looks like it wouldn't be hard to implement. It could
>> be a builtin named defaultdict. The first, required, argument to the
>> constructor should be the default value. Remaining arguments (even
>> keyword args) are passed unchanged to the dict constructor.
> 
> -1 (atleast until you can explain why that's better than .getorset())

Because the "default default" is a fundamental characteristic of the default 
dictionary (meaning it works with normal indexing syntax), whereas "getorset" 
makes it a characteristic of the method call.

Besides, if there are going to be any method changes on normal dicts, I'd 
rather see a boolean third argument "set" to the get method.

That is (for a normal dict):

   def get(self, key, *args):
   set = False
   no_default = False
   if len(args) == 2:
   default, set = args
   elif args:
   default, = args
   else:
   no_default = True

   if key in self:
   return self[key]
   if no_default:
   raise KeyError(repr(key))
   if set:
   self[key] = default
   return default

Using Guido's original example:

   d.get(key, [], True).append(value)

I don't really think this is a replacement for defaultdict, though.

Cheers,
Nick.

-- 
Nick Coghlan   |   [EMAIL PROTECTED]   |   Brisbane, Australia
---
 http://www.boredomandlaziness.org
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposal: defaultdict

[Fredrik Lundh]

Nick Coghlan wrote:

> Using Guido's original example:
>
>   d.get(key, [], True).append(value)

hmm.  are you sure you didn't just reinvent setdefault ?

 



___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Stateful codecs [Was: str object going in Py3K]

[M.-A. Lemburg]

Walter Dörwald wrote:
> M.-A. Lemburg wrote:
> 
>> Walter Dörwald wrote:
>>> Guido van Rossum wrote:
>>>
 [...]
 Years ago I wrote a prototype; checkout sandbox/sio/.
>>> However sio.DecodingInputFilter and sio.EncodingOutputFilter don't work
>>> for encodings that need state (e.g. when reading/writing UTF-16).
>>> Switching to stateful encoders/decoders isn't so easy, because the
>>> stateful codecs require a stream-API, which brings in a whole bunch of
>>> other functionality (readline() etc.), which we'd probably like to keep
>>> separate. I have a patch (http://bugs.python.org/1101097) that should
>>> fix this problem (at least for all codecs derived from
>>> codecs.StreamReader/codecs.StreamWriter). Additionally it would make
>>> stateful codecs more useful in the context for iterators/generators.
>>>
>>> I'd like this patch to go into 2.5.
>>
>> The patch as-is won't go into 2.5. It's simply the wrong approach:
>> StreamReaders and -Writers work on streams (hence the name). It
>> doesn't make sense adding functionality to side-step this behavior,
>> since it undermines the design.
> 
> I agree that using a StreamWriter without a stream somehow feels wrong.
> 
>> Like I suggested in the patch discussion, such functionality could
>> be factored out of the implementations of StreamReaders/Writers
>> and put into new StatefulEncoder/Decoder classes, the objects of
>> which then get used by StreamReader/Writer.
>>
>> In addition to that we could extend the codec registry to also
>> maintain slots for the stateful encoders and decoders, if needed.
> 
> We *have* to do it like this otherwise there would be no way to get a
> StatefulEncoder/Decoder from an encoding name.
>
> Does this mean that codecs.lookup() would have to return a 6-tuple? 
> But this would break if someone uses codecs.lookup("foo")[-1].

Right; though I'd much rather see that people use the direct
codecs module lookup APIs:

getencoder(), getdecoder(), getreader() and getwriter()

instead of using codecs.lookup() directly.

> So maybe
> codecs.lookup() should return an instance of a subclass of tuple which
> has the StatefulEncoder/Decoder as attributes. But then codecs.lookup()
> must be able to handle old 4-tuples returned by old search functions and
> update those to the new 6-tuples. (But we could drop this again after
> several releases, once all third party codecs are updated).

This was a design error: I should have not made
codecs.lookup() a documented function.

I'd suggest we keep codecs.lookup() the way it is and
instead add new functions to the codecs module, e.g.
codecs.getencoderobject() and codecs.getdecoderobject().

Changing the codec registration is not much of a problem:
we could simply allow 6-tuples to be passed into the
registry.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Feb 17 2006)
>>> Python/Zope Consulting and Support ...http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/


::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! 
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposal: defaultdict

[skip]

Guido> Over lunch with Alex Martelli, he proposed that a subclass of
Guido> dict with this behavior (but implemented in C) would be a good
Guido> addition to the language.

Instead, why not define setdefault() the way it should have been done in the
first place?  When you create a dict it has the current behavior.  If you
then call its setdefault() method that becomes the default value for missing
keys.

d = {'a': 1}'
d['b']  # raises KeyError
d.get('c')  # evaluates to None
d.setdefault(42)
d['b']  # evaluates to 42
d.get('c')  # evaluates to 42

For symmetry, setdefault() should probably be undoable: deldefault(),
removedefault(), nodefault(), default_free(), whatever.

The only question in my mind is whether or not getting a non-existent value
under the influence of a given default value should stick that value in the
dictionary or not.

down-with-more-builtins-ly, y'rs,

Skip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposal: defaultdict

[Nick Coghlan]

Fredrik Lundh wrote:
> Nick Coghlan wrote:
> 
>> Using Guido's original example:
>>
>>   d.get(key, [], True).append(value)
> 
> hmm.  are you sure you didn't just reinvent setdefault ?

I'm reasonably sure I copied it on purpose, only with a name that isn't 100% 
misleading as to what it does ;)

I think collections.defaultdict is a better approach, though.

Cheers,
Nick.

-- 
Nick Coghlan   |   [EMAIL PROTECTED]   |   Brisbane, Australia
---
 http://www.boredomandlaziness.org
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposal: defaultdict

[Georg Brandl]

[EMAIL PROTECTED] wrote:
> Guido> Over lunch with Alex Martelli, he proposed that a subclass of
> Guido> dict with this behavior (but implemented in C) would be a good
> Guido> addition to the language.
> 
> Instead, why not define setdefault() the way it should have been done in the
> first place?  When you create a dict it has the current behavior.  If you
> then call its setdefault() method that becomes the default value for missing
> keys.

That puts it off until 3.0.

>From what I read I think defaultdict won't become builtin anyway.

Georg

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 2.5 PEP

[Fred L. Drake, Jr.]

On Thursday 16 February 2006 17:06, Martin v. Löwis wrote:
 > I'm still unhappy with that change, and still nobody has told me how to
 > maintain PyXML so that it can continue to work both for 2.5 and for 2.4.

Martin,

I do intend to write a proper response for you, but have been massively 
swamped.  python-dev topics occaissionally pop up for me, but time has been 
too limited to get back to the important items, like this one.


  -Fred

-- 
Fred L. Drake, Jr.   
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Stateful codecs [Was: str object going in Py3K]

[Walter Dörwald]

M.-A. Lemburg wrote:
> Walter Dörwald wrote:
>> M.-A. Lemburg wrote:
>>
>>> [...]
>>> Like I suggested in the patch discussion, such functionality could
>>> be factored out of the implementations of StreamReaders/Writers
>>> and put into new StatefulEncoder/Decoder classes, the objects of
>>> which then get used by StreamReader/Writer.
>>>
>>> In addition to that we could extend the codec registry to also
>>> maintain slots for the stateful encoders and decoders, if needed.
>> We *have* to do it like this otherwise there would be no way to get a
>> StatefulEncoder/Decoder from an encoding name.
>>
>> Does this mean that codecs.lookup() would have to return a 6-tuple? 
>> But this would break if someone uses codecs.lookup("foo")[-1].
> 
> Right; though I'd much rather see that people use the direct
> codecs module lookup APIs:
> 
> getencoder(), getdecoder(), getreader() and getwriter()
> 
> instead of using codecs.lookup() directly.

OK.

>> So maybe
>> codecs.lookup() should return an instance of a subclass of tuple which
>> has the StatefulEncoder/Decoder as attributes. But then codecs.lookup()
>> must be able to handle old 4-tuples returned by old search functions and
>> update those to the new 6-tuples. (But we could drop this again after
>> several releases, once all third party codecs are updated).
> 
> This was a design error: I should have not made
> codecs.lookup() a documented function.
> 
> I'd suggest we keep codecs.lookup() the way it is and
> instead add new functions to the codecs module, e.g.
> codecs.getencoderobject() and codecs.getdecoderobject().
> 
> Changing the codec registration is not much of a problem:
> we could simply allow 6-tuples to be passed into the
> registry.

OK, so codecs.lookup() returns 4-tuples, but the registry stores 
6-tuples and the search functions must return 6-tuples. And we add 
codecs.getencoderobject() and codecs.getdecoderobject() as well as new 
classes codecs.StatefulEncoder and codecs.StatefulDecoder. What about 
old search functions that return 4-tuples?

Bye,
Walter Dörwald

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] bytes type discussion

[Bengt Richter]

On Fri, 17 Feb 2006 00:43:50 -0500, Steve Holden <[EMAIL PROTECTED]> wrote:

>Fredrik Lundh wrote:
>> Barry Warsaw wrote:
>> 
>> 
>>>We know at least there will never be a 2.10, so I think we still have
>>>time.
>> 
>> 
>> because there's no way to count to 10 if you only have one digit?
>> 
>> we used to think that back when the gas price was just below 10 SEK/L,
>> but they found a way...
>> 
>IIRC Guido is on record as saying "There will be no Python 2.10 because 
>I hate the ambiguity of double-digit minor release numbers", or words to 
>that effect.
>
Hex?

Regards,
Bengt Richter

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Stateful codecs [Was: str object going in Py3K]

[M.-A. Lemburg]

Walter Dörwald wrote:
> M.-A. Lemburg wrote:
>> Walter Dörwald wrote:
>>> M.-A. Lemburg wrote:
>>>
 [...]
 Like I suggested in the patch discussion, such functionality could
 be factored out of the implementations of StreamReaders/Writers
 and put into new StatefulEncoder/Decoder classes, the objects of
 which then get used by StreamReader/Writer.

 In addition to that we could extend the codec registry to also
 maintain slots for the stateful encoders and decoders, if needed.
>>> We *have* to do it like this otherwise there would be no way to get a
>>> StatefulEncoder/Decoder from an encoding name.
>>>
>>> Does this mean that codecs.lookup() would have to return a 6-tuple?
>>> But this would break if someone uses codecs.lookup("foo")[-1].
>>
>> Right; though I'd much rather see that people use the direct
>> codecs module lookup APIs:
>>
>> getencoder(), getdecoder(), getreader() and getwriter()
>>
>> instead of using codecs.lookup() directly.
> 
> OK.
> 
>>> So maybe
>>> codecs.lookup() should return an instance of a subclass of tuple which
>>> has the StatefulEncoder/Decoder as attributes. But then codecs.lookup()
>>> must be able to handle old 4-tuples returned by old search functions and
>>> update those to the new 6-tuples. (But we could drop this again after
>>> several releases, once all third party codecs are updated).
>>
>> This was a design error: I should have not made
>> codecs.lookup() a documented function.
>>
>> I'd suggest we keep codecs.lookup() the way it is and
>> instead add new functions to the codecs module, e.g.
>> codecs.getencoderobject() and codecs.getdecoderobject().
>>
>> Changing the codec registration is not much of a problem:
>> we could simply allow 6-tuples to be passed into the
>> registry.
> 
> OK, so codecs.lookup() returns 4-tuples, but the registry stores
> 6-tuples and the search functions must return 6-tuples. And we add
> codecs.getencoderobject() and codecs.getdecoderobject() as well as new
> classes codecs.StatefulEncoder and codecs.StatefulDecoder. What about
> old search functions that return 4-tuples?

The registry should then simply set the missing entries to None
and the getencoderobject()/getdecoderobject() would then have
to raise an error.

Perhaps we should also deprecate codecs.lookup() in Py 2.5 ?!

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Feb 17 2006)
>>> Python/Zope Consulting and Support ...http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/


::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! 
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposal: defaultdict

[Jack Diederich]

On Thu, Feb 16, 2006 at 01:11:49PM -0800, Guido van Rossum wrote:
[snip]
> Google has an internal data type called a DefaultDict which gets
> passed a default value upon construction. Its __getitem__ method,
> instead of raising KeyError, inserts a shallow copy (!) of the given
> default value into the dict when the value is not found. So the above
> code, after
> 
>   d = DefaultDict([])
> 
> can be written as simply
> 
>   d[key].append(value)
> 
> Note that of all the possible semantics for __getitem__ that could
> have produced similar results (e.g. not inserting the default in the
> underlying dict, or not copying the default value), the chosen
> semantics are the only ones that makes this example work.

Having __getitem__ insert the returned default value allows it to
work with a larger variety of classes.  My own ForgivingDict does not
do this and works fine for ints and lists but not much else.

fd = ForgivingDict(list)
fd[key] += [val] # extends the list and does a __setitem__

The += operator isn't useful for dicts.

How can you make a defaultdict with a defaultdict as the default?
My head asploded when I tried it with the constructor arg.
It does seem possible with the 'd.default = func' syntax

# empty defaultdict constructor
d = defaultdict()
d.default = d
tree = defaultdict()
tree.default = d.copy

-jackdied
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Please comment on PEP 357 -- adding nb_index slot to PyNumberMethods

[Armin Rigo]

Hi Travis,

On Tue, Feb 14, 2006 at 08:41:19PM -0700, Travis E. Oliphant wrote:
> 2) The __index__ special method will have the signature
> 
>def __index__(self):
>return obj
>
>Where obj must be either an int or a long or another object that has 
> the
>__index__ special method (but not self).

The "anything but not self" rule is not consistent with any other
special method's behavior.  IMHO we should just do the same as
__nonzero__():

* __nonzero__(x) must return exactly a bool or an int.

This ensures that there is no infinite loop in C created by a
__nonzero__ that returns something that has a further __nonzero__
method.

The rule that the PEP proposes for __index__ (returns anything but not
'self') is not useful, because you can still get infinite loops (you
just have to work slightly harder, and even not much).  We should just
say that __index__ must return an int or a long.


A bientot,

Armin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposal: defaultdict

[CM]

+1It's about time!- COn 2/16/06, Guido van Rossum <[EMAIL PROTECTED]> wrote:
A bunch of Googlers were discussing the best way of doing thefollowing (a common idiom when maintaining a dict of lists of valuesrelating to a key, sometimes called a multimap):  if key not in d: d[key] = []
  d[key].append(value)An alternative way to spell this uses setdefault(), but it's not very readable:  d.setdefault(key, []).append(value)and it also suffers from creating an unnecessary list instance.
(Timings were inconclusive; the approaches are within 5-10% of eachother in speed.)My conclusion is that setdefault() is a failure -- it was awell-intentioned construct, but doesn't actually create more readable
code.Google has an internal data type called a DefaultDict which getspassed a default value upon construction. Its __getitem__ method,instead of raising KeyError, inserts a shallow copy (!) of the given
default value into the dict when the value is not found. So the abovecode, after  d = DefaultDict([])can be written as simply  d[key].append(value)Note that of all the possible semantics for __getitem__ that could
have produced similar results (e.g. not inserting the default in theunderlying dict, or not copying the default value), the chosensemantics are the only ones that makes this example work.Over lunch with Alex Martelli, he proposed that a subclass of dict
with this behavior (but implemented in C) would be a good addition tothe language. It looks like it wouldn't be hard to implement. It couldbe a builtin named defaultdict. The first, required, argument to the
constructor should be the default value. Remaining arguments (evenkeyword args) are passed unchanged to the dict constructor.Some more design subtleties:- "key in d" still returns False if the key isn't there
- "d.get(key)" still returns None if the key isn't there- "d.default" should be a read-only attribute giving the default valueFeedback?Guido van Rossum (home page: 
http://www.python.org/~guido/)___Python-Dev mailing listPython-Dev@python.org
http://mail.python.org/mailman/listinfo/python-devUnsubscribe: http://mail.python.org/mailman/options/python-dev/monpublic%40gmail.com
-- "A programmer learning programming from Perl is like a chemistry student learning the definition of 'exothermic' with dynamite." - evilpenguin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] bytes type discussion

[Fredrik Lundh]

Bengt Richter wrote:

> >> because there's no way to count to 10 if you only have one digit?
> >>
> >> we used to think that back when the gas price was just below 10 SEK/L,
> >> but they found a way...
> >>
> >IIRC Guido is on record as saying "There will be no Python 2.10 because
> >I hate the ambiguity of double-digit minor release numbers", or words to
> >that effect.
>
> Hex?

or roman numbers.

I've payed X.35 SEK/L for gas...





___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Please comment on PEP 357 -- adding nb_index slot to PyNumberMethods

[Thomas Wouters]

On Fri, Feb 17, 2006 at 05:29:32PM +0100, Armin Rigo wrote:
> >Where obj must be either an int or a long or another object that has 
> > the
> >__index__ special method (but not self).

> The "anything but not self" rule is not consistent with any other
> special method's behavior.  IMHO we should just do the same as
> __nonzero__():

> * __nonzero__(x) must return exactly a bool or an int.

Yes, very much so. And in case people worry that this makes wrapping
objects harder: proxy objects (for instance) would do 'return
operator.index(self._real)'.

-- 
Thomas Wouters <[EMAIL PROTECTED]>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Stateful codecs [Was: str object going in Py3K]

[Walter Dörwald]

M.-A. Lemburg wrote:
> Walter Dörwald wrote:
>> M.-A. Lemburg wrote:
>>> Walter Dörwald wrote:
  [...]
 So maybe
 codecs.lookup() should return an instance of a subclass of tuple which
 has the StatefulEncoder/Decoder as attributes. But then codecs.lookup()
 must be able to handle old 4-tuples returned by old search functions and
 update those to the new 6-tuples. (But we could drop this again after
 several releases, once all third party codecs are updated).
>>> This was a design error: I should have not made
>>> codecs.lookup() a documented function.
>>>
>>> I'd suggest we keep codecs.lookup() the way it is and
>>> instead add new functions to the codecs module, e.g.
>>> codecs.getencoderobject() and codecs.getdecoderobject().
>>>
>>> Changing the codec registration is not much of a problem:
>>> we could simply allow 6-tuples to be passed into the
>>> registry.
>> OK, so codecs.lookup() returns 4-tuples, but the registry stores
>> 6-tuples and the search functions must return 6-tuples. And we add
>> codecs.getencoderobject() and codecs.getdecoderobject() as well as new
>> classes codecs.StatefulEncoder and codecs.StatefulDecoder. What about
>> old search functions that return 4-tuples?
> 
> The registry should then simply set the missing entries to None
> and the getencoderobject()/getdecoderobject() would then have
> to raise an error.

Sounds simple enough and we don't loose backwards compatibility.

> Perhaps we should also deprecate codecs.lookup() in Py 2.5 ?!

+1, but I'd like to have a replacement for this, i.e. a function that 
returns all info the registry has about an encoding:

1. Name
2. Encoder function
3. Decoder function
4. Stateful encoder factory
5. Stateful decoder factory
6. Stream writer factory
7. Stream reader factory

and if this is an object with attributes, we won't have any problems if 
we extend it in the future.

BTW, if we change the API, can we fix the return value of the stateless 
functions? As the stateless function always encodes/decodes the complete 
string, returning the length of the string doesn't make sense. 
codecs.getencoder() and codecs.getdecoder() would have to continue to 
return the old variant of the functions, but 
codecs.getinfo("latin-1").encoder would be the new encoding function.

Bye,
Walter Dörwald
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Copying zlib compression objects

[Chris AtLee]

I'm writing a program in python that creates tar files of a certain  maximum size (to fit onto CD/DVD).  One of the problems I'm running  into is that when using compression, it's pretty much impossible to 
 determine if a file, once added to an archive, will cause the archive  size to exceed the maximum size.  I believe that to do this properly, you need to copy the state of tar 
 file (basically the current file offset as well as the state of the  compression object), then add the file.  If the new size of the archive  exceeds the maximum, you need to restore the original state. 
 
The critical part is being able to copy the compression object.  Without compression it is trivial to determine if a given file will  "fit" inside the archive.  When using compression, the compression 
 ratio of a file depends partially on all the data that has been  compressed prior to it.  The current implementation in the standard library does not allow you 
 to copy these compression objects in a useful way, so I've made some  minor modifications (patch attached) to the standard 2.4.2 library:   - Add copy() method to zlib compression object.  This returns a new 
 compression object with the same internal state.  I named it copy() to   keep it consistent with things like sha.copy().   - Add snapshot() / restore() methods to GzipFile and TarFile.  These  work only in write mode.  snapshot() returns a state object.  Passing 
 in this state object to restore() will restore the state of the   GzipFile / TarFile to the state represented by the object.  Future work: 
 - Decompression objects could use a copy() method too  - Add support for copying bzip2 compression objects  Although this patch isn't complete, does this seem like a good approach? 
 Cheers,  Chris  

diff -ur Python-2.4.2.orig/Lib/gzip.py Python-2.4.2/Lib/gzip.py
--- Python-2.4.2.orig/Lib/gzip.py	2005-06-09 10:22:07.0 -0400
+++ Python-2.4.2/Lib/gzip.py	2006-02-14 13:12:29.0 -0500
@@ -433,6 +433,17 @@
 else:
 raise StopIteration
 
+def snapshot(self):
+if self.mode == READ:
+raise IOError("Can't create a snapshot in READ mode")
+return (self.size, self.crc, self.fileobj.tell(), self.offset, self.compress.copy())
+
+def restore(self, s):
+if self.mode == READ:
+raise IOError("Can't restore a snapshot in READ mode")
+self.size, self.crc, offset, self.offset, self.compress = s
+self.fileobj.seek(offset)
+self.fileobj.truncate()
 
 def _test():
 # Act like gzip; with -d, act like gunzip.
diff -ur Python-2.4.2.orig/Lib/tarfile.py Python-2.4.2/Lib/tarfile.py
--- Python-2.4.2.orig/Lib/tarfile.py	2005-08-27 06:08:21.0 -0400
+++ Python-2.4.2/Lib/tarfile.py	2006-02-15 12:05:48.0 -0500
@@ -1825,6 +1825,28 @@
 """
 if level <= self.debug:
 print >> sys.stderr, msg
+
+def snapshot(self):
+"""Save the current state of the tarfile
+"""
+self._check("_aw")
+if hasattr(self.fileobj, "snapshot"):
+return self.fileobj.snapshot(), self.offset, self.members[:], self.inodes.copy()
+else:
+return self.fileobj.tell(), self.offset, self.members[:], self.inodes.copy()
+
+def restore(self, s):
+"""Restore the state of the tarfile from a previous snapshot
+"""
+self._check("_aw")
+if hasattr(self.fileobj, "restore"):
+snapshot, self.offset, self.members, self.inodes = s
+self.fileobj.restore(snapshot)
+else:
+offset, self.offset, self.members, self.inodes = s
+self.fileobj.seek(offset)
+self.fileobj.truncate()
+
 # class TarFile
 
 class TarIter:
diff -ur Python-2.4.2.orig/Modules/zlibmodule.c Python-2.4.2/Modules/zlibmodule.c
--- Python-2.4.2.orig/Modules/zlibmodule.c	2004-12-28 15:12:31.0 -0500
+++ Python-2.4.2/Modules/zlibmodule.c	2006-02-14 14:05:35.0 -0500
@@ -653,6 +653,36 @@
 return RetVal;
 }
 
+PyDoc_STRVAR(comp_copy__doc__,
+"copy() -- Return a copy of the compression object.");
+
+static PyObject *
+PyZlib_copy(compobject *self, PyObject *args)
+{
+compobject *retval;
+
+retval = newcompobject(&Comptype);
+
+/* Copy the zstream state */
+/* TODO: Are the ENTER / LEAVE needed? */
+ENTER_ZLIB
+deflateCopy(&retval->zst, &self->zst);
+LEAVE_ZLIB
+
+/* Make references to the original unused_data and unconsumed_tail
+ * They're not used by compression objects so we don't have to do
+ * anything special here */
+retval->unused_data = self->unused_data;
+retval->unconsumed_tail = self->unconsumed_tail;
+Py_INCREF(retval->unused_data);
+Py_INCREF(retval->unconsumed_tail);
+
+/* Mark it as being initialized */
+retval->is_initialised = 1;
+
+return (PyObject*)retval;
+}
+
 PyDoc_STRVAR(decomp_flush__doc__,
 "flush() -- Return a string containing any remaining decompressed dat

Re: [Python-Dev] bytes.from_hex() [Was: PEP 332 revival in coordination with pep 349?]

[Bengt Richter]

On Fri, 17 Feb 2006 00:33:49 +0100, =?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?= 
<[EMAIL PROTECTED]> wrote:

>Josiah Carlson wrote:
>> I would agree that zip is questionable, but 'uu', 'rot13', perhaps 'hex',
>> and likely a few others that the two of you may be arguing against
>> should stay as encodings, because strictly speaking, they are defined as
>> encodings of data.  They may not be encodings of _unicode_ data, but
>> that doesn't mean that they aren't useful encodings for other kinds of
>> data, some text, some binary, ...
>
>To support them, the bytes type would have to gain a .encode method,
>and I'm -1 on supporting bytes.encode, or string.decode.
>
>Why is
>
>s.encode("uu")
>
>any better than
>
>binascii.b2a_uu(s)
>
One aspect is that dotted notation method calling is serially composable,
whereas function calls nest, and you have to find and read from the innermost,
which gets hard quickly unless you use multiline formatting. But even then
you can't read top to bottom as processing order.

If we had a general serial composition syntax for function calls
something like unix piping (which is a big part of the power of unix shells IMO)
we could make the choice of appropriate composition semantics better.

Decorators already compose functions in a limited way, but processing
order would read like forth horizontally. Maybe '->' ? How about

foo(x, y) -> bar() -> baz(z)

as as sugar for

baz.__get__(bar.__get__(foo(x, y))())(z)

? (Hope I got that right ;-)

I.e., you'd have self-like args to receive results from upstream. E.g.,

 >>> def foo(x, y): return 'foo(%s, %s)'%(x,y)
 ...
 >>> def bar(stream): return 'bar(%s)'%stream
 ...
 >>> def baz(stream, z): return 'baz(%s, %s)'%(stream,z)
 ...
 >>> x = 'ex'; y='wye'; z='zed'
 >>> baz.__get__(bar.__get__(foo(x, y))())(z)
 'baz(bar(foo(ex, wye)), zed)'

Regards,
Bengt Richter

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 2.5 release schedule

[Armin Rigo]

Hi,

On Tue, Feb 14, 2006 at 09:24:57PM -0800, Neal Norwitz wrote:
> http://www.python.org/peps/pep-0356.html

There is at least one SF bug, namely "#1333982 Bugs of the new AST
compiler", that in my humble opinion absolutely needs to be fixed before
the release, even though I won't hide that I have no intention of fixing
it myself.  Should I raise the issue here in python-dev, and see if we
agree that it is critical?

(Sorry if I should know about the procedure.  Does it then go in the
PEP's Planned Features list?)


A bientot,

Armin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Adventures with ASTs - Inline Lambda

[skip]

Steve> It appears to hang together, but I'm not sure I see how it
Steve> overcomes objections to lambda by replacing it with another
Steve> keyword.

Well, it does replace it with a word which has meaning in common English.

FWIW, I would require the parens around the arguments and avoid the
ambiguity altogether.

Skip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposal: defaultdict

[Terry Reedy]

"Fredrik Lundh" <[EMAIL PROTECTED]> wrote in message 
news:[EMAIL PROTECTED]
> Raymond Hettinger wrote:
>
>> I would like to add something like this to the collections module, but a 
>> PEP is
>> probably needed to deal with issues like:
>
> frankly, now that Guido is working 50% on Python, do we really have to 
> use
> the full PEP process also for simple things like this?
>
> I'd say we let the BDFL roam free.

PEPs are useful for question-answering purposes even after approval.  The 
design phase can be cut short by simply posting the approved design doc.




___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 2.5 release schedule

[Jeremy Hylton]

It is critical, but I hadn't seen the bug report.  Feel free to assign
AST bugs to me and assign them a > 5 priority.

Jeremy

On 2/17/06, Armin Rigo <[EMAIL PROTECTED]> wrote:
> Hi,
>
> On Tue, Feb 14, 2006 at 09:24:57PM -0800, Neal Norwitz wrote:
> > http://www.python.org/peps/pep-0356.html
>
> There is at least one SF bug, namely "#1333982 Bugs of the new AST
> compiler", that in my humble opinion absolutely needs to be fixed before
> the release, even though I won't hide that I have no intention of fixing
> it myself.  Should I raise the issue here in python-dev, and see if we
> agree that it is critical?
>
> (Sorry if I should know about the procedure.  Does it then go in the
> PEP's Planned Features list?)
>
>
> A bientot,
>
> Armin
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/jeremy%40alum.mit.edu
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 2.5 release schedule

[Jeremy Hylton]

Actually, it might be easier to assign separate bugs.  A number of the
old bugs appear to have been fixed.  It's hard to track individual
items within a bug report.

Jeremy

On 2/17/06, Jeremy Hylton <[EMAIL PROTECTED]> wrote:
> It is critical, but I hadn't seen the bug report.  Feel free to assign
> AST bugs to me and assign them a > 5 priority.
>
> Jeremy
>
> On 2/17/06, Armin Rigo <[EMAIL PROTECTED]> wrote:
> > Hi,
> >
> > On Tue, Feb 14, 2006 at 09:24:57PM -0800, Neal Norwitz wrote:
> > > http://www.python.org/peps/pep-0356.html
> >
> > There is at least one SF bug, namely "#1333982 Bugs of the new AST
> > compiler", that in my humble opinion absolutely needs to be fixed before
> > the release, even though I won't hide that I have no intention of fixing
> > it myself.  Should I raise the issue here in python-dev, and see if we
> > agree that it is critical?
> >
> > (Sorry if I should know about the procedure.  Does it then go in the
> > PEP's Planned Features list?)
> >
> >
> > A bientot,
> >
> > Armin
> > ___
> > Python-Dev mailing list
> > Python-Dev@python.org
> > http://mail.python.org/mailman/listinfo/python-dev
> > Unsubscribe: 
> > http://mail.python.org/mailman/options/python-dev/jeremy%40alum.mit.edu
> >
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposal: defaultdict

[Ian Bicking]

Raymond Hettinger wrote:
>>>Over lunch with Alex Martelli, he proposed that a subclass of dict
>>>with this behavior (but implemented in C) would be a good addition to
>>>the language
> 
> 
> I would like to add something like this to the collections module, but a PEP 
> is 
> probably needed to deal with issues like:
> 
> * implications of a __getitem__ succeeding while get(value, x) returns x 
> (possibly different from the overall default)
> * implications of a __getitem__ succeeding while __contains__ would fail
> * whether to add this to the collections module (I would say yes)
> * whether to allow default functions as well as default values (so you could 
> instantiate a new default list)
> * comparing all the existing recipes and third-party modules that have 
> already 
> done this
> * evaluating its fitness for common use cases (i.e. bags and dict of lists).

It doesn't seem that useful for bags, assuming we're talking about an 
{object: count} implementation of bags; bags should really have a more 
set-like interface than a dict-like interface.

A dict of lists typically means a multi-valued dict.  In that case it 
seems like x[key_not_found] should return the empty list, as that means 
zero values; even though zero values also means that 
x.has_key(key_not_found) should return False as well.  *but* getting 
x[key_not_found] does not (for a multi-valued dict) mean that suddently 
has_key should return true.  I find the side-effect nature of 
__getitem__ as proposed in default_dict to be rather confusing, and when 
reading code it will very much break my expectations.  I assume that 
attribute access and [] access will not have side effects.  Coming at it 
from that direction, I'm -1, though I'm +1 on dealing with the specific 
use case that started this (x.setdefault(key, []).append(value)).

An implementation targetted specifically at multi-valued dictionaries 
seems like it would be better.  Incidentally, on Web-SIG we've discussed 
wsgiref, and it includes a mutli-values, ordered, case-insensitive 
dictionary.  Such a dictionary(ish) object has clear applicability for 
HTTP headers, but certainly it is something I've used many times 
elsewhere.  In a case-sensitive form it applies to URL variables. 
Really there's several combinations of features, each with different uses.

So we have now...

dicts: unordered, key:value (associative), single-value
sets: unordered, not key:value, single-value
lists: ordered, not key:value, multi-value

We don't have...

bags: unordered, not key:value, multi-value
multi-dict: unordered, key:value, multi-value
ordered-dict: ordered, key:value, single-value
ordered-multi-dict: ordered, key:value, single-value

For all key:value collections, normalized keys can be useful.  (Though 
notably the wsgiref Headers object does not have normalized keys, but 
instead does case-insensitive comparisons.)  I don't know where 
dict-of-dict best fits in here.



-- 
Ian Bicking  /  [EMAIL PROTECTED]  /  http://blog.ianbicking.org
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposal: defaultdict

[Fredrik Lundh]

Terry Reedy wrote:

> > I'd say we let the BDFL roam free.
>
> PEPs are useful for question-answering purposes even after approval.  The
> design phase can be cut short by simply posting the approved design doc.

not for trivialities.  it'll take Guido more time to write a PEP than to
implement the damn thing.  is that really a good use of his time ?

why is python-dev suddenly full of control freaks ?





___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposal: defaultdict

[skip]

>> Also, I think has_key/in should return True if there is a default.

Fredrik> and keys should return all possible key values!

I think keys() and in should reflect reality.  Only when you do something
like

x = d['nonexistent']

or

x = d.get('nonexistent')

should the default value come into play.

Skip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The decorator(s) module

[Ian Bicking]

Georg Brandl wrote:
> Hi,
> 
> it has been proposed before, but there was no conclusive answer last time:
> is there any chance for 2.5 to include commonly used decorators in a module?

One peculiar aspect is that decorators are a programming technique, not 
a particular kind of functionality.  So the module seems kind of funny 
as a result.

> Of course not everything that jumps around should go in, only pretty basic
> stuff that can be widely used.
> 
> Candidates are:
>  - @decorator. This properly wraps up a decorator function to change the
>signature of the new function according to the decorated one's.

Yes, I like this, and it is purely related to "decorators" not anything 
else.  Without this, decorators really hurt introspectability.

>  - @contextmanager, see PEP 343.

This is abstract enough that it doesn't belong anywhere in particular.

>  - @synchronized/@locked/whatever, for thread safety.

Seems better in the threading module.  Plus contexts and with make it 
much less important as a decorator.

>  - @memoize

Also abstract, so I suppose it would make sense.

>  - Others from wiki:PythonDecoratorLibrary and Michele Simionato's decorator
>module at .

redirecting_stdout is better implemented using contexts/with.  @threaded 
(which runs the decorated function in a thread) seems strange to me. 
@blocking seems like it is going into async directions that don't really 
fit in with "decorators" (as a general concept).

I like @tracing, though it doesn't seem like it is really implemented 
there, it's just an example?

> Unfortunately, a @property decorator is impossible...

It already works!  But only if you want a read-only property.  Which is 
actually about 50%+ of the properties I create.  So the status quo is 
not really that bad.


-- 
Ian Bicking  /  [EMAIL PROTECTED]  /  http://blog.ianbicking.org
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The decorator(s) module

[skip]

>> it has been proposed before, but there was no conclusive answer last
>> time: is there any chance for 2.5 to include commonly used decorators
>> in a module?

Georg> No interest at all?

I would think the decorators that allow proper introspection (func_name,
__doc__, etc) should be available, probably in a decorators module.  Beyond
that I'm not sure.  I think it would be better to be conservative.

Skip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The decorator(s) module

[Georg Brandl]

Ian Bicking wrote:

>> Unfortunately, a @property decorator is impossible...
> 
> It already works!  But only if you want a read-only property.  Which is 
> actually about 50%+ of the properties I create.  So the status quo is 
> not really that bad.

I have abused it this way too and felt bad every time.
Kind of like keeping your hat on in the church. :)

Georg

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposal: defaultdict

[Bengt Richter]

On Fri, 17 Feb 2006 08:09:23 +0100, =?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?= 
<[EMAIL PROTECTED]> wrote:

>Guido van Rossum wrote:
>> Feedback?
>
>I would like this to be part of the standard dictionary type,
>rather than being a subtype.
>
>d.setdefault([]) (one argument) should install a default value,
>and d.cleardefault() should remove that setting; d.default
>should be read-only. Alternatively, d.default could be assignable
>and del-able.
I like the latter, but d.default_factory = callable # or None
>
>Also, I think has_key/in should return True if there is a default.
That seems iffy. ISTM potential should not define actual status.

Regards,
Bengt Richter

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposal: defaultdict

[Guido van Rossum]

On 2/16/06, Guido van Rossum <[EMAIL PROTECTED]> wrote:
> Over lunch with Alex Martelli, he proposed that a subclass of dict
> with this behavior (but implemented in C) would be a good addition to
> the language. It looks like it wouldn't be hard to implement. It could
> be a builtin named defaultdict. The first, required, argument to the
> constructor should be the default value. Remaining arguments (even
> keyword args) are passed unchanged to the dict constructor.

Thanks for all the constructive feedback. Here are some responses and
a new proposal.

- Yes, I'd like to kill setdefault() in 3.0 if not sooner.

- It would indeed be nice if this was an optional feature of the
standard dict type.

- I'm ignoring the request for other features (ordering, key
transforms). If you want one of these, write a PEP!

- Many, many people suggested to use a factory function instead of a
default value. This is indeed a much better idea (although slightly
more cumbersome for the simplest cases).

- Some people seem to think that a subclass constructor signature must
match the base class constructor signature. That's not so. The
subclass constructor must just be careful to call the base class
constructor with the correct arguments. Think of the subclass
constructor as a factory function.

- There's a fundamental difference between associating the default
value with the dict object, and associating it with the call. So
proposals to invent a better name/signature for setdefault() don't
compete. (As to one specific such proposal, adding an optional bool as
the 3rd argument to get(), I believe I've explained enough times in
the past that flag-like arguments that always get a constant passed in
at the call site are a bad idea and should usually be refactored into
two separate methods.)

- The inconsistency introduced by __getitem__() returning a value for
keys while get(), __contains__(), and keys() etc. don't show it,
cannot be resolved usefully. You'll just have to live with it.
Modifying get() to do the same thing as __getitem__() doesn't seem
useful -- it just takes away a potentially useful operation.

So here's a new proposal.

Let's add a generic missing-key handling method to the dict class, as
well as a default_factory slot initialized to None. The implementation
is like this (but in C):

def on_missing(self, key):
  if self.default_factory is not None:
value = self.default_factory()
self[key] = value
return value
  raise KeyError(key)

When __getitem__() (and *only* __getitem__()) finds that the requested
key is not present in the dict, it calls self.on_missing(key) and
returns whatever it returns -- or raises whatever it raises.
__getitem__() doesn't need to raise KeyError any more, that's done by
on_missing().

The on_missing() method can be overridden to implement any semantics
you want when the key isn't found: return a value without inserting
it, insert a value without copying it, only do it for certain key
types/values, make the default incorporate the key, etc.

But the default implementation is designed so that we can write

d = {}
d.default_factory = list

to create a dict that inserts a new list whenever a key is not found
in __getitem__(), which is most useful in the original use case:
implementing a multiset so that one can write

d[key].append(value)

to add a new key/value to the multiset without having to handle the
case separately where the key isn't in the dict yet. This also works
for sets instead of lists:

d = {}
d.default_factory = set
...
d[key].add(value)

I went through several iterations to obtain this design; my first
version of on_missing() would just raise KeyError(key), requiring you
to always provide a subclass; this is more minimalistic but less
useful and would probably raise the bar for using the feature to some
extent.

To saev you attempts to simplify this, here are some near-misses I
considered that didn't quite work out:

- def on_missing(self, key):
if self.default_factory is not None:
  return self.default_factory()
raise KeyError(key)

This would require the multiset example to subclass, since
default_factory doesn't see the key so it can't insert it.

- def on_missing(self, key):
if self.default_factory is not None:
  return self.default_factory(key)
raise KeyError(key)

This appears to fix that problem, but now you can't write
"d.default_value = list" since (a) list(key) doesn't return an empty
list, and (b) it also doesn't insert the key into the dict; attempting
to assign a callback function to default_factory that solves these
issues fail because the callback doesn't have access to the dict
instance (unless there's only one).

- Do away with on_missing() and just include its body at the end of
__getitem__(), to be invoked when the key isn't found.

This is less general in case you want different default semantics
(e.g. not inserting the default, or making the default a function of
the key) -- you'd have to override __getitem__() for th

Re: [Python-Dev] Proposal: defaultdict

[Fred L. Drake, Jr.]

On Friday 17 February 2006 14:09, Guido van Rossum wrote:
 > So here's a new proposal.

I like the version you came up with.  It has sufficient functionality to make 
it easy to use, and enough flexibility to be useful in more specialized 
cases.  I'm quite certain it would handle all the cases I've actually dealt 
with where I wanted a variation of a mapping with default values.


  -Fred

-- 
Fred L. Drake, Jr.   
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposal: defaultdict

[Alex Martelli]

On 2/16/06, Guido van Rossum <[EMAIL PROTECTED]> wrote:
> A bunch of Googlers were discussing the best way of doing the
   ...
Wow, what a great discussion!  As you'll recall, I had also mentioned
the "callable factory" as a live possibility, and there seems to be a
strong sentiment in favor of that; not really a "weakness case" for
HOFs, as you feared it might be during the lunchtime discussion.

Out of all I've read here, I like the idea of having a
collections.autodict (a much nicer name than defaultdict, a better
collocation for 2.5 than the builtins). One point I think nobody has
made is that whenever reasonably possible the setting of a callback
(the callable factory here) should include *a and **k to use when
calling back.  So, for example:

ad = collections.autodict(copy.copy, whatever)

would easily cover the use case of Google's DefaultDict (yes, partial
would also cover this use case, but having *a and **k is usefully more
general).  If you're convinced copy.copy is an overwhelmingly popular
use case (I'm not, personally), then this specific idiom might also be
summarized in a classmethod, a la

ad = collections.autodict.by_copy(whatever)


This way, all autodicts would start out empty (and be filled by update
if needed).  An alternative would be to have autodict's ctor have the
same signature as dict's, with a separate .set_initial method to pass
the factory (and *a, **k) -- this way an autodict might start out
populated, but would always start with some default factory, such as
lambda:None I guess. I think the first alternative (autodict always
starts empty, but with a specifically chosen factory [including *a,
**k]) is more useful.


Alex
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Counter proposal: multidict (was: Proposal: defaultdict)

[Ian Bicking]

I really don't like that defaultdict (or a dict extension) means that 
x[not_found] will have noticeable side effects.  This all seems to be a 
roundabout way to address one important use case of a dictionary with 
multiple values for each key, and in the process breaking an important 
quality of good Python code, that attribute and getitem access not have 
noticeable side effects.

So, here's a proposed interface for a new multidict object, borrowing 
some methods from Set but mostly from dict.  Some things that seemed 
particularly questionable to me are marked with ??.

class multidict:

 def __init__([mapping], [**kwargs]):
 """
 Create a multidict:

 multidict() -> new empty multidict
 multidict(mapping) -> equivalent to:
 ob = multidict()
 ob.update(mapping)
 multidict(**kwargs) -> equivalent to:
 ob = multidict()
 ob.update(kwargs)
 """

 def __contains__(key):
 """
 True if ``self[key]`` is true
 """

 def __getitem__(key):
 """
 Returns a list of items associated with the given key.  If
 nothing, then the empty list.

 ??: Is the list mutable, and to what effect?
 """

 def __delitem__(key):
 """
 Removes any instances of key from the dictionary.  Does
 not raise an error if there are no values associated.

 ??: Should this raise a KeyError sometimes?
 """

 def __setitem__(key, value):
 """
 Same as:

 del self[key]
 self.add(key, value)
 """

 def get(key, default=[]):
 """
 Returns a list of items associated with the given key,
 or if that list would be empty it returns default
 """

 def getfirst(key, default=None):
 """
 Equivalent to:
 if key in self:
 return self[key][0]
 else:
 return default
 """

 def add(key, value):
 """
 Adds the value with the given key, so that
 self[key][-1] == value
 """

 def remove(key, value):
 """
 Remove (key, value) from the mapping (raising KeyError if not
 present).
 """

 def discard(key, value):
 """
 Remove like self.remove(key, value), except do not raise
 KeyError if missing.
 """

 def pop(key):
 """
 Removes key and returns the value; returns [] and does nothing
 if the key is not found.
 """

 def keys():
 """
 Returns all the keys which have some associated value.
 """

 def items():
 """
 Returns [(key, value)] for every key/value pair.  Keys that
 have multiple values will be returned as multiple (key, value)
 tuples.
 """

 def __len__():
 """
 Equivalent to len(self.items())

 ??: Not len(self.keys())?
 """

 def update(E, **kwargs):
 """
 if E has iteritems then::

 for k, v in E.iteritems():
 self.add(k, v)

 elif E has keys:

 for k in E:
 self.add(k, E[k])

 else:

 for k, v in E:
 self.add(k, v)

 ??: Should **kwargs be allowed?  If so, should it the values
 be sequences?
 """

 # iteritems, iterkeys, iter, has_key, copy, popitem, values, clear
 # with obvious implementations
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposal: defaultdict

[Guido van Rossum]

On 2/17/06, Alex Martelli <[EMAIL PROTECTED]> wrote:
> On 2/16/06, Guido van Rossum <[EMAIL PROTECTED]> wrote:
> > A bunch of Googlers were discussing the best way of doing the
>...
> Wow, what a great discussion!  As you'll recall, I had also mentioned
> the "callable factory" as a live possibility, and there seems to be a
> strong sentiment in favor of that; not really a "weakness case" for
> HOFs, as you feared it might be during the lunchtime discussion.

:-)

You seem to have missed my revised proposal.

> Out of all I've read here, I like the idea of having a
> collections.autodict (a much nicer name than defaultdict, a better
> collocation for 2.5 than the builtins). One point I think nobody has
> made is that whenever reasonably possible the setting of a callback
> (the callable factory here) should include *a and **k to use when
> calling back.

That's your C/C++ brain talking. :-)

If you need additional data passed to a callback (to be provided at
the time the callback is *set*, not when it is *called*) the customary
approach is to make the callback a parameterless lambda; you can also
use a bound method, etc. There's no need to complicate ever piece of
code that calls a callback with the machinery to store and use
arbirary arguments and keyword arguments.

I forgot to mention in my revised proposal that the API for setting
the default_factory is slightly odd:

  d = {}   # or dict()
  d.default_factory = list

rather than

  d = dict(default_factory=list)

This is of course because we cut off that way when we defined what
arbitrary keyword arguments to the dict constructor would do. My
original proposal solved this by creating a subclass. But there were
several suggestions that this would be fine functionality to add to
the standard dict type -- and then I really don't see any other way to
do this. (Yes, I could have a set_default_factory() method -- but a
simple settable attribute seems more pythonic!)

--
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Counter proposal: multidict (was: Proposal: defaultdict)

[Guido van Rossum]

On 2/17/06, Ian Bicking <[EMAIL PROTECTED]> wrote:
> I really don't like that defaultdict (or a dict extension) means that
> x[not_found] will have noticeable side effects.  This all seems to be a
> roundabout way to address one important use case of a dictionary with
> multiple values for each key, and in the process breaking an important
> quality of good Python code, that attribute and getitem access not have
> noticeable side effects.
>
> So, here's a proposed interface for a new multidict object, borrowing
> some methods from Set but mostly from dict.  Some things that seemed
> particularly questionable to me are marked with ??.

Have you seen my revised proposal (which is indeed an addition to the
standard dict rather than a subclass)?

Your multidict addresses only one use case for the proposed behavior;
what's so special about dicts of lists that they should have special
support? What about dicts of dicts, dicts of sets, dicts of
user-defined objects?

--
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Counter proposal: multidict (was: Proposal: defaultdict)

[Fred L. Drake, Jr.]

On Friday 17 February 2006 14:51, Ian Bicking wrote:
 > This all seems to be a
 > roundabout way to address one important use case of a dictionary with
 > multiple values for each key, 

I think there are use cases that do not involve multiple values per key.  That 
is one place where this commonly comes up, but not the only one.

 > and in the process breaking an important 
 > quality of good Python code, that attribute and getitem access not have
 > noticeable side effects.

I'm not sure that's quite as well-defined or agreed upon as you do.


  -Fred

-- 
Fred L. Drake, Jr.   
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com