Re: [Python-Dev] Tricky way of of creating a generator via a comprehension expression

2017-11-24 Thread Hrvoje Niksic
Guido van Rossum writes: And my mind boggles when considering a generator expression containing yield that is returned from a function. I tried this and cannot say I expected the outcome:    def f():        return ((yield i) for i in range(3))  

Re: [Python-Dev] Make the stable API-ABI usable

2017-11-20 Thread Hrvoje Niksic
On 11/19/2017 12:50 PM, Serhiy Storchaka wrote: But if PyTuple_GET_ITEM() is used for getting a reference to a C array of items it can't be replaced with PyTuple_GetItem(). And actually there is no replacement for this case in the limited API. PyObject **items = &PyTuple_GET_ITEM(tuple, 0)

Re: [Python-Dev] API design question: how to extend sys.settrace()?

2017-09-27 Thread Hrvoje Niksic
On 09/27/2017 02:56 PM, Victor Stinner wrote: Hi, In bpo-29400, it was proposed to add the ability to trace not only function calls but also instructions at the bytecode level. I like the idea, but I don't see how to extend sys.settrace() to add a new "trace_instructions: bool" optional (keyword

Re: [Python-Dev] List mutation in list_repr?

2016-12-06 Thread Hrvoje Niksic
> and I also don’t see any clue in the source as to when [list mutation] > would actually happen. Since inside the loop, the list object `v` is > never accessed other than passing `v->ob_item[i]` to the recursive > repr call, there shouldn’t be any mutation on the list object itself. The individu

Re: [Python-Dev] Inconsistency of PyModule_AddObject()

2016-04-27 Thread Hrvoje Niksic
On 04/27/2016 09:14 AM, Serhiy Storchaka wrote: There are three functions (or at least three documented functions) in C API that "steals" references: PyList_SetItem(), PyTuple_SetItem() and PyModule_AddObject(). The first two "steals" references even on failure, and this is well known behaviour.

Re: [Python-Dev] Reading Python source file

2015-11-18 Thread Hrvoje Niksic
On 11/18/2015 04:48 PM, Guido van Rossum wrote: That trick doesn't work unless the data looks like Python comments or data (e.g. a docstring). Python has always insisted on being able to parse until EOF. The only extreme case would be a small script followed by e.g. 4 GB of comments (where the ol

Re: [Python-Dev] Reading Python source file

2015-11-18 Thread Hrvoje Niksic
On 11/18/2015 03:31 AM, Nick Coghlan wrote: That behaviour is then inherited at the command line by both the -m switch and the support for executing directories and zip archives. When we consider that the "-c" switch also executes an in-memory string, direct script execution is currently the odd

Re: [Python-Dev] How is obmalloc safe with "Invalid read of size 4" ?

2015-03-24 Thread Hrvoje Niksic
On 03/24/2015 03:28 PM, Karl Pickett wrote: So we then tried running it under valgrind, and we got a lot of nasty errors. Even after reading the Misc/README.valgrind, which talks about *uninitialized* reads being ok, I still don't see how reading from *freed* memory would ever be safe, and why t

Re: [Python-Dev] Python 2.x and 3.x use survey, 2014 edition

2014-12-17 Thread Hrvoje Niksic
On 12/16/2014 08:18 PM, R. David Murray wrote: On Tue, 16 Dec 2014 10:48:07 -0800, Mark Roberts wrote: > Besides, using iteritems() and friends is generally a premature > optimization, unless you know you'll have very large containers. > Creating a list is cheap. [...] No. A premature optimi

Re: [Python-Dev] Please reconsider PEP 479.

2014-11-26 Thread Hrvoje Niksic
On 11/26/2014 12:24 PM, Nick Coghlan wrote: Now needs to be written out explicitly as: def my_generator(): ... try: yield next(it) except StopIteration return ... To retrieve a single value from an iterator, one can use the for

Re: [Python-Dev] Internal representation of strings and Micropython

2014-06-06 Thread Hrvoje Niksic
On 06/06/2014 05:59 PM, Terry Reedy wrote: The other problem is that a small slice view of a large object keeps the large object alive, so a view user needs to think carefully about whether to make a copy or create a view, and later to copy views to delete the base object. This is not for beginne

Re: [Python-Dev] Internal representation of strings and Micropython

2014-06-06 Thread Hrvoje Niksic
On 06/04/2014 05:52 PM, Mark Lawrence wrote: On 04/06/2014 16:32, Steve Dower wrote: If copying into a separate list is a problem (memory-wise), re.finditer('\\S+', string) also provides the same behaviour and gives me the sliced string, so there's no need to index for anything. Out of idl

Re: [Python-Dev] Returning None from methods that mutate object state

2014-05-19 Thread Hrvoje Niksic
On 05/17/2014 10:26 AM, Terry Reedy wrote: > When list.pop was added, the convention was changed to > "do not return the 'self' parameter" Do you have a reference for this? It is my understanding that the convention is for mutators to return None, in order to make it clear that the change is de

Re: [Python-Dev] Intricacies of calling __eq__

2014-03-19 Thread Hrvoje Niksic
On 03/18/2014 10:19 PM, Paul Moore wrote: Surely in the presence of threads the optimisation is invalid anyway Why? As written, the code uses no synchronization primitives to ensure that the modifications to the dict are propagated at a particular point. As a consequence, it cannot rely on th

Re: [Python-Dev] Python 3.4: Cherry-picking into rc2 and final

2014-02-19 Thread Hrvoje Niksic
On 02/19/2014 01:20 PM, Antoine Pitrou wrote: On Tue, 18 Feb 2014 18:46:16 -0800 Guido van Rossum wrote: I do think there's one legitimate concern -- someone might pull a diff from Larry's branch and then accidentally push it back to the public repo, and then Larry would be in trouble if he was

Re: [Python-Dev] RFC: PEP 460: Add bytes % args and bytes.format(args) to Python 3.5

2014-01-07 Thread Hrvoje Niksic
On 01/07/2014 02:22 PM, Serhiy Storchaka wrote: Most popular formatting codes in Mercurial sources: 2519 %s 493 %d 102 %r 48 %Y 47 %M 41 %H 39 %S 38 %m 33 %i 29 %b [...] Are you sure you're not including str[fp]time formats in t

Re: [Python-Dev] type.__subclasses__() doesn't work

2013-10-09 Thread Hrvoje Niksic
On 10/09/2013 02:22 PM, Peter Otten wrote: py> type.__subclasses__(type) [, ] The underlying problem seems to be that there is no helper function to bypass the instance attribute. Note that the problem is specific to the "type" type, which is its own metatype. With other types that get __sub

Re: [Python-Dev] sys.intern should work on bytes

2013-09-23 Thread Hrvoje Niksic
On 09/20/2013 06:50 PM, PJ Eby wrote: On Fri, Sep 20, 2013 at 9:54 AM, Jesus Cea wrote: Why str/bytes doesn't support weakrefs, beside memory use? The typical use case for weakrefs is to break reference cycles, Another typical use case, and the prime reason why languages without reference

Re: [Python-Dev] Add a "transformdict" to collections

2013-09-10 Thread Hrvoje Niksic
On 09/10/2013 02:24 PM, Paul Moore wrote: td['FOO'] = 42 td['foo'] = 32 list(td.keys()) ['FOO'] or ['foo']? Both answers are justifiable. Note that the same question can be reasonably asked for dict itself: >>> d = {} >>> d[1.0] = 'foo' >>> d[1] = 'bar' >>> d {1.0: 'bar'} So, dict.__setitem

Re: [Python-Dev] PEP 409 and the stdlib

2013-06-21 Thread Hrvoje Niksic
On 05/21/2013 10:36 AM, Serhiy Storchaka wrote: 21.05.13 10:17, Hrvoje Niksic написав(ла): On 05/20/2013 05:15 PM, Ethan Furman wrote: 1) Do nothing and be happy I use 'raise ... from None' in my own libraries 2) Change the wording of 'During handling of the above exc

Re: [Python-Dev] PEP 409 and the stdlib

2013-05-21 Thread Hrvoje Niksic
On 05/21/2013 02:57 PM, Serhiy Storchaka wrote: 21.05.13 13:05, Hrvoje Niksic написав(ла): On 05/21/2013 11:56 AM, Serhiy Storchaka wrote: try: x = d['key'] except KeyError: x = fallback('key') def fallback(key): if key not in a:

Re: [Python-Dev] PEP 409 and the stdlib

2013-05-21 Thread Hrvoje Niksic
On 05/21/2013 11:56 AM, Serhiy Storchaka wrote: try: x = d['key'] except KeyError: x = fallback('key') def fallback(key): if key not in a: raise BusinessError(...) return 1 / a[key] # possible TypeError, ZeroDivisionError, etc Yes, in that case the exception w

Re: [Python-Dev] PEP 409 and the stdlib

2013-05-21 Thread Hrvoje Niksic
On 05/21/2013 10:36 AM, Serhiy Storchaka wrote: The above exception was converted to the following exception: ... That makes it clear that the conversion was explicit and (hopefully) intentional, and that the latter exception supersedes the former. How do you distinguish intentional

Re: [Python-Dev] PEP 409 and the stdlib

2013-05-21 Thread Hrvoje Niksic
On 05/20/2013 05:15 PM, Ethan Furman wrote: 1) Do nothing and be happy I use 'raise ... from None' in my own libraries 2) Change the wording of 'During handling of the above exception, another exception occurred' (no ideas as to what at the moment) The word "occurred" misleads one to think

Re: [Python-Dev] Semantics of __int__(), __index__()

2013-04-04 Thread Hrvoje Niksic
Eric Snow: On Wed, Apr 3, 2013 at 6:47 AM, Hrvoje Niksic wrote: It seems like a good feature that an __int__ implementation can choose to return an int subclass with additional (and optional) information. After all, int subclass instances should be usable everywhere where ints are, including

Re: [Python-Dev] Semantics of __int__(), __index__()

2013-04-03 Thread Hrvoje Niksic
On 04/03/2013 01:17 PM, Nick Coghlan wrote: > > > > > I like Nick's answer to that: int *should* always return something of > > exact type int. Otherwise you're always left wondering whether you > > have to do "int(int(x))", or perhaps even "int(int(int(x)))", to be > > absolutely sure of g

Re: [Python-Dev] can't assign to function call

2013-03-18 Thread Hrvoje Niksic
On 03/18/2013 04:40 PM, Steven D'Aprano wrote: In analogy with that, Python could implement what looks like assignment to function call like this: val = f(arg) # val = f.__call__(arg) f(arg) = val # f.__setcall__(arg, val) That's all very well, but what would it do? It's not

Re: [Python-Dev] can't assign to function call

2013-03-18 Thread Hrvoje Niksic
On 03/18/2013 03:23 PM, Chris Angelico wrote: The languages that permit you to assign to a function call all have some notion of a reference type. Assigning to function calls is orthogonal to reference types. For example, Python manages assignment to subscripts without having references just

Re: [Python-Dev] Rationale for different signatures of tuple.__new__ and namedtuple.__new__

2013-02-18 Thread Hrvoje Niksic
On 02/18/2013 03:32 PM, John Reid wrote: I can do tuple([1,2,3]) but not: from collections import namedtuple namedtuple('B', 'x y z')([1,2,3]) I get a TypeError: __new__() takes exactly 4 arguments (2 given) However I can do: namedtuple('B', 'x y z')._make([1,2,3]) So namedtuple's _make cla

Re: [Python-Dev] PEP 433: second try

2013-01-30 Thread Hrvoje Niksic
On 01/30/2013 01:00 PM, Victor Stinner wrote: Disable inheritance by default (...) * It violates the principle of least surprise. Developers using the os module may expect that Python respects the POSIX standard and so that close-on-exec flag is not set by default. Oh, I just saw that Perl

Re: [Python-Dev] [Python-checkins] Daily reference leaks (aef7db0d3893): sum=287

2013-01-14 Thread Hrvoje Niksic
On 01/12/2013 02:46 PM, Eli Bendersky wrote: The first report is legit, however. PyTuple_New(0) was called and its return value wasn't checked for NULL. The author might have been relying on Python caching the empty tuple. ___ Python-Dev mailing list

Re: [Python-Dev] type vs. class terminology

2012-11-25 Thread Hrvoje Niksic
On 11/26/2012 06:01 AM, Chris Jerdonek wrote: I would like to know when we should use "class" in the Python 3 documentation, and when we should use "type." Are these terms synonymous in Python 3, and do we have a preference for which to use and when? Some people like to use "class" for the sub

Re: [Python-Dev] Why not using the hash when comparing strings?

2012-10-19 Thread Hrvoje Niksic
On 10/19/2012 03:22 AM, Benjamin Peterson wrote: It would be interesting to see how common it is for strings which have their hash computed to be compared. Since all identifier-like strings mentioned in Python are interned, and therefore have had their hash computed, I would imagine comparing

Re: [Python-Dev] AST optimizer implemented in Python

2012-08-14 Thread Hrvoje Niksic
On 08/14/2012 03:32 PM, Victor Stinner wrote: I had the idea (perhaps not an original one) that peephole optimization would be much better done in python than in C. The C code is clunky and unwieldly, wheras python would be much better suited, being able to use nifty regexes and the like.

Re: [Python-Dev] Adding types.build_class for 3.3

2012-05-07 Thread Hrvoje Niksic
On 05/07/2012 02:15 PM, Nick Coghlan wrote: Benjamin's suggestion of a class method on type may be a good one, though. Then the invocation (using all arguments) would be: mcl.build_class(name, bases, keywords, exec_body) Works for me, so unless someone else can see a problem I've missed, we'

Re: [Python-Dev] Suggested addition to PEP 8 for context managers

2012-04-18 Thread Hrvoje Niksic
On 04/17/2012 04:21 PM, Guido van Rossum wrote: I hope that's now what it says about slices -- that was meant for dict displays. For slices it should be symmetrical. In this case I would remove the spaces around the +, but it's okay to add spaces around the : too. It does look odd to have an oper

Re: [Python-Dev] Hashing proposal: change only string-only dicts

2012-01-20 Thread Hrvoje Niksic
On 01/18/2012 06:55 PM, "Martin v. Löwis" wrote: I was thinking about adding the field at the end, Will this make all strings larger, or only those that create dict collisions? Making all strings larger to fix this issue sounds like a really bad idea. Also, would it be acceptable to simply

Re: [Python-Dev] Status of the fix for the hash collision vulnerability

2012-01-18 Thread Hrvoje Niksic
On 01/17/2012 09:29 PM, "Martin v. Löwis" wrote: I(0) = H& MASK PERTURB(0) = H I(n+1) = (5*I(n) + 1 + PERTURB(n))& MASK PERTURN(n+1) = PERTURB(n)>> 5 So if two objects O1 and O2 have the same hash value H, the sequence of probed indices is the same for any MASK value. It will

Re: [Python-Dev] Compiling the source without stat

2011-12-15 Thread Hrvoje Niksic
On 12/14/2011 05:04 PM, Hossein wrote: If there is anything I should do You can determine what the code that calls stat() is trying to do, and implement that with other primitives that your platform provides. For example, you can determine whether a file exists by trying to open it in read-

Re: [Python-Dev] Identifier API

2011-10-11 Thread Hrvoje Niksic
On 10/11/2011 02:45 PM, Amaury Forgeot d'Arc wrote: It should also check for errors; in this case the initialization is a bit more verbose: if (PY_IDENTIFIER_INIT(update) < 0) ; Error checking is somewhat more controversial because behavior in case of error differs between situations and codin

Re: [Python-Dev] Identifier API

2011-10-11 Thread Hrvoje Niksic
On 10/08/2011 04:54 PM, "Martin v. Löwis" wrote: tmp = PyObject_CallMethod(result, "update", "O", other); would be replaced with PyObject *tmp; Py_identifier(update); ... tmp = PyObject_CallMethodId(result,&PyId_update, "O", other); An alternative I am fond o

Re: [Python-Dev] RFC: Add a new builtin strarray type to Python?

2011-10-03 Thread Hrvoje Niksic
On 10/02/2011 06:34 PM, Alex Gaynor wrote: There are a number of issues that are being conflated by this thread. 1) Should str += str be fast. In my opinion, the answer is an obvious and resounding no. Strings are immutable, thus repeated string addition is O(n**2). This is a natural and

Re: [Python-Dev] PyObject_RichCompareBool identity shortcut

2011-04-28 Thread Hrvoje Niksic AVL HR
On 04/28/2011 04:31 AM, Stephen J. Turnbull wrote: Are you saying you would expect that nan = float('nan') a = [1, ..., 499, nan, 501, ..., 999]# meta-ellipsis, not Ellipsis a == a False ?? I would expect l1 == l2, where l1 and l2 are both lists, to be semantically equivalent to len

[Python-Dev] PyObject_RichCompareBool identity shortcut

2011-04-27 Thread Hrvoje Niksic
The other day I was surprised to learn this: >>> nan = float('nan') >>> nan == nan False >>> [nan] == [nan] True # also True in tuples, dicts, etc. # also: >>> l = [nan] >>> nan in l True >>> l.index(nan) 0 >>> l[0] == nan False The identity test is not in container comparators

Re: [Python-Dev] I am now lost - committed, pulled, merged, what is "collapse"?

2011-03-22 Thread Hrvoje Niksic
On 03/21/2011 05:44 PM, s...@pobox.com wrote: Thanks for the example, Hrvoje. Hrvoje> This automatic merging often causes people who migrate to a DVCS Hrvoje> to feel that they have to go through an unnecessary extra step Hrvoje> in their workflows. But once you grasp the "ho

Re: [Python-Dev] I am now lost - committed, pulled, merged, what is "collapse"?

2011-03-21 Thread Hrvoje Niksic
On 03/21/2011 01:34 PM, Stephen J. Turnbull wrote: > Subversion never ever creates versions in the repository that > didn't before exist in some working copy. John Arbash-Meinel disagrees with you, so I think I'll go with his opinion Besides, it's easy to confirm: # create a repository

Re: [Python-Dev] %-formatting depracation

2011-02-23 Thread Hrvoje Niksic
On 02/22/2011 11:03 PM, Antoine Pitrou wrote: I think there are many people still finding %-style more practical for simple uses, It's also a clash of cultures. People coming from a C/Unix background typically find %-style format obvious and self-explanatory, while people coming from Java/Dot

Re: [Python-Dev] nonlocal x = value

2010-12-24 Thread Hrvoje Niksic
On 12/23/2010 10:03 PM, Laurens Van Houtven wrote: On Thu, Dec 23, 2010 at 9:51 PM, Georg Brandl wrote: Yes and no -- there may not be an ambiguity to the parser, but still to the human. Except if you disallow the syntax in any case, requiring people to write nonlocal x = (3, y) which i

Re: [Python-Dev] Using logging in the stdlib and its unit tests

2010-12-10 Thread Hrvoje Niksic
On 12/10/2010 10:47 AM, Stephen J. Turnbull wrote: Vinay Sajip writes: > Indeed, and the very first code sample in the logging documentation > shows exactly the simplistic easy usage you're talking about. I > can't see why anyone would be scared off by that example. They're not scare

Re: [Python-Dev] [Python-checkins] r86633 - in python/branches/py3k: Doc/library/inspect.rst Doc/whatsnew/3.2.rst Lib/inspect.py Lib/test/test_inspect.py Misc/NEWS

2010-11-22 Thread Hrvoje Niksic
On 11/22/2010 04:37 PM, Antoine Pitrou wrote: +1. The problem with int constants is that the int gets printed, not the name, when you dump them for debugging purposes :) Well, it's trivial to subclass int to something with a nicer __repr__. PyGTK uses that technique for wrapping C enums: >>

Re: [Python-Dev] Breaking undocumented API

2010-11-12 Thread Hrvoje Niksic
On 11/11/2010 11:24 PM, Greg Ewing wrote: Nick Coghlan wrote: My personal opinion is that we should be trying to get the standard library to the point where __all__ definitions are unnecessary - if a name isn't in __all__, it should start with an underscore (and if that is true, then the __

Re: [Python-Dev] Breaking undocumented API

2010-11-10 Thread Hrvoje Niksic
On 11/10/2010 05:12 AM, Stephen J. Turnbull wrote: But these identifiers will appear at the module level, not global, no? Otherwise this technique couldn't be used. I don't really understand what Tres is talking about when he writes "modules that expect to be imported this way". The *imported*

Re: [Python-Dev] On breaking modules into packages Was: [issue10199] Move Demo/turtle under Lib/

2010-11-03 Thread Hrvoje Niksic
On 11/03/2010 01:47 AM, Ben Finney wrote: If someone wants to depend on some undocumented detail of the directory layout it's their problem (like people depending on bytecode and other stuff). I would say that names without a single leading underscore are part of the public API, whether docu

Re: [Python-Dev] Issue 10194 - Adding a gc.remap() function

2010-10-27 Thread Hrvoje Niksic
On 10/26/2010 07:11 PM, Peter Ingebretson wrote: The main argument is that preserving immutable objects increases the complexity of remapping and does not actually solve many problems. The primary reason for objects to be immutable is so that their comparison operators and hash value can remain c

Re: [Python-Dev] Issue 10194 - Adding a gc.remap() function

2010-10-26 Thread Hrvoje Niksic
On 10/26/2010 07:04 AM, Peter Ingebretson wrote: I have a patch that adds a new function to the gc module. The gc.remap() function uses the tp_traverse mechanism to find all references to any keys in a provided mapping, and remaps these references in-place to instead point to the value correspon

Re: [Python-Dev] Resource leaks warnings

2010-09-29 Thread Hrvoje Niksic
On 09/29/2010 02:42 PM, Antoine Pitrou wrote: It seems like a slippery slope. Sometimes you really don't care like when you're just hacking together a quick script. Isn't the "with" statement appropriate in these cases? A hacked-together quick script might contain code like: parse(open(bla

Re: [Python-Dev] Behaviour of max() and min() with equal keys

2010-09-08 Thread Hrvoje Niksic
On 09/07/2010 11:40 PM, Jeffrey Yasskin wrote: Decimal may actually have this backwards. The idea would be that min(*lst) == sorted(lst)[0], and max(*lst) == sorted(lst)[-1]. Here you mean "is" rather than "==", right? The relations you spelled are guaranteed regardless of stability. (This

Re: [Python-Dev] Buffer protocol for io.BytesIO?

2010-09-03 Thread Hrvoje Niksic
On 09/02/2010 10:35 PM, Antoine Pitrou wrote: Then it came to me then perhaps it would be too automatic. So I'm currently floating between: - add implicit buffer protocol support to BytesIO objects - add explicit buffer protocol support through the call of a getbuffer() method, which would re

Re: [Python-Dev] 'hasattr' is broken by design

2010-08-24 Thread Hrvoje Niksic
On 08/24/2010 02:31 PM, Benjamin Peterson wrote: 2010/8/24 Hrvoje Niksic: The __length_hint__ lookup expects either no exception or AttributeError, and will propagate others. I'm not sure if this is a bug. On the one hand, throwing anything except AttributeError from __getattr__ i

Re: [Python-Dev] 'hasattr' is broken by design

2010-08-24 Thread Hrvoje Niksic
On 08/23/2010 04:56 PM, Guido van Rossum wrote: On Mon, Aug 23, 2010 at 7:46 AM, Benjamin Peterson wrote: 2010/8/23 Yury Selivanov: 1) I propose to change 'hasattr' behaviour in Python 3, making it to swallow only AttributeError exceptions (exactly like 'getattr'). Probably, Python 3.2 re

Re: [Python-Dev] mkdir -p in python

2010-07-28 Thread Hrvoje Niksic
On 07/27/2010 06:18 PM, Alexander Belopolsky wrote: On Tue, Jul 20, 2010 at 10:20 AM, R. David Murray wrote: I'd go with putting it in shutil. +1 I would also call it shutil.mktree which will go well with shutil.rmtree next to it. Note that mktree is not analogous to rmtree - while rmtree

Re: [Python-Dev] New regex module for 3.2?

2010-07-23 Thread Hrvoje Niksic
On 07/22/2010 01:34 PM, Georg Brandl wrote: Timings (seconds to run the test suite): re 26.689 26.015 26.008 regex 26.066 25.797 25.865 So, I thought there wasn't a difference in performance for this use case (which is compiling a lot of regexes and matching most of them only a few tim

Re: [Python-Dev] Possible patch for functools partial - Interested?

2010-05-17 Thread Hrvoje Niksic
On 05/14/2010 06:39 AM, Daniel Urban wrote: I've made a new patch, in which the keywords attribute is a read-only proxy of the dictionary. What about backward compatibility? This looks like an incompatible change. ___ Python-Dev mailing list Python-D

Re: [Python-Dev] recursive closures - reference leak

2009-12-08 Thread Hrvoje Niksic
Kristján Valur Jónsson wrote: The problem with this is that once you have called factorial() once, you end up with a recursive cycle. „factorial“ has become a cell object, referencing the „helper“ function, which again refers to the outer cell object. This requires „gc“ to clean up. Also, it

Re: [Python-Dev] PEP 389: argparse - new command line parsing module

2009-10-07 Thread Hrvoje Niksic
Paul Moore wrote: Traceback (most recent call last): File "hello.py", line 13, in main() File "hello.py", line 7, in main sys.stdout.flush() IOError: [Errno 9] Bad file descriptor (Question - is it *ever* possible for a Unix program to have invalid file descriptors 0,1 and 2? At sta

Re: [Python-Dev] Py_TPFLAGS_HEAPTYPE too overloaded

2009-07-30 Thread Hrvoje Niksic
Campbell Barton wrote: I'm not expert enough in this area to know if malloc'ing PyTypeObject and initializing has some other problems. The only problem is that such types will be expected to be around forever - they are not reference-counted like heap types, so there is no mechanism to free t

Re: [Python-Dev] Functions that steal references (Re: [pygame] [patch] minor memory leaks...)

2009-06-17 Thread Hrvoje Niksic
Christian Heimes wrote: I assumed that since PyModule_AddObject is documented as stealing a reference, it always stole a reference. But in reality it only does so conditionally, when it succeeds. As an aside, is this a general feature of functions that steal references, or is PyModule_AddObject

Re: [Python-Dev] PEP 383: Non-decodable Bytes in System Character Interfaces

2009-04-29 Thread Hrvoje Niksic
Zooko O'Whielacronx wrote: If you switch to iso8859-15 only in the presence of undecodable UTF-8, then you have the same round-trip problem as the PEP: both b'\xff' and b'\xc3\xbf' will be converted to u'\u00ff' without a way to unambiguously recover the original file name. Why do you say

Re: [Python-Dev] PEP 383: Non-decodable Bytes in System Character Interfaces

2009-04-28 Thread Hrvoje Niksic
Lino Mastrodomenico wrote: Since this byte sequence [b'\xed\xb3\xbf'] doesn't represent a valid character when decoded with UTF-8, it should simply be considered an invalid UTF-8 sequence of three bytes and decoded to '\udced\udcb3\udcbf' (*not* '\udcff'). "Should be considered" or "will be co

Re: [Python-Dev] PEP 383: Non-decodable Bytes in System Character Interfaces

2009-04-28 Thread Hrvoje Niksic
Thomas Breuel wrote: But the biggest problem with the proposal is that it isn't needed: if you want to be able to turn arbitrary byte sequences into unicode strings and back, just set your encoding to iso8859-15. That already works and it doesn't require any changes. Are you proposing to unc

Re: [Python-Dev] PEP 383 (again)

2009-04-28 Thread Hrvoje Niksic
Lino Mastrodomenico wrote: Let's suppose that I use Python 2.x or something else to create a file with name b'\xff'. My (Linux) system has a sane configuration and the filesystem encoding is UTF-8, so it's an invalid name but the kernel will blindly accept it anyway. With this PEP, Python 3.1 li

Re: [Python-Dev] Getting values stored inside sets

2009-04-06 Thread Hrvoje Niksic
Raymond Hettinger wrote: Hrvoje Niksic wrote: I've stumbled upon an oddity using sets. It's trivial to test if a value is in the set, but it appears to be impossible to retrieve a stored value, See: http://code.activestate.com/recipes/499299/ Thanks, this is *really* good, t

[Python-Dev] Getting values stored inside sets

2009-04-03 Thread Hrvoje Niksic
I've stumbled upon an oddity using sets. It's trivial to test if a value is in the set, but it appears to be impossible to retrieve a stored value, other than by iterating over the whole set. Let me describe a concrete use case. Imagine a set of objects identified by some piece of informatio

Re: [Python-Dev] Let's update CObject API so it is safe and regular!

2009-04-03 Thread Hrvoje Niksic
Larry Hastings wrote: If we're adding type information, then please make it a Python object rather than a C string. That way the creator and the consumer can use a richer API to query the "type", such as by calling its methods or by inspecting it in some other way. I'm not writing my patch t

Re: [Python-Dev] Let's update CObject API so it is safe and regular!

2009-04-02 Thread Hrvoje Niksic
Greg Ewing wrote: Attaching some kind of type info to a CObject and having an easy way of checking it makes sense to me. If the existing CObject API can't be changed, maybe a new enhanced one could be added. I thought the entire *point* of C object was that it's an opaque box without any info

Re: [Python-Dev] suggestion for try/except program flow

2009-03-27 Thread Hrvoje Niksic
Mark Donald wrote: Thanks for the idea, but I believe it has a different outcome. You'd have to copy the generic handler into an except clause to get exactly the behaviour I'm looking for, which is worse than nesting the try blocks Then simply change Exception to BaseException. Since all excep

Re: [Python-Dev] suggestion for try/except program flow

2009-03-27 Thread Hrvoje Niksic
Mark Donald wrote: I frequently have this situation: try: try: raise Thing except Thing, e: # handle Thing exceptions raise except: # handle all exceptions, including Thing How about: try: ... code that can raise Thing or another exception ... except Ex

Re: [Python-Dev] speeding up PyObject_GetItem

2009-03-24 Thread Hrvoje Niksic
Nick Coghlan wrote: Many of the routines in abstract.c check their parameters for NULL, as a sanity check, and throw a SystemError if NULL is detected. I'm tempted to suggest making these checks only when Py_DEBUG is defined, but I suspect if you wanted it that way, you would have changed it alr

Re: [Python-Dev] Proposal: new list function: pack

2009-03-20 Thread Hrvoje Niksic
Isaac Morland wrote: I propose this because i need a lot of times pack and slide function over list and this one combine the two in a generator way. I've written functions with a subset of this functionality on more than one occasion. Having it in itertools looks like it would be useful to a

Re: [Python-Dev] In-place operators

2009-03-18 Thread Hrvoje Niksic
Martin v. Löwis wrote: Certainly, the doc string is wrong: isub(a, b) -- Same as a -= b. That's not quite the same - you would have to write a = isub(a, b) -- Same as a -= b. It seems a perfectly fine solution is simply to fix the docstring, exactly like that: a = isub(a, b) -- Same a

Re: [Python-Dev] Ext4 data loss

2009-03-11 Thread Hrvoje Niksic
Christian Heimes wrote: Guido van Rossum wrote: Let's not think too Unix-specific. If we add such an API it should do something on Windows too -- the app shouldn't have to test for the presence of the API. (And thus the API probably shouldn't be called fsync.) In my initial proposal one and a

Re: [Python-Dev] Ext4 data loss

2009-03-11 Thread Hrvoje Niksic
Joachim König wrote: To me, the flaw seem to be in the close() call (of the operating system). I'd expect the data to be in a persistent state once the close() returns. I wouldn't, because that would mean that every cp -r would effectively do an fsync() for each individual file it copies, whi

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

2009-03-06 Thread Hrvoje Niksic
Greg Ewing wrote: Even if you don't agree that using O_NONBLOCK with select/poll is the best approach to non-blocking, I think there is enough existing practice of doing this to warrant separate consideration of non-blocking sockets (in the OS sense) and select/poll. I'm not saying there isn'

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

2009-03-06 Thread Hrvoje Niksic
Greg Ewing wrote: Antoine Pitrou wrote: For starters, since py3k is supposed to support non-blocking IO, why not write a portable API to make a raw file or socket IO object non-blocking? I think we need to be clearer what we mean when we talk about non-blocking in this context. Normally when

Re: [Python-Dev] PEP 372 -- Adding an ordered directory to collections ready for pronouncement

2009-03-04 Thread Hrvoje Niksic
Steven D'Aprano wrote: Gisle Aas wrote: Instead of introducing a sorteddict I would instead suggest that the future should bring an odict with a sort method; possibly also keys_sorted and items_sorted methods. Instead of odict.sorted(), that can be spelled: sorted(odict) # sort the keys so

Re: [Python-Dev] multiprocessing not compatible with functional.partial

2009-02-12 Thread Hrvoje Niksic
Calvin Spealman wrote: I don't think it would be unreasonable to consider either 1) making functools.partial picklable (I don't know how feasible this is) It's not only feasible, but quite easy and, I think, useful. A "partial" instance is a simple triplet of (function, args, kwds), and it c

Re: [Python-Dev] Missing operator.call

2009-02-05 Thread Hrvoje Niksic
Guido van Rossum wrote: def call(o, *args, **kwds): return o(*args, **kwds) which would make call a synonym for apply (and would also provide for the first definition as a special case). However, with that API, it isn't so easy anymore to pass the same arguments to all callables (unless it is

Re: [Python-Dev] Missing operator.call

2009-02-04 Thread Hrvoje Niksic
Nick Coghlan wrote: I'm somewhere between -0 and +0 though (-0 due to the lack of concrete use cases, +0 because the improved consistency is appealing) The operator module is one of the rare cases in python where consistency is valued more than concrete use cases. But, for what it's worth, I

Re: [Python-Dev] Missing operator.call

2009-02-04 Thread Hrvoje Niksic
Andrew Bennetts wrote: A patch to add operator.caller(*args, **kwargs) may be a good idea. Your example would then be: map(operator.caller(), lst) Regarding the name, note that I proposed operator.call (and operator.__call__) because it corresponds to the __call__ special method, which

[Python-Dev] Missing operator.call

2009-02-04 Thread Hrvoje Niksic
Is there a reason why the operator module doesn't have an operator.call function? It would seem logical to be able to write: map(operator.call, lst) which calls each object in lst, just like map(operator.neg, lst) negates every object. Of course, operator.call is equivalent to lambda x: x(),

Re: [Python-Dev] C API for appending to arrays

2009-02-04 Thread Hrvoje Niksic
Mike Klaas wrote: Do you need to append, or are you just looking to create/manipulate an array with a bunch of c-float values? Mostly real-life examples I've seen of this were creating an array from C values obtained from an external source, such as an on-disk file, or another process. The

Re: [Python-Dev] C API for appending to arrays

2009-02-03 Thread Hrvoje Niksic
Raymond Hettinger wrote: [Hrvoje Niksic] The one thing missing from the array module is the ability to directly access array values from C. Please put a feature request on the bug tracker. Done, http://bugs.python.org/issue5141 ___ Python-Dev

[Python-Dev] C API for appending to arrays

2009-02-02 Thread Hrvoje Niksic
The array.array type is an excellent type for storing a large amount of "native" elements, such as integers, chars, doubles, etc., without involving the heavy machinery of numpy. It's both blazingly fast and reasonably efficient with memory. The one thing missing from the array module is the

Re: [Python-Dev] subprocess crossplatformness and async communication

2009-01-26 Thread Hrvoje Niksic
Nick Craig-Wood wrote: But for the conversational case (eg using it to do a remote login) it doesn't work at all :- run child send stuff to stdin child reads stdin and writes stdout Can this really be made safe without an explicit flow control protocol, such as a pseudo-TTY? stdio read

Re: [Python-Dev] Python under valgrind

2008-11-28 Thread Hrvoje Niksic
Amaury Forgeot d'Arc wrote: Did you use the suppressions file as suggested in Misc/README.valgrind? Thanks for the suggestion (as well as to Gustavo and Victor), but my question wasn't about how to suppress the messages, but about why the messages appear in the first place. I think my last p

[Python-Dev] Python under valgrind

2008-11-28 Thread Hrvoje Niksic
A friend pointed out that running python under valgrind (simply "valgrind python") produces a lot of "invalid read" errors. Reading up on Misc/README.valgrind only seems to describe why "uninitialized reads" should occur, not invalid ones. For example: $ valgrind python [... lots of output .

Re: [Python-Dev] __import__ problems

2008-11-28 Thread Hrvoje Niksic
Mart Somermaa wrote: I meant that you have to import sys only to access sys.modules (i.e. importing sys may not be necessary otherwise). I understand that, but I'm arguing that that's a non-problem. Importing sys is a regular thing in Python, not an exception. You need sys to get to sys

Re: [Python-Dev] __import__ problems

2008-11-28 Thread Hrvoje Niksic
Mart Somermaa wrote: The variant proposed by Hrvoje Niksic: >>> __import__(modname) >>> mod = sys.modules[modname] looks more appealing, but comes with the drawback that sys has to be imported for that purpose only. That is not a real drawback, as "sys" will

Re: [Python-Dev] __import__ problems

2008-11-27 Thread Hrvoje Niksic
Mart Somermaa wrote: There at least two workarounds: * the getattr approach documented in [3] I can't comment on the rest, but the getattr seems overly complicated. If you need just the module, what's wrong with: __import__(modname) modobj = sys.modules[modname] ___