PythonWin, python thread and PostQuitMessage?
Hi All, I'm not so much involved in any Windows programming however I needed to write a client for the Windows platform. I have this very simple question which I've been unable to answer. I'm listening for keyboard strokes using the pyhook library. I'm doing this in a dedicated thread. The gui just controls the thread. When I want to pause listening for keyboard strokes I wanted to do a PostQuitMessage() to the thread. However this does not work since it either kills the whole app or just does not happen in the thread it's supposed to. I've now made an ugly workaround using PumpWaitingMessages and a delay. def run(self): print "Wkeylog run called" # Hook Keyboard self.hm.HookKeyboard() while self.log: win32gui.PumpWaitingMessages() time.sleep(0.02) i can now just cancel the process by setting self.log to False. I wanted to do just: def run(self): print "Wkeylog run called" # Hook Keyboard self.hm.HookKeyboard() win32gui.PumpMessages() However I'm unable to stop this process. Can anyone shred al light here? Rg, Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: socket programming
On 6 May 2013 17:05, Chris Angelico wrote: > I've never used Twisted, so I can't say how good it is. All I know is > that what I learned about socket programming in C on OS/2 is still > valid on Windows and on Linux, and in every language I've ever used > (bar JavaScript and ActionScript, which are deliberately sandboxed and > thus don't have direct socket calls available). So take your pick: Go > with Twisted, and bind yourself to a particular library, or go with > BSD sockets, and do the work yourself. Like everything else in > programming, it's a tradeoff. OTOH Twisted can handle much more than socket programming. On the third hand Twisted has its own learning curve as well... -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: Encapsulation, inheritance and polymorphism
On 17 July 2012 13:01, Lipska the Kat wrote:
> Well I've set myself a task.
> I have a text file containing a list of stock items
> each line contains the number in stock followed by a tab followed by the
> name of the item. I need to implement something that reads in the text file
> and outputs the stock list in ascending or descending order of quantity.
>
> Please note I am NOT asking for solutions.
>
> In bash this is laughably trivial
> sort -nr $1 | head -${2:-10}
Here's a Python translation that won't help :)
from sys import argv
print ''.join(sorted(open(argv[1]), key=lambda l:
-int(l.split('\t')[0]))[:10 if len(argv) < 3 else int(argv[2])])
Who said Python was readabl'y yours,
--
Arnaud
--
http://mail.python.org/mailman/listinfo/python-list
Re: Generating valid identifiers
On 26 July 2012 13:26, Laszlo Nagy wrote:
[...]
> I do not want this program to generate very long identifiers. It would
> increase SQL parsing time, and don't look good. Let's just say that the
> limit should be 32 characters. But I also want to recognize the identifiers
> when I look at their modified/truncated names.
[...]
> print repr(Connection.makename("group1_group2_group3_some_field_name"))
> 'group1_group2_group3_some_fiel$AyQVQUXoyf'
>>> len('group1_group2_group3_some_fiel$AyQVQUXoyf')
41
You've exceeded 32 characters! Perhaps you could change:
return basename[:30]+"$"+tail
to:
return basename[:21]+"$"+tail
But then you'd get something like this for your long identifier:
group1_group2_group3_$AyQVQUXoyf
which seems to miss out on a crucial bit: namely the field name.
Also it's hard to imagine a way to keep things readable when we don't
know what the original identifiers look like :)
--
Arnaud
--
http://mail.python.org/mailman/listinfo/python-list
Re: OT: Text editors
On 29 July 2012 06:36, rusi wrote: > Just curious about your emacs+python usage. > Do you use the emacs builtin python mode or the separate python-mode? > Do you use pdb? > Any other special setups? One thing that I find very useful is to configure flymake to use pyflakes. Very useful to get feedback on unused imports / unused variables / undefined variables (which means you spot typos on variable names straight away). For instructions, see e.g. http://www.plope.com/Members/chrism/flymake-mode -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: looking for a neat solution to a nested loop problem
On 6 August 2012 16:52, Tom P wrote: > consider a nested loop algorithm - > > for i in range(100): > for j in range(100): > do_something(i,j) > > Now, suppose I don't want to use i = 0 and j = 0 as initial values, but some > other values i = N and j = M, and I want to iterate through all 10,000 > values in sequence - is there a neat python-like way to this? I realize I > can do things like use a variable for k in range(1): and then derive > values for i and j from k, but I'm wondering if there's something less > clunky. For example: for i in range(100): for j in range(100): do_something((i + N)%100, (j + N)%100) Cheers, -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Tkinter bug in Entry widgets on OS X
Hi all, I'm writing a small GUI on OS X (v. 10.7.4) using Tkinter. I'm using stock Python. It mostly works fine but there is a bug in Entry widgets: if and Entry widget has focus and I press the UP arrow, a "\uf700" gets inserted in the widget. If I press the DOWN arrow, a "\uf701" gets inserted. I'm very inexperienced with Tkinter (I've never used it before). All I'm looking for is a workaround, i.e. a way to somehow suppress that output. Thanks in advance, -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: Tkinter bug in Entry widgets on OS X
On 31 August 2012 15:25, Kevin Walzer wrote: > On 8/31/12 6:18 AM, Arnaud Delobelle wrote: >> >> I'm very inexperienced with Tkinter (I've never used it before). All >> I'm looking for is a workaround, i.e. a way to somehow suppress that >> output. > > > What are you trying to do? Navigate the focus to another widget? You should > use the tab bar for that, not the arrow key. The entry widget is a > single-line widget, and doesn't have up/down as the text widget does. I'm not trying to do anything. When a user presses the UP or DOWN arrow, then a strange character is inserted in the Entry box. I'd rather nothing happened. -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: Tkinter bug in Entry widgets on OS X
On 31 August 2012 16:41, Alister wrote: > On Fri, 31 Aug 2012 11:21:14 -0400, Kevin Walzer wrote: > >> On 8/31/12 11:18 AM, Arnaud Delobelle wrote: >> >> >>> I'm not trying to do anything. When a user presses the UP or DOWN >>> arrow, then a strange character is inserted in the Entry box. I'd >>> rather nothing happened. >>> >> Why is the user doing that? If they are trying to navigate to a >> different part of the interface, they need to use the tab key, not the >> arrow key. It's not a multi-line text widget and shouldn't be expected >> to work like one. So you make software that only behaves well when the user does what they're supposed to do? > I agree that it is unexpected in a single line entry box but isn't the 1st > rule of user interface design to assume the user is a moron & will do > things they are not supposed to do? > > Therefore invalid inputs should be handled gracefully (not just insert > random characters) which is what I think the original poster is > suggesting. Indeed. -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: Tkinter bug in Entry widgets on OS X
On Friday, 31 August 2012, Dennis Lee Bieber wrote: > On Fri, 31 Aug 2012 15:41:01 GMT, Alister > > > > declaimed the following in gmane.comp.python.general: > > > I agree that it is unexpected in a single line entry box but isn't the > 1st > > rule of user interface design to assume the user is a moron & will do > > things they are not supposed to do? > > > > Therefore invalid inputs should be handled gracefully (not just insert > > random characters) which is what I think the original poster is > > suggesting. > > To which I'd suggest the programmer (vs the user), probably needs > to > code some sort of per-character validation check... After all, there may > be situations where accepting pretty much any key-code is desired, and > if the widget filters characters before they get to the programmer level > that becomes impossible. > > It would be good if I could intercept the key press event and cancel its action on the Entry widget. It's easy to intercept the key event, but I haven't found out how to prevent the funny characters from being inserted into the Entry widget input area. I've struggled to find good tkinter docs on the web. > caveat -- I've only written one simple form using Tkinter, so what > do I know... It's about as much as I've done! -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: Tkinter bug in Entry widgets on OS X
On 1 September 2012 11:30, Peter Otten <[email protected]> wrote: > Arnaud Delobelle wrote: >> It would be good if I could intercept the key press event and cancel its >> action on the Entry widget. It's easy to intercept the key event, but I >> haven't found out how to prevent the funny characters from being inserted >> into the Entry widget input area. > > Untested as I have no Mac: > > import Tkinter as tk > > def suppress(event): > if event.keycode in {111, 116}: > print "ignoring", event.keycode > return "break" > print event.keycode, "accepted" > > root = tk.Tk() > entry = tk.Entry(root) > entry.bind("", suppress) > entry.pack() > > root.mainloop() This works fine! return "break" is the piece of knowledge that I was missing. Thanks a lot! In fact I lied a bit in my original message - I do use the UP and DOWN arrows on one Entry widget for navigating its command history. To do this I was binding the "" and "" events to a function, which simply has to return "break" to work around the bug. On other Entry widgets, I just bind "" and "" to a suppress function which simply returns "break". >> I've struggled to find good tkinter >> docs on the web. > > For my (basic) needs I keep coming back to > > http://infohost.nmt.edu/tcc/help/pubs/tkinter/ Thanks for the link. I was using the docs on effbot which are nice but probably incomplete (and old). I guess I should flag up this bug but I don't even know if it's a Python or Tk problem and have little time or expertise to investigate. -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
running Lua in Python
Hi all, I'm looking for a way to run Lua scripts in Python, and also send and receive data between the two. Something like lunatic-python [1] would be ideal. However, so far I haven't been able to build it on the machines it's supposed to run on (macs with OS X Lion) and it seems to be dormant. My requirements are stock OS X Python (which is 2.7) and Lua 5.2. I'm looking for either a way to make lunatic-python work or another tool that would do the job. Thanks in advance. -- Arnaud [1] http://labix.org/lunatic-python -- http://mail.python.org/mailman/listinfo/python-list
Re: running Lua in Python
On 2 September 2012 10:39, Alec Taylor wrote: > Or you can use a module made for this task: > > http://labix.org/lunatic-python As I said in my original message, I couldn't get this to work. > http://pypi.python.org/pypi/lupa I'll check this out, thanks. -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: running Lua in Python
On 2 September 2012 10:49, Arnaud Delobelle wrote: > On 2 September 2012 10:39, Alec Taylor wrote: >> http://pypi.python.org/pypi/lupa > > I'll check this out, thanks. Mmh it seems to be lua 5.1 and more importantly it seems to require a custom build of Python, which I don't want. So it seems the simplest solution is to run Lua in a child process... -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: running Lua in Python
On 2 September 2012 19:42, Stefan Behnel wrote: > Arnaud Delobelle, 02.09.2012 20:34: >> On 2 September 2012 10:49, Arnaud Delobelle wrote: >>> On 2 September 2012 10:39, Alec Taylor wrote: >>>> http://pypi.python.org/pypi/lupa >>> >>> I'll check this out, thanks. >> >> Mmh it seems to be lua 5.1 and more importantly it seems to require a >> custom build of Python, which I don't want. > > I have no idea why you say that, for neither of the two. * For the custom build of Python on OS X, this blog post: http://t-p-j.blogspot.co.uk/2010/11/lupa-on-os-x-with-macports-python-26.html which is actually linked to in lupa's own INSTALL.rst file * For Lua 5.1, the official LuaJIT page: http://luajit.org/luajit.html where I quote: """ LuaJIT implements the full set of language features defined by Lua 5.1. """ Of course I'd be glad to be proved wrong on both counts. -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: Compairing filenames in a list
On 30 September 2012 02:27, Kevin Anthony wrote:
> I have a list of filenames, and i need to find files with the same name,
> different extensions, and split that into tuples. does anyone have any
> suggestions on an easy way to do this that isn't O(n^2)?
>>> import os, itertools
>>> filenames = ["foo.png", "bar.csv", "foo.html", "bar.py"]
>>> dict((key, tuple(val)) for key, val in itertools.groupby(sorted(filenames),
>>> lambda f: os.path.splitext(f)[0]))
{'foo': ('foo.html', 'foo.png'), 'bar': ('bar.csv', 'bar.py')}
--
Arnaud
--
http://mail.python.org/mailman/listinfo/python-list
Re: Obnoxious postings from Google Groups
On 31 October 2012 22:33, Steven D'Aprano wrote: [...] > I don't killfile merely for posting from Gmail And we are humbly grateful. -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: sort order for strings of digits
On 31 October 2012 23:09, Steven D'Aprano wrote: > The trick is to take each string and split it into a leading number and a > trailing alphanumeric string. Either part may be "empty". Here's a pure > Python solution: > > from sys import maxsize # use maxint in Python 2 > def split(s): > for i, c in enumerate(s): > if not c.isdigit(): > break > else: # aligned with the FOR, not the IF > return (int(s), '') > return (int(s[:i] or maxsize), s[i:]) > > Now sort using this as a key function: > > py> L = ['9', '1000', 'abc2', '55', '1', 'abc', '55a', '1a'] > py> sorted(L, key=split) > ['1', '1a', '9', '55', '55a', '1000', 'abc', 'abc2'] You don't actually need to split the string, it's enough to return a pair consisting of the number of leading digits followed by the string as the key. Here's an implementation using takewhile: >>> from itertools import takewhile >>> def prefix(s): ... return sum(1 for c in takewhile(str.isdigit, s)) or 1000, s ... >>> L = ['9', '1000', 'abc2', '55', '1', 'abc', '55a', '1a'] >>> sorted(L, key=prefix) ['1', '1a', '9', '55', '55a', '1000', 'abc', 'abc2'] Here's why it works: >>> map(prefix, L) [(1, '9'), (4, '1000'), (1000, 'abc2'), (2, '55'), (1, '1'), (1000, 'abc'), (2, '55a'), (1, '1a')] -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Retrieving an object from a set
Dear Pythoneers, I've got a seemingly simple problem, but for which I cannot find a simple solution. I have a set of objects (say S) containing an object which is equal to a given object (say x). So x in S is true. So there is an object y in S which is equal to x. My problem is how to retrieve y, without going through the whole set. Here is a simple illustration with tuples (my actual scenario is not with tuples but with a custom class): >>> y = (1, 2, 3) # This is the 'hidden object' >>> S = set([y] + range(1)) >>> x = (1, 2, 3) >>> x in S True >>> x is y False I haven't found y. It's a very simple problem, and this is the simplest solution I can think of: class FindEqual(object): def __init__(self, obj): self.obj = obj def __hash__(self): return hash(self.obj) def __eq__(self, other): equal = self.obj == other if equal: self.lastequal = other return equal >>> yfinder = FindEqual(x) >>> yfinder in S True >>> yfinder.lastequal is y True I've found y! I'm not happy with this as it really is a trick. Is there a cleaner solution? -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: Iterate from 2nd element of a huge list
On 1 February 2012 03:16, Paulo da Silva wrote: > Em 01-02-2012 01:39, Paulo da Silva escreveu: >> Hi! >> >> What is the best way to iterate thru a huge list having the 1st element >> a different process? I.e.: >> >> process1(mylist[0]) >> for el in mylist[1:]: >> process2(el) >> >> This way mylist is almost duplicated, isn't it? >> >> Thanks. > > > I think iter is nice for what I need. > Thank you very much to all who responded. Nobody mentioned itertools.islice, which can be handy, especially if you weren't interested in the first element of the list: from itertools import islice: for el in islice(mylist, 1): process2(el) -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: 'class' named tuple
On 1 February 2012 00:54, Emmanuel Mayssat wrote: > I have the following program. > I am trying to have index the attributes of an object using __getitem__. > Reading them this way works great, but assigning them a value doesn't > Is there a way to do such a thing? > (Almost like a named tuple, but with custom methods) > > class LIter(object): > def __init__(self,parent=None): > super(LIter, self).__init__() > self.toto = 3 > self.tata = 'terto' > Add _attrs = 'toto', 'tata' def __getitem__(self, index): return getattr(self, _attrs[index]) def __setitem__(self, index, value) setattr(self, _attrs[index], value) -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: Iterate from 2nd element of a huge list
On 1 February 2012 08:11, Peter Otten <[email protected]> wrote: > Arnaud Delobelle wrote: > The example should be > >> from itertools import islice: > > for el in islice(mylist, 1, None): >> process2(el) Oops! -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: Generator problem: parent class not seen
On Feb 1, 2012 9:01 PM, "Russell E. Owen" wrote: > > I have an odd and very intermittent problem in Python script. > Occasionally it fails with this error: > > Traceback (most recent call last): > File > "/Applications/APO/TTUI.app/Contents/Resources/lib/python2.7/TUI/Base/Bas > eFocusScript.py", line 884, in run > File > "/Applications/APO/TTUI.app/Contents/Resources/lib/python2.7/TUI/Base/Bas > eFocusScript.py", line 1690, in initAll > TypeError: unbound method initAll() must be called with BaseFocusScript > instance as first argument (got ScriptClass instance instead) > self=; class hierarchy=[( 'TUI.Base.BaseFocusScript.ImagerFocusScript'>, ( 'TUI.Base.BaseFocusScript.BaseFocusScript'>,)), [(, > (,))]] > Looks like you have loaded the same module twice. So you have two versions of your class hierarchies. You can check by printing the ids of your classes. You will get classes with the same name but different ids. Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: Common LISP-style closures with Python
On 4 February 2012 10:14, Antti J Ylikoski wrote: > On 4.2.2012 4:47, Chris Rebert wrote: >> Out of curiosity, what would be non-Common-Lisp-style closures? >> >> Cheers, >> Chris > > > I understand that a "closure" is something which is typical of functional > programming languages. -- Scheme-style closures, for example. > > I don't know Haskell, ML etc. but I do suspect that we could create closures > in those languages as well. Maybe someone more expert than me can help? I think what Chris asking is: what is the feature of Common-Lisp closures that Python closures share but other languages don't? I think what he is implying is that there is no such feature. Python closures are no more "Common-Lisp-style" than they are "Scheme-style" or "Smalltalk-like" or any other language-like. -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: iterating over list with one mising value
On 7 February 2012 20:23, Sammy Danso wrote:
>
> Hi Expert,
> Thanks for your responses and help. thought I should provide more information
> for clarity.
Please don't top-post.
> Please find the error message below for more information
>
> for (key, value) in wordFreq2:
> ValueError: need more than 1 value to unpack
>
> this is a sample of my data
>
> ['with', 3, 'which', 1, [...] , 3, 'other']
>
> What I would like to do is to pad the last value 'other' with a default so I
> can iterate sucessfully
* It would be better if you provided the code that produces the error,
rather than just the error. This would allow others to diagnose your
problem without having to guess what you're really doing (see my guess
below)
* Also, don't truncate the traceback.
My guess: you're running a loop like this, where each item is unpacked:
for (key, value) in wordFreq2:
print key, value
on data like that:
wordFreq2 = ['with', 3, 'which', 1, 'were', 2, 'well', 1]
Your list is flat so the unpacking fails. For it to work, you need
your list to be of the form:
wordFreq2 = [('with', 3), ('which', 1), ('were', 2), ('well', 1)]
Then it will work. The quickest way to transform your list to the
required form is something like this:
def pairs(seq, fillvalue):
it = iter(seq)
return list(izip_longest(it, it, fillvalue=fillvalue)
so you can do:
word_freq_pairs = pairs(wordFreq2)
--
Arnaud
--
http://mail.python.org/mailman/listinfo/python-list
Re: iterating over list with one mising value
On 7 February 2012 22:57, Dennis Lee Bieber wrote:
> On Tue, 7 Feb 2012 21:37:20 +0000, Arnaud Delobelle
> wrote:
>
>
>>
>>Your list is flat so the unpacking fails. For it to work, you need
>>your list to be of the form:
>>
>> wordFreq2 = [('with', 3), ('which', 1), ('were', 2), ('well', 1)]
>>
>>Then it will work. The quickest way to transform your list to the
>>required form is something like this:
>>
> Well... From my viewpoint, the quickest way is to load the list
> correctly in the first place... Where does this list come from? If it's
> a file of
Of course, but how much guessing can you do in a reply? The OP
provided a Python list, that's what I used.
--
Arnaud
--
http://mail.python.org/mailman/listinfo/python-list
Re: Naming convention for in-house modules (Newbie question)
On 9 February 2012 14:00, Laurent Claessens wrote: > >> Here is my question: I would like to start an in-house library of small >> modules to import, for things like error handling/logging. That's easy >> enough, but is there a recommended way of naming such modules? I am >> concerned about avoiding name clashes with standard modules and site >> packages. >> >> Thanks. >> > > This is not 100% an answer to the question, but you should read that : > http://www.python.org/dev/peps/pep-0008/ The OP mentions PEP 8 in the bit of his message that you *don't* quote. -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: round down to nearest number
On 10 February 2012 06:21, Ian Kelly wrote: >>>>> (3219 + 99) // 100 * 100 >> 3300 >>>>> (3289 + 99) // 100 * 100 >> 3300 >>>>> (328678 + 99) // 100 * 100 >> 328700 >>>>> (328 + 99) // 100 * 100 >> 400 >> >> Those are all rounded up to the nearest 100 correctly. > > One thing to be aware of though is that while the "round down" formula > works interchangeably for ints and floats, the "round up" formula does > not. > >>>> (3300.5 + 99) // 100 * 100 > 3300.0 > I'm surprised I haven't seen: >>> 212 - (212 % -100) 300 Here's a function that: * rounds up and down * works for both integers and floats * is only two operations (as opposed to 3 in the solutions given above) >>> def round(n, k): ... return n - n%k ... >>> # Round down with a positive k: ... round(167, 100) 100 >>> round(-233, 100 ... ) -300 >>> # Round up with a negative k: ... round(167, -100) 200 >>> round(-233, -100) -200 >>> # Edge cases ... round(500, -100) 500 >>> round(500, 100) 500 >>> # Floats ... round(100.5, -100) 200.0 >>> round(199.5, 100) 100.0 -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: Read-only attribute in module
On 10 February 2012 03:27, Terry Reedy wrote: > On 2/9/2012 8:04 PM, Steven D'Aprano wrote: > >> Python happily violates "consenting adults" all over the place. We have >> properties, which can easily create read-only and write-once attributes. > > > So propose that propery() work at module level, for module attributes, as > well as for class attributes. I think Steven would like something else: bare names that cannot be rebound. E.g. something like: >>> const a = 42 >>> a = 7 Would raise an exception. Is that right? -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: multiple namespaces within a single module?
On 10 February 2012 00:05, Ethan Furman wrote:
> Ethan Furman wrote:
>>
>> Hrm -- and functions/classes/etc would have to refer to each other that
>> way as well inside the namespace... not sure I'm in love with that...
>
>
>
> Not sure I hate it, either. ;)
>
> Slightly more sophisticated code:
>
>
> class NameSpace(object):
> def __init__(self, current_globals):
> self.globals = current_globals
> self.saved_globals = current_globals.copy()
>
> def __enter__(self):
> return self
> def __exit__(self, *args):
> new_items = []
> for key, value in self.globals.items():
> if (key not in self.saved_globals and value is not self
> or key in self.saved_globals
> and value != self.saved_globals[key]):
>
> new_items.append((key, value))
> for key, value in new_items:
> setattr(self, key, value)
> del self.globals[key]
> self.globals.update(self.saved_globals)
>
> if __name__ == '__main__':
> x = 'inside main!'
> with NameSpace(globals()) as a:
> x = 'inside a?'
> def fn1():
> print(a.x)
> with NameSpace(globals()) as b:
> x = 'inside b?'
> def fn1():
> print(b.x)
> def fn2():
> print('hello!')
> b.fn1()
> y = 'still inside main'
> a.fn1()
> b.fn1()
> print(x)
> print(y)
>
Please! Metaclasses are the obvious way to proceed here :) Then there
is no need for the 'a.x' and 'b.x' cruft.
marigold:junk arno$ cat namespace.py
function = type(lambda:0)
def change_globals(f, g):
return function(f.__code__, g, f.__name__, f.__defaults__, f.__closure__)
class MetaNamespace(type):
def __new__(self, name, bases, attrs):
attrs['__builtins__'] = __builtins__
for name, value in attrs.items():
if isinstance(value, function):
attrs[name] = change_globals(value, attrs)
return type.__new__(self, name, bases, attrs)
class Namespace(metaclass=MetaNamespace):
pass
x = "inside main"
class a(Namespace):
x = "inside a"
def fn1():
print(x)
class b(Namespace):
x = "inside b"
def fn1():
print(x)
def fn2():
print("hello")
fn1()
y = "inside main"
a.fn1()
b.fn1()
b.fn2()
print(x)
print(y)
marigold:junk arno$ python3 namespace.py
inside a
inside b
hello
inside b
inside main
inside main
A bit more work would be needed to support nested functions and closures...
--
Arnaud
--
http://mail.python.org/mailman/listinfo/python-list
Re: when to use import statements in the header, when to use import statements in the blocks where they are used?
On 8 February 2012 01:48, Lei Cheng wrote: > Hi all, > > In a py file, when to use import statements in the header, when to use > import statements in the blocks where they are used? > What are the best practices? > Thanks! Aside from other answers: in some rare cases, importing within a function can avoid circularity problems (e.g. A imports B which tries itself to import A) -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: log and figure out what bits are slow and optimize them.
On 10 February 2012 12:30, sajuptpm wrote: > Hi, > > I want to log time taken to complete database requests inside a method/ > function using decorator . is it possible > I think, i have to inject log code inside the method/fuctions or > modify it. > I wrote a decorator to log taken by a method/function to complete it > execution and its working well. > > My requirement : log everything and figure out what bits are slow and > optimize them. > > What are your suggestions ?? Are you familiar with this? http://docs.python.org/library/profile.html -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: How can I catch misnamed variables?
On 10 February 2012 21:06, John Gordon wrote: > Recently I was been bitten by some stupid errors in my code, and I'm > wondering if there's a simple way to catch them. > > One error was of the form: > > my_object.some_function() > > .. when I hadn't declared an object named "my_object". > > The other error was similar: > > x = my_module.CONSTANT > > .. when I hadn't imported my_module. > > Of course both of these errors were deep inside a long-running function > call, so it took a while for them to crop up. > > Is there an automated way to catch errors like these? I'm using the > compileall module to build my program and it does catch some errors > such as incorrect indentation, but not errors like the above. There's pychecker and pylint -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: Strange Behavior on Python 3 Windows Command Line
On 13 February 2012 19:50, waylan wrote: > When I try running any Python Script on the command line with Python > 3.2 I get this weird behavior. The cursor dances around the command > line window and nothing ever happens. Pressing Ctr+C does nothing. > When I close the window (mouse click on X in top right corner), an > error dialog appears asking me to "force" it to close. I'm not a Windows user, so I can't be of assistance but it may help others if you explained how you installed Python 3.2 on your computer. Also have you tried reinstalling it? > Strangely it was working fine the other day. Then while debugging a > script it suddenly started do this and now does this for every script How were you debugging? -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: name of a sorting algorithm
On 14 February 2012 15:31, Dennis Lee Bieber wrote: > On Tue, 14 Feb 2012 16:01:05 +0100, Jabba Laci > wrote: > >>Could someone please tell me what the following sorting algorithm is called? >> >>Let an array contain the elements a_1, a_2, ..., a_N. Then: >> >>for i = 1 to N-1: >> for j = i+1 to N: >> if a_j < a_i then swap(a_j, a_i) >> > Off hand... The ancient Bubble-Sort... > > http://en.wikipedia.org/wiki/Bubble_sort Ahem... No, it's not Bubble Sort. Bubble sort only swaps adjacent terms. I don't know what this sort is called, if it even has a name. It's a kind of Selection Sort, as each pass it looks for the minimum of the remaining unsorted items. But it ruffles the unsorted list each pass, seemingly to save using an extra register to store the current minumum (there was a time when registers were at a premium). -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: OT: Entitlements [was Re: Python usage numbers]
On 15 February 2012 09:47, Duncan Booth wrote: > Rick Johnson wrote: [...] Perhaps it's a bit presumptuous of me but... It's tempting to react to his inflammatory posts, but after all Rick is a troll and experience shows that trolls are best left alone. Also, please spare a thought for all of us who have him in our killfiles. -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: Interactive keyword help
On 15 February 2012 17:23, Andrew Berg wrote:
> On 2/15/2012 10:04 AM, Mark Lawrence wrote:
>> I didn't realise that this was available until today. It doesn't appear
>> to be prominent in the official docs or have I missed something?
>> Certainly I'd have thought a couple of sentences here
>> http://www.python.org/about/help/ would be justified, what do y'all think?
>>
> help() is a built-in function, not a keyword.
I think he's referring to help *on* keywords, e.g.
>>> help('yield')
--
Arnaud
--
http://mail.python.org/mailman/listinfo/python-list
Re: Wanted: Criticism of code for a Python module, plus a Mac tester
On 16 February 2012 05:03, Ian Kelly wrote: > On Wed, Feb 15, 2012 at 6:11 PM, HoneyMonster > wrote: >> As to your first suggestion though, I am having some difficulty. Note >> that the vulnerability rotates; i.e. CONDITIONS[4] is not the same as >> CONDITIONS[0]. >> Is there a better way of doing it than a simple list.append()? > > Ah, it's more complicated than I realized. I believe this generates > the correct result, although adding None to the dealers list is > somewhat unsatisfactory: > > DEALERS = ["Dealer North", "Dealer East", "Dealer South", "Dealer West", None] > VULNERABILITIES = [ > "Neither vulnerable", "North-South vulnerable", > "East-West vulnerable", "Both vulnerable", > ] > CONDITIONS = [(d, v) for (d, v) in zip(DEALERS * 4, VULNERABILITIES * 5) if d] You could also do: DEALERS = ["Dealer North", "Dealer East", "Dealer South", "Dealer West"] VULNERABILITIES = [ "Neither vulnerable", "North-South vulnerable", "East-West vulnerable", "Both vulnerable", ] CONDITIONS = [ (DEALERS[j], VULNERABILITIES[(i + j)%4]) for i in range(4) for j in range(4) ] If you don't care about the order in which the conditions are listed, you could use CONDITIONS = itertools.product(DEALERS, VULNERABILITIES) (But maybe you do, I haven't looked at the code) -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: TEST AN EXECUTABLE PYTHON SCRIPT SPEED UNDER A PYTHON SHELL
On 16 February 2012 21:10, Prasad, Ramit wrote: >>> When you reply to a known bot, please include some indication of the >>> fact, so we know your message can be ignored as well. > >>Sometimes I wonder about 8. Is there a real person there, as well as the >>bot? A lot of his/its > posts look too intelligent to be computer-generated - or maybe I'm > underestimating the quality of AI. > > I was wondering the exact same thing. I think it may be that what you are underestimating is your ability, as a human being, to create meaning where there is none. -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: question about function pointer
On 17 February 2012 07:53, Zheng Li wrote:
> def method1(a = None):
> print a
>
> i can call it by
> method1(*(), **{'a' : 1})
>
> I am just curious why it works and how it works?
> and what do *() and **{'a' : 1} mean?
>
> when I type *() in python shell, error below happens
>
> File "", line 1
> *()
> ^
> SyntaxError: invalid syntax
It's worth reading the Python tutorial. Here's the relevant section:
http://docs.python.org/tutorial/controlflow.html#unpacking-argument-lists
You could read all of 4.7
--
Arnaud
--
http://mail.python.org/mailman/listinfo/python-list
Re: HTTP logging
On 20 February 2012 16:03, Jason Friedman wrote: > I am logging to HTTP: > > logger.addHandler(logging.handlers.HTTPHandler(host, url)) > > Works great, except if my HTTP server happens to be unavailable: > > socket.error: [Errno 111] Connection refused > > Other than wrapping all my logger.log() calls in try/except blocks, is > there a way to skip logging to the HTTPhandler if the HTTP server is > unavailable? Here's one: subclass HTTPHandler :) -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: sum() requires number, not simply __add__
On 23 February 2012 21:19, Buck Golemon wrote: > I feel like the design of sum() is inconsistent with other language > features of python. Often python doesn't require a specific type, only > that the type implement certain methods. > > Given a class that implements __add__ why should sum() not be able to > operate on that class? It can. You need to pass a second argument which will be the start value. Try help(sum) for details. -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: sum() requires number, not simply __add__
On 23 February 2012 21:23, Buck Golemon wrote: > def sum(values, > base=0): > values = > iter(values) > > try: > result = values.next() > except StopIteration: > return base > > for value in values: > result += value > return result This is definitely not backward compatible. To get something that has a better chance of working with existing code, try this (untested): _sentinel = object() def sum(iterable, start=_sentinel): if start is _sentinel: iterable = iter(iterable) try: start = iterable.next() except StopIteration: return 0 for x in iterable: start += x return start del _sentinel -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: sum() requires number, not simply __add__
On 23 February 2012 21:53, Chris Angelico wrote: > On Fri, Feb 24, 2012 at 8:41 AM, Arnaud Delobelle wrote: >> _sentinel = object() >> >> def sum(iterable, start=_sentinel): >> if start is _sentinel: >> >> del _sentinel > > Somewhat off-topic: Doesn't the if statement there do a lookup for a > global, which would mean that 'del _sentinel' will cause it to fail? > Or have I missed something here? Yes, you're right :) Change the signature to def sum(iterable, start=_sentinel, _sentinel=_sentinel): This is not pretty... -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: sum() requires number, not simply __add__
On 23 February 2012 22:04, Chris Angelico wrote:
> On Fri, Feb 24, 2012 at 8:59 AM, Arnaud Delobelle wrote:
>> def sum(iterable, start=_sentinel, _sentinel=_sentinel):
>
> Is this a reason for Python to introduce a new syntax, such as:
>
> def foo(blah, optional=del):
> if optional is del: print("No argument was provided")
>
> Basically, 'del' is treated like a unique non-providable object, only
> possible in an argument list and only if the argument was omitted. No
> more proliferation of individual sentinels... what do you think?
The problem with these proposals is to avoid the leakage of 'del'.
Here you could do:
def get_del(x=del):
return x
And then you're in trouble again.
--
Arnaud
--
http://mail.python.org/mailman/listinfo/python-list
Re: Does turtledemo in Python 3.2 actually work?
On 24 February 2012 12:25, Steven D'Aprano wrote: > Python 3.2 includes turtledemo, a demonstration program for the turtle > module. > > When I run it, I can load the turtle scripts, and the GUI application > says "Press the start button", but there is no start button. > > Can anyone else confirm this as a bug? > > http://docs.python.org/py3k/library/turtle.html#demo-scripts Just tested with Python 3.2.1 on Mac OS X 10.6.8 and all seems fine. Perhaps if you say which platform it's failing on, others will be able to reproduce the failure on the same platform? -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: A quirk/gotcha of for i, x in enumerate(seq) when seq is empty
On 24 February 2012 14:54, Steven D'Aprano wrote: > for...else is a very useful construct, but the name is misleading. It > took me a long time to stop thinking that the else clause executes when > the for loop was empty. This is why I think we should call this construct "for / break / else" rather than "for / else". -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: PyWart: Language missing maximum constant of numeric types!
On 26 February 2012 13:38, Wolfgang Meiners wrote: > do_it = (len(str) <= maxlength) if maxlength is not None else True That's a funny way to spell: do_it = maxlength is None or len(str) <= maxlength -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: Listing children processes
On 28 February 2012 21:39, Mihai Badoiu wrote: > On Tue, Feb 28, 2012 at 4:35 PM, Chris Rebert wrote: >> >> On Tue, Feb 28, 2012 at 10:33 AM, Mihai Badoiu wrote: >> > I'm trying to compute the total CPU load of an external process and it's >> > children. (so I cannot use resource.getrusage) For the load of the >> > process >> > I can just grab it from /proc/X/stat. How do I get the CPU load of the >> > children processes? Is there an easy way to get a list of the children >> > processes? >> >> http://code.google.com/p/psutil/ >> >> Cheers, >> Chris > Looked at that before. psutil doesn't do children. > > --mihai Please don't top-post! Also, psutil.Process.get_children() looks to me like it "does" children. -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: exec
On 1 March 2012 13:07, Rolf Wester wrote: > Hi, > > I would like to define methods using exec like this: > > class A: > def __init__(self): > cmd = "def sqr(self, x):\n return x**2\nself.sqr = sqr\n" > exec cmd > a = A() > print a.sqr(a, 2) > > This works, but I have to call sqr with a.sqr(a, 2), a.sqr(2) does not work > (TypeError: sqr() takes exactly 2 arguments (1 given)). I'm curious to know your motivation for doing this. -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: decompilation
ntListToStringList(list(range(1,
27))), 'How many guesses will you need? > ')
m = int(reply)
availableLetters = [True]*26
for i in range(m):
guess, availableLetters = getNewLetterGuess(availableLetters)
wordFamilyCounter, familyIndexList = getWordFamilyCounter(L, n, guess)
bestFamily = wordFamilyCounter.index(max(wordFamilyCounter))
if bestFamily == 0:
print('Letter not in word.')
else:
print('Letter is in word!!!')
L = extractLargestFamily(L, familyIndexList, wordFamilyCounter)
patternList = updatePatternList(patternList, guess, bestFamily)
print(''.join(patternList))
if '_ ' not in patternList:
break
if '_ ' not in patternList:
print('SURELY you must be CHEATING, but you guessed my word in
' + str(i + 1) + ' tries!!!')
else:
bogusWord = pickWordFrom(L)
print('You lose. The word was: ' + bogusWord)
>>>
I haven't actually checked if this code runs :)
--
Arnaud
--
http://mail.python.org/mailman/listinfo/python-list
Re: Jython callable. How?
On Mar 4, 2012 9:04 AM, "Sirotin Roman" wrote: > > Hi. > How exactly jython decides is object callable or not? I defined > __call__ method but interpreter says it's still not callable. > BTW, my code works in cpython It will help if you show us the code. -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: Fast file data retrieval?
On 12 March 2012 19:39, Virgil Stokes wrote: > I have a rather large ASCII file that is structured as follows > > header line > 9 nonblank lines with alphanumeric data > header line > 9 nonblank lines with alphanumeric data > ... > ... > ... > header line > 9 nonblank lines with alphanumeric data > EOF > > where, a data set contains 10 lines (header + 9 nonblank) and there can be > several thousand > data sets in a single file. In addition, each header has a unique ID code. > > Is there a fast method for the retrieval of a data set from this large file > given its ID code? It depends. I guess if it's a long running application, you could load up all the data into a dictionary at startup time (several thousand data sets doesn't sound like that much). Another option would be to put all this into a dbm database file (http://docs.python.org/library/dbm.html) - it would be very easy to do. Or you could have your own custom solution where you scan the file and build a dictionary mapping keys to file offsets, then when requesting a dataset you can seek directly to the correct position. -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: Style question (Poll)
On 14 March 2012 20:37, Croepha wrote: > Which is preferred: > > for value in list: > if not value is another_value: > value.do_something() > break > > --or-- > > if list and not list[0] is another_value: > list[0].do_something() Hard to say, since they don't do the same thing :) I suspect you meant: for value in list: if not value is another_value: value.do_something() break I always feel uncomfortable with this because it's misleading: a loop that never loops. -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: Style question (Poll)
On 14 March 2012 22:15, Prasad, Ramit wrote: > Only use 'is' if you are looking for objects like True, > False, None or something that MUST be exactly the same object. I've rarely seen valid uses of 'is True' or 'is False'. -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: Python is readable
On 14 March 2012 23:34, Kiuhnm wrote:
> I've just started to read
> The Quick Python Book (2nd ed.)
> The author claims that Python code is more readable than Perl code and
> provides this example:
>
> --- Perl ---
> sub pairwise_sum {
> my($arg1, $arg2) = @_;
> my(@result) = ();
> @list1 = @$arg1;
> @list2 = @$arg2;
> for($i=0; $i < length(@list1); $i++) {
> push(@result, $list1[$i] + $list2[$i]);
> }
> return(\@result);
> }
>
> --- Python ---
> def pairwise_sum(list1, list2):
> result = []
> for i in range(len(list1)):
> result.append(list1[i] + list2[i])
> return result
> --- ---
>
> It's quite clear that he knows little about Perl.
> Here's what I would've written:
>
> sub pairwise_sum {
> my ($list1, $list2) = @_;
> my @result;
> push @result, $list1->[$_] + $list2->[$_] for (0..@$list1-1);
> \@result;
> }
>
> Having said that, the Python code is still more readable, so there's no need
> to misrepresent Perl that way.
> Now I'm wondering whether the author will show me "good" or "bad" Python
> code throughout the book. Should I keep reading?
I don't know this book and there may be a pedagogical reason for the
implementation you quote, but pairwise_sum is probably better
implemented in Python 3.X as:
def pairwise_sum(list1, list2):
return [x1 + x2 for x1, x2 in zip(list1, list2)]
Or in Python 2.X:
from itertools import izip
def pairwise_sum(list1, list2):
return [x1 + x2 for x1, x2 in izip(list1, list2)]
Or even:
from operator import add
def pairwise_sum(list1, list2):
return map(add, list1, list2)
--
Arnaud
--
http://mail.python.org/mailman/listinfo/python-list
Re: Python is readable
On 15 March 2012 00:27, Chris Angelico wrote: > On Thu, Mar 15, 2012 at 10:54 AM, Arnaud Delobelle wrote: >> I don't know this book and there may be a pedagogical reason for the >> implementation you quote, but pairwise_sum is probably better >> implemented in Python 3.X as: >> >> def pairwise_sum(list1, list2): >> return [x1 + x2 for x1, x2 in zip(list1, list2)] > > Okay, here's something for debate. > > Should the readability of a language be gauged on the basis of its > standard library, or should you be comparing actual code? But here's the code posted by the OP: --- Python --- def pairwise_sum(list1, list2): result = [] for i in range(len(list1)): result.append(list1[i] + list2[i]) return result --- --- The code I posted uses one builtin function (zip), the code posted by the OP uses two (range and len). Neither uses the standard library. -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: Python is readable
On 15 March 2012 22:35, Ben Finney wrote:
> Kiuhnm writes:
>
>> Moreover, I think that
>> if (
>>
>> ):
>>
>>
>>
>> is not very readable anyway.
>
> I agree, and am glad PEP 8 has been updated to recommend an extra level
> of indentation for continuation, to distinguish from the new block that
> follows http://www.python.org/dev/peps/pep-0008/#indentation>.
Personally I solve this by never writing if conditions that span more
than one line. If the worst comes to the worst, I would write:
aptly_named_condition = (
very long condition
that goes over
plenty of lines
)
if aptly_named_condition:
do stuff
--
Arnaud
--
http://mail.python.org/mailman/listinfo/python-list
Re: Currying in Python
On 19 March 2012 23:20, Ian Kelly wrote:
> I hope you don't mind if I critique your code a bit!
>
> On Fri, Mar 16, 2012 at 7:21 PM, Kiuhnm
> wrote:
>> Here we go.
>>
>> --->
>> def genCur(f, unique = True, minArgs = -1):
>
> It is customary in Python for unsupplied arguments with no default to
> use the value None, not -1. That's what it exists for.
>
>> """ Generates a 'curried' version of a function. """
>> def geng(curArgs, curKwargs):
>> def g(*args, **kwargs):
>> nonlocal f, curArgs, curKwargs, minArgs; # our STATIC data
I don't know if all the rest of the code is below, but this line above
would only be necessary if you want to rebind f, curArgs, minArgs.
You don't seem to do it, so I think this line is unnecessary.
Also, your naming of variables disagrees with PEP 8 :)
>> if len(args) or len(kwargs):
>
> Collections evaluate as true if they are not empty, so this could just be:
>
> if args or kwargs:
>
>> # Allocates data for the next 'g'. We don't want to modify our
>> # static data.
>> newArgs = curArgs[:];
Semicolon to end a statement?
>> newKwargs = dict.copy(curKwargs);
>>
>> # Adds positional arguments.
>> newArgs += args;
>>
>> # Adds/updates keyword arguments.
>> if unique:
>> # We don't want repeated keyword arguments.
>> for k in kwargs.keys():
>> if k in newKwargs:
>> raise(Exception("Repeated kw arg while unique =
>> True"));
>> newKwargs.update(kwargs);
>
> Since you're writing this for Python 3 (as evidenced by the use of the
> nonlocal keyword), you could take advantage here of the fact that
> Python 3 dictionary views behave like sets. Also, you should use a
> more specific exception type:
>
> if unique and not kwargs.keys().isdisjoint(newKwargs):
> raise ValueError("A repeated keyword argument was
> supplied")
>
>> # Checks whether it's time to evaluate f.
>> if minArgs >= 0 and minArgs <= len(newArgs) + len(newKwargs):
>
> With minArgs defaulting to None, that would be:
>
> if minArgs is not None and minArgs <= len(newArgs) +
> len(newKwargs):
>
>> return f(*newArgs, **newKwargs); # f has enough args
>> else:
>> return geng(newArgs, newKwargs); # f needs some more
>> args
>> else:
>> return f(*curArgs, **curKwargs); # the caller forced the
>> evaluation
>> return g;
>> return geng([], {});
>>
>> def cur(f, minArgs = -1):
>> return genCur(f, True, minArgs);
>>
>> def curr(f, minArgs = -1):
>> return genCur(f, False, minArgs);
>
> The names "cur" and "curr" are terrible. Good names should describe
> what the function does without being too onerous to type, and the
> addition of the duplicate "r" is not an obvious mnemonic for
> remembering that the first one prohibits duplicate keyword arguments
> and the second one allows them. Why not more descriptive names like
> "curry" and "curry_unique"?
>
> That's all I've got. All in all, it's pretty decent for a Python newbie.
>
> Cheers,
> Ian
> --
> http://mail.python.org/mailman/listinfo/python-list
--
http://mail.python.org/mailman/listinfo/python-list
Re: Odd strip behavior
On Mar 22, 2012 7:49 PM, "Rodrick Brown" wrote: > > #!/usr/bin/python > > def main(): > >str1='this is a test' >str2='t' > >print "".join([ c for c in str1 if c not in str2 ]) >print(str1.strip(str2)) > > if __name__ == '__main__': >main() > > ./remove_str.py > his is a es > his is a tes > > Why wasnt the t removed ? Try help(ste.strip) -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: Odd strip behavior
On 22 March 2012 20:04, Rodrick Brown wrote: > > On Mar 22, 2012, at 3:53 PM, Arnaud Delobelle wrote: > Try help(ste.strip) > > It clearly states "if chars is given and not None, remove characters in > chars instead. > > Does it mean remove only the first occurrence of char? That's the behavior > I'm seeing. The key words in the docstring are "leading and trailing" -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Gotcha's?
On 5 April 2012 21:06, Emile van Sebille wrote:
> Kind of begs for a contains method that returns the appropriate boolean:
>
> if text.contains('bob')
It's already there:
text.__contains__('bob')
It's usually spelt otherwise though:
'bob' in text
--
Arnaud
--
http://mail.python.org/mailman/listinfo/python-list
Re: Zipping a dictionary whose values are lists
On 13 April 2012 17:35, Kiuhnm wrote: > On 4/13/2012 17:58, Alexander Blinne wrote: >> >> zip(*[x[1] for x in sorted(d.items(), key=lambda y: y[0])]) > > Or > zip(*[d[k] for k in sorted(d.keys())]) .keys() is superfluous here: zip(*(d[k] for k in sorted(d))) -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: Making helper methods more concise
On 16 April 2012 13:01, Alan Ristow wrote:
> Hi all,
>
> I have defined a class that includes a number of helper methods that
> are useful to me, but pretty redundant. Something like so, where I
> maintain a list of tuples:
>
> class A(object):
> def __init__(self):
> self.listing = []
>
> # This method does the work.
> def append_text(self, text, style):
> self.listing.append((text, style))
>
> # The rest of the methods are just helpers.
> def append_paragraph(self, text):
> self.append_text(text, 'Paragraph')
>
> def append_header(self, text):
> self.append_text(text, 'Header')
>
> def append_title(self, text):
> self.append_title(text, 'Title')
>
> obj = A()
> obj.append_title('On Learning Something New About Python')
> obj.append_header('A Question Is Posed')
> obj.append_paragraph('Where is lorem ipsum when you need it?')
>
>
> To me, this code is simple, clear, and easy to maintain, but
> incredibly redundant. That's fine, I have no problem with that, but it
> got me thinking: Is there a way to make the code more concise?
>
> I know that concise is not necessarily better, but I am asking
> primarily to educate myself rather than to improve this particular
> piece of code -- the code only inspired the question. I'm trying to
> wrap my head around decorators (aside from the simplest ones) and
> metaclasses at the moment, so feel free to throw those sorts of
> solutions into the mix if something like that would be appropriate.
You can do this (untested), but no doubt it won't be to everybody's taste:
class A(object):
def __init__(self):
self.listing = []
# This method does the work.
def append_text(self, text, style):
self.listing.append((text, style))
# The rest of the methods are just helpers.
for style in 'paragraph', 'header', 'title':
exec """def append_%s(self, text):
self.append_text(text, "%s")""" % (style, style.capitalize())
del style
--
Arnaud
--
http://mail.python.org/mailman/listinfo/python-list
Re: Making helper methods more concise
On 16 April 2012 13:29, Arnaud Delobelle wrote: > You can do this (untested), but no doubt it won't be to everybody's taste: > > class A(object): > def __init__(self): > self.listing = [] > > # This method does the work. > def append_text(self, text, style): > self.listing.append((text, style)) > > # The rest of the methods are just helpers. > for style in 'paragraph', 'header', 'title': > exec """def append_%s(self, text): > self.append_text(text, "%s")""" % (style, style.capitalize()) > del style Oops I sent too quickly. Another way would be class A(object): def __init__(self): self.listing = [] # This method does the work. def append_text(self, text, style): self.listing.append((text, style)) for style in 'paragraph', 'header', 'title': setattr(A, 'append_' + style, lambda s, t, style=style: A.append_text(s, t, style)) Untested too :) Cheers, Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: Framework for a beginner
On 17 April 2012 09:54, Bryan wrote: > Django has emphasized backwards compatibility with the > down-side that, last I heard, there was no plan to move to Python 3. Not quite: https://www.djangoproject.com/weblog/2012/mar/13/py3k/ -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: why () is () and [] is [] work in other way?
On 26 April 2012 12:42, Adam Skutt wrote: > On Apr 26, 5:10 am, Steven D'Aprano [email protected]> wrote: >> On Wed, 25 Apr 2012 20:50:21 -0700, Adam Skutt wrote: >> > On Apr 25, 8:01 pm, Steven D'Aprano > > [email protected]> wrote: >> >> On Wed, 25 Apr 2012 13:49:24 -0700, Adam Skutt wrote: >> >>> [Sterile pedantry] >> >> [More sterile pedantry] >> > [Yet more sterile pedantry] >> [And more] > [Record breaking levels of sterile pedantry] Please stop! Steven, I've learnt a lot from your posts on this list over the years, but too often you spoil it with your compulsion to have the last word on every argument you get involved in at any cost. Some arguments aren't worth winning... -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: confusing doc: mutable and hashable
(sent from my phone) On Apr 28, 2012 7:36 PM, "Chris Rebert" wrote: > Correct. Pedantically, you can define __hash__() on mutable objects; > it's just not very useful or sensible, so people generally don't. I find it's fine to define __hash__ on mutable objects as long as __eq__ only relies on immutable state (and then so should __hash__ of course). A typical example would be an object that does some caching. -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: syntax for code blocks
(sent from my phone) On May 1, 2012 6:42 PM, "Jerry Hill" wrote: > > On Tue, May 1, 2012 at 1:07 PM, Kiuhnm > wrote: > > If you had read the module's docstring you would know that the public > > version uses > > Are you aware that you've never posted a link to your module, nor it's > docstrings? Are you also aware that your module is essentially > unfindable on google? Certainly nothing on the first two pages of > google results for 'python codeblocks' jumps out at me as a python > module of that name. > > Perhaps you would have better luck if you either post the actual code > you want people to critique, or posted a link to that code. He did post a link to a blog post describing his module and also a link to the actual code, on bitbucket IIRC. Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: return respective values when mutiple keys are passed in dictionary
On 7 May 2012 12:31, Nikhil Verma wrote:
> HI All
>
> I was clearing my concepts on dictionary and stuck in this problem.
> I have a dictionary which i have formed by using zip function on two list so
> that one list (which i have hardcoded) becomes the keys and the other list
> becomes its values.
>
> Now i want to know how can i get the values of keys at once if i pass the
> keys in a dictionary.
>
> Let say I have a dictionary
>
> mydict = {'a':'apple' , 'b':'boy' ,'c' : 'cat', 'd':'duck','e':'egg'}
>
> Now if i do :-
>
> mydict.get('a')
> 'apple'
mydict['a'] is the usual way to get the value associated with a key.
The difference is that it will throw an exception if the key doesn't
exist, which is most of the time the sanest thing to do.
> What i want is some i pass keys in get and in return i should have all the
> values of those keys which i pass.
>
> ##
> mydict.get('a','b','c') ###demo for what i want
> 'apple','boy','cat' ### Output i want
> #
1. You can use a list comprehension
>>> [mydict[k] for k in 'a', 'b', 'c']
['apple', 'boy', 'cat']
2. You can use map (for python 3.X, you need to wrap this in list(...))
>>> map(mydict.__getitem__, ['a', 'b', 'c'])
['apple', 'boy', 'cat']
3. You can use operator.itemgetter
>>> from operator import itemgetter
>>> itemgetter('a', 'b', 'c')(mydict)
('apple', 'boy', 'cat')
--
Arnaud
--
http://mail.python.org/mailman/listinfo/python-list
Re: Good data structure for finding date intervals including a given date
On 13 May 2012 13:29, Alec Taylor wrote: > There is an ordered dict type since Python 3.1[1] and Python 2.7.3[2]. I don't think that'll help the OP. Python's OrderedDict keeps track of the order in which the keys were inserted into the dictionary (a bit like a list), it doesn't keep the keys sorted. > If you are looking for the best possible self-sorting structure for > searching, then perhaps you are looking for what's outlined in the > 2002 article by Han & Thorup: Integer Sorting in O(n sqrt(log log n)) > Expected Time and Linear Space[3]. I can't access it but it seems to me it's not about self sorted data structures, which is what the OP is looking for. -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: Carbon Event Manager (Carbon.CarbonEvt module) - working?
On 15 May 2012 20:55, Ned Deily wrote: > In article > , > msmucr wrote: >> i would like to ask you for some information regarding Carbon Event >> Manager ( Carbon.CarbonEvt ) library in Python. >> I need to recieve and work with few Carbon events in my program. I've >> followed some examples on PyObjC site, but wasn't successful. I know, >> that both Carbon library and this Python module is deprecated, but >> frankly i didn't find any usable alternative except of writing of >> something from scratch.. >> I ended on basic import of required objects from Carbon.CarbonEvt >> module - >> like "from Carbon.CarbonEvt import RegisterEventHotKey" >> I tried it in Python 2.7.1 (standard distribution in OS X 10.7 Lion) >> and Python 2.6.1 (standard Apple distribution in OS X 10.6). >> Do I have something wrong or is it simply broken and unmaintained now? > > You may want to ask OS X specific questions on the macpython mailing > list (http://www.python.org/community/sigs/current/pythonmac-sig/). The > main reason that these modules are deprecated (and have been removed in > Python 3) is because they depend on obsolete APIs in OS X that are only > available in 32-bit versions. Apple has made it clear that they will > not be made available as 64-bit and will eventually go away. You should > try to find ways not to use the Carbon model in your applications. That > said, you can use these modules, even with the Apple-supplied Pythons in > OS X 10.6 and 10.7, if you force Python to run in 32-bit-mode. For > those Apple-supplied Pythons, see the Apple man page (man 1 python) for > system Python specific ways to force 32-bit mode persistently. Another > way that should work for any OS X universal Python 2.7.x: > > arch -i386 python2.7 This is what I have with system python 2.6: $ cat ~/bin/python_32 #! /bin/bash export VERSIONER_PYTHON_PREFER_32_BIT=yes /usr/bin/python "$@" I use it for wxpython, which only seems to work in 32 bit mode. I can't remember where I found it. Maybe I read the man page? -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: How to generate only rotationally-unique permutations?
On 19 May 2012 06:23, John O'Hagan wrote: > To revisit a question which I'm sure none of you remember from when I posted > it > a year or so ago - there were no takers at the time - I'd like to try again > with > a more concise statement of the problem: > > How to generate only the distinct permutations of a sequence which are not > rotationally equivalent to any others? More precisely, to generate only the > most > "left-packed" of each group of rotationally equivalent permutations, such that > for each permutation p: This makes me think of Lyndon words. A Lyndon word is a word which is the only lexicographical minimum of all its rotations. There is a very effective way of generating Lyndon words of length <= n. It may be possible to adapt it to what you want (obviously as Lyndon words are aperiodic you'd have to generate Lyndon word of length d | n when suitable). -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: Help doing it the "python way"
On 24 May 2012 21:22, Scott Siegler wrote: > Hello, > > I am an experienced programmer but a beginner to python. As such, I can > figure out a way to code most algorithms using more "C" style syntax. > > I am doing something now that I am sure is a more python way but i can't > quite get it right. I was hoping someone might help. > > So I have a list of grid coordinates (x, y). From that list, I want to > create a new list that for each coordinate, I add the coordinate just above > and just below (x,y+1) and (x,y-1) > > right now I am using a for loop to go through all the coordinates and then > separate append statements to add the top and bottom. > > is there a way to do something like: [(x,y-1), (x,y+1) for zzz in coord_list] > or something along those lines? AFAICS nobody's suggested yet the simple yet effective: new_list = [(x, y + i) for x, y in coord_list for i in (-1, 1)] IMHO these list comprehensions are often overlooked too quickly in favour of itertools (in this case chain.from_iterable). -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: python3 raw strings and \u escapes
On 30 May 2012 12:54, Thomas Rachel wrote: > There is a 3rd one: use r'[ ' + '\u3000' + ']'. Not very nice to read, but > should do the trick... You could even take advantage of string literal concatenation:) r'[' '\u3000' r']' -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: Pythonic cross-platform GUI desingers à la Interface Builder (Re: what gui designer is everyone using)
On 10 June 2012 07:16, rusi wrote: > This is worth a read in this context: http://osteele.com/archives/2004/11/ides Interesting! I definitely fall nicely at one extreme of this dichotomy. Every time I've tried to use an IDE, it's made me feel inadequate and I've quickly retreated to my comfort zone (emacs + xterm). I felt inadequate because I felt like the IDE was hindering me rather than helping me. All I ask from the program that I use to write code is: * syntax highlighting * sensible auto-indenting * as little reliance on the mouse as possible * emacs key bindings :) This article makes me feel more positive about my inability to feel comfortable in an IDE. Thanks for the link! -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: Problem with ImapLib and subject in French
On 18 June 2012 12:31, Valentin Mercier wrote:
> Hi,
>
> I'm trying to search some mails with SUBJECT criteria, but the problem is
> the encoding, I'm trying to search french terms (impalib and python V2.7)
>
> I've tried few things, but I think the encoding is the problem, in my mail
> header I have something like this:
>
> =?iso-8859-1?Q?Job:_"Full_Backup_Les_Gr=E8ves_du_Lac")_?=
>
> I need to search unseen mail with the FROM and the SUBJECT criteria. I take
> those two in parameters in my python script and the second contain special
> caracters like é or è, so I encode the string like this:
>
> temp = header_encode(unicode('Les grèves','utf-8'),
> charset='iso-8859-1')
>
> And the string is exactly the same as the header:
>
> =?iso-8859-1?q?Full_Backup_Les_Gr=E8ves?=
>
> But the search function doesn't find my email, and I don't know why, even if
> I try with the entire string of the subject.
Can you post the code that doesn't work? It's hard to debug "search
function doesn't find my email". It would be easier to debug actual
code (note: I'm not a user of imaplib so I'm not promising anything :)
--
Arnaud
--
http://mail.python.org/mailman/listinfo/python-list
Getting a module's code object
Hi all, In Python 3, a function f's code object can be accessed via f.__code__. I'm interested in getting a module's code object, i.e. the code that is executed when the module is run. I don't think it's accessible via the module object itself (although I would be glad if somebody proved me wrong :). In the marshal module docs [1] it is mentioned that: """ The marshal module exists mainly to support reading and writing the “pseudo-compiled” code for Python modules of .pyc files. """ So it seems that the module's code object is marshalled into the .pyc file - so there may be a way to unmarshal it - but I can't easily find information about how to do this. Is this a good lead, or is there another way to obtained a module's code object? Thanks -- Arnaud [1] http://docs.python.org/py3k/library/marshal.html -- http://mail.python.org/mailman/listinfo/python-list
Re: Run time default arguments
On Aug 25, 3:30 pm, [email protected] wrote: > What is the most common way to handle default function arguments that > are set at run time, rather than at compile time? The snippet below is > the current technique that I've been using, but it seems inelegant. > > defaults = { 'debug' : false } > def doSomething (debug = None): > debug = debug if debug != None else defaults['debug'] > # blah blah blah You're close to the usual idiom: def doSomething(debug=None): if debug is None: debug = defaults['debug'] ... Note the use of 'is' rather than '==' HTH -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: Getting a module's code object
On 25 August 2011 16:07, Peter Otten <[email protected]> wrote: > Arnaud Delobelle wrote: > >> In Python 3, a function f's code object can be accessed via f.__code__. >> >> I'm interested in getting a module's code object, [...] > > Taken from pkgutil.py: > > def read_code(stream): > # This helper is needed in order for the PEP 302 emulation to > # correctly handle compiled files > import marshal > > magic = stream.read(4) > if magic != imp.get_magic(): > return None > > stream.read(4) # Skip timestamp > return marshal.load(stream) This works fine, thanks a lot! > Alternatively you can compile the source yourself: > > module = compile(source, filename, "exec") I don't necessarily have access to the source file. -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
The RAISE_VARARGS opcode in Python 3
Hi all, Here is an extract from the dis module doc [1] """ RAISE_VARARGS(argc) Raises an exception. argc indicates the number of parameters to the raise statement, ranging from 0 to 3. The handler will find the traceback as TOS2, the parameter as TOS1, and the exception as TOS. """ OTOH, looking at PEP 3109: """ In Python 3, the grammar for raise statements will change from [2] raise_stmt: 'raise' [test [',' test [',' test]]] to raise_stmt: 'raise' [test] """ So it seems that RAISE_VARARGS's argument can only be 0 or 1 in Python 3, whereas it could be up to 3 in Python 2. Can anyone confirm that this is the case? In this case, I guess the dis docs need to be updated. Thank you, Arnaud [1] http://docs.python.org/py3k/library/dis.html#opcode-RAISE_VARARGS [2] http://www.python.org/dev/peps/pep-3109/#grammar-changes -- http://mail.python.org/mailman/listinfo/python-list
Re: The RAISE_VARARGS opcode in Python 3
On 27 August 2011 07:49, Peter Otten <[email protected]> wrote: > Arnaud Delobelle wrote: > >> Here is an extract from the dis module doc [1] >> >> """ >> RAISE_VARARGS(argc) >> Raises an exception. argc indicates the number of parameters to the >> raise statement, ranging from 0 to 3. The handler will find the >> traceback as TOS2, the parameter as TOS1, and the exception as TOS. >> """ >> >> OTOH, looking at PEP 3109: >> >> """ >> In Python 3, the grammar for raise statements will change from [2] >> >> raise_stmt: 'raise' [test [',' test [',' test]]] >> to >> >> raise_stmt: 'raise' [test] >> """ >> >> So it seems that RAISE_VARARGS's argument can only be 0 or 1 in Python >> 3, whereas it could be up to 3 in Python 2. > > It can be up to 2 in Python 3, > >>>> help("raise") > The ``raise`` statement > *** > > raise_stmt ::= "raise" [expression ["from" expression]] > ... > > confirmed by a quick look into ceval.c: > > TARGET(RAISE_VARARGS) > v = w = NULL; > switch (oparg) { > case 2: > v = POP(); /* cause */ > case 1: > w = POP(); /* exc */ > case 0: /* Fallthrough */ > why = do_raise(w, v); > break; > default: > PyErr_SetString(PyExc_SystemError, > "bad RAISE_VARARGS oparg"); > why = WHY_EXCEPTION; > break; > } > break; Thanks again, Peter! I'm out of Python practice, and I've forgotten some things like help("keyword"). Also PEP 3109 does indeed mention the raise .. from .. syntax in an example at the end. -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
how to format long if conditions
Hi all, I'm wondering what advice you have about formatting if statements with long conditions (I always format my code to <80 colums) Here's an example taken from something I'm writing at the moment and how I've formatted it: if (isinstance(left, PyCompare) and isinstance(right, PyCompare) and left.complist[-1] is right.complist[0]): py_and = PyCompare(left.complist + right.complist[1:]) else: py_and = PyBooleanAnd(left, right) What would you do? -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: how to format long if conditions
On 27 August 2011 08:24, Steven D'Aprano wrote: > Arnaud Delobelle wrote: > >> Hi all, >> >> I'm wondering what advice you have about formatting if statements with >> long conditions (I always format my code to <80 colums) >> >> Here's an example taken from something I'm writing at the moment and >> how I've formatted it: >> >> >> if (isinstance(left, PyCompare) and isinstance(right, PyCompare) >> and left.complist[-1] is right.complist[0]): >> py_and = PyCompare(left.complist + right.complist[1:]) >> else: >> py_and = PyBooleanAnd(left, right) >> >> What would you do? > > I believe that PEP 8 now suggests something like this: > > if ( > isinstance(left, PyCompare) and isinstance(right, PyCompare) > and left.complist[-1] is right.complist[0]): > ) > py_and = PyCompare(left.complist + right.complist[1:] > else: > py_and = PyBooleanAnd(left, right) > > > I consider that hideous and would prefer to write this: > > > if (isinstance(left, PyCompare) and isinstance(right, PyCompare) > and left.complist[-1] is right.complist[0]): > py_and = PyCompare(left.complist + right.complist[1:] > else: > py_and = PyBooleanAnd(left, right) > > > Or even this: > > tmp = ( > isinstance(left, PyCompare) and isinstance(right, PyCompare) > and left.complist[-1] is right.complist[0]) > ) > if tmp: > py_and = PyCompare(left.complist + right.complist[1:] > else: > py_and = PyBooleanAnd(left, right) > > > But perhaps the best solution is to define a helper function: > > def is_next(left, right): > """Returns True if right is the next PyCompare to left.""" > return (isinstance(left, PyCompare) and isinstance(right, PyCompare) > and left.complist[-1] is right.complist[0]) > # PEP 8 version left as an exercise. > > > # later... > if is_next(left, right): > py_and = PyCompare(left.complist + right.complist[1:] > else: > py_and = PyBooleanAnd(left, right) > Thanks Steven and Hans for you suggestions. For this particular instance I've decided to go for a hybrid approach: * Add two methods to PyCompare: class PyCompare(PyExpr): ... def extends(self, other): if not isinstance(other, PyCompare): return False else: return self.complist[0] == other.complist[-1] def chain(self, other): return PyCompare(self.complist + other.complist[1:]) * Rewrite the if as: if isinstance(right, PyCompare) and right.extends(left): py_and = left.chain(right) else: py_and = PyBooleanAnd(left, right) The additional benefit is to hide the implementation details of PyCompare (I suppose this could illustrate the thread on when to create functions). -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: Returning a value from exec or a better solution
On 29 August 2011 23:14, Jack Trades wrote:
> On Mon, Aug 29, 2011 at 12:30 PM, Rob Williscroft wrote:
>>
>> Jack Trades wrote in
>> > ... I wanted to allow the user to manually return the
>> > function from the string, like this:
>> >
>> > a = exec("""
>> > def double(x):
>> > return x * 2
>> > double
>> > """)
>> >
>> > However it seems that exec does not return a value as it produces a
>> > SyntaxError whenever I try to assign it.
>>
>> def test():
>> src = (
>> "def double(x):"
>> " return x * 2"
>> )
>> globals = {}
>> exec( src, globals )
>> return globals[ "double" ]
>>
>> print( test() )
>
> I looked into doing it that way but it still requires that the user use a
> specific name for the function they are defining. The docs on exec say that
> an implementation may populate globals or locals with whatever they want so
> that also rules out doing a simple "for item in globals", as there may be
> more than just the one function in there (though I suppose I may be able to
> work around that).
Hi Jack,
Here is a possible solution for your problem (Python 3):
>>> class CapturingDict(dict):
... def __setitem__(self, key, val):
... self.key, self.val = key, val
... dict.__setitem__(self, key, val)
...
>>> c = CapturingDict()
>>> exec("def myfunction(x): return 1", c)
>>> c.key
'myfunction'
>>> c.val
HTH,
--
Arnaud
--
http://mail.python.org/mailman/listinfo/python-list
Re: killing a script
On 29 August 2011 04:08, Russ P. wrote: > On Aug 28, 7:51 pm, Chris Angelico wrote: >> On Mon, Aug 29, 2011 at 12:41 PM, Russ P. wrote: >> > On Aug 28, 6:52 pm, MRAB wrote: >> >> You could look at the return value of os.system, which may tell you the >> >> exit status of the process. >> >> > Thanks for the suggestion. Yeah, I guess I could do that, but it seems >> > that there should be a simpler way to just kill the "whole enchilada." >> > Hitting Control-C over and over is a bit like whacking moles. >> >> I believe the idea of this suggestion is for the outer script to >> notice that the inner script terminated via Ctrl-C, and would then >> immediately choose to terminate itself - thus avoiding the >> whack-a-mole effect. >> >> ChrisA > > Yes, but if I am not mistaken, that will require me to put a line or > two after each os.system call. That's almost like whack-a-mole at the > code level rather than the Control-C level. OK, not a huge deal for > one script, but I was hoping for something simpler. I was hoping I > could put one line at the top of the script and be done with it. Write a function! That's what they're for after all :) -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: Returning a value from exec or a better solution
On 30 August 2011 13:31, Jack Trades wrote: > > > On Tue, Aug 30, 2011 at 2:37 AM, Rob Williscroft wrote: > >> >> > That's brilliant and works flawlessly. Thank you very much! >> >> If an impementation (as you say up thread) can populate globals >> or locals with whatever they want, then how do you know that last >> item added was the function definition the user supplied ? That's not an issue. The last statement that is executed will be the "def" statement. >> Rob. > > I spoke a bit too soon with the "works flawlessly" post. In addition to > your issue, there is also the problem that supplying an empty environment > does not allow the user to call necessary functions (like scheme_eval). You could simply prepend the function definition string with whatever imports are needed. -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: Returning a value from exec or a better solution
On 30 August 2011 22:48, Rob Williscroft wrote: > Arnaud Delobelle wrote in > news:CAJ6cK1YVi3NQgdZOUdhAESf133pUkdazM1PkSP=p6xfayvo...@mail.gmail.com in > gmane.comp.python.general: > >> On 30 August 2011 13:31, Jack Trades wrote: >>> >>> >>> On Tue, Aug 30, 2011 at 2:37 AM, Rob Williscroft wrote: >>> >>>> >>>> > That's brilliant and works flawlessly. ś˙Thank you very much! >>>> >>>> If an impementation (as you say up thread) can populate globals >>>> or locals with whatever they want, then how do you know that last >>>> item added was the function definition the user supplied ? > >> That's not an issue. The last statement that is executed will be the >> "def" statement. > > You don't know that, an implementation may for example set __bultins__ > to None, prior to returning, its not an unreasonable thing to do and > the docs don't say they can't. I haven't studied the docs but I'm certain that such an implementation would break a lot of code. -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: Closures and Partial Function Application
On 31 August 2011 17:45, Travis Parks wrote: > I was a little disappointed the other day when I realized that > closures were read-only. I like to use closures quite a bit. > > Can someone explain why this limitation exists? Secondly, since I can > cheat by wrapping the thing being closure-ified, how can I write a > simple wrapper that has all the same members as the thing (decorator), > that then applies them to the underlying thing? I don't understand. Can you give an example? > I also like partial function application. What is the easiest way of > achieving this in Python? Would it look something like this: > > def foo(x, y): > return x + y > > xFoo = lambda y: foo(10, y) from functools import partial foo10 = partial(foo, 10) HTH Arnaud -- http://mail.python.org/mailman/listinfo/python-list
PyGILState_Release called twice in embedded application
Hi all,
I'm running in a crash due to a ResourceWarning (some socket is not
closed in a used module) after calling PyGILState_Release.
I'm running Python in a native thread (so a thread created by C not
Python). I'm acquiring the GIL through PyGILState_Ensure before doing
any CPYthon calls and releasing when finished using PyGILState_Release.
This has worked fine. But now I'm using a python module (openai) which
doesn't close its socket correctly which results in a ResourceWarning
which triggers an assert.
In the backtrace (below) I can see PyGILState_Release is called again.
(7) while I've already called it (126).
I can make the crash go away by adding
import warnings
warnings.simplefilter("ignore", ResourceWarning)
to my python code. But I'd rather prevent this from happening in the
first place.
Any suggestion very welcomed cause I'm puzzled.
Rg,
Arnaud
1 __pthread_kill_implementation pthread_kill.c 44
0x76fcaccc
2 __pthread_kill_internalpthread_kill.c 78
0x76fcad2f
3 __GI_raise raise.c 26
0x76f7bef2
4 __GI_abort abort.c 79
0x76f66472
5 __assert_fail_base assert.c 92
0x76f66395
6 __GI___assert_fail assert.c 101
0x76f74df2
7 PyGILState_Release pystate.c1742
0x77abcf9f
8 tracemalloc_raw_alloc _tracemalloc.c 779
0x77afa803
9 tracemalloc_raw_malloc _tracemalloc.c 789
0x77afa844
10 PyMem_RawMallocobmalloc.c 586
0x779dab8d
11 decode_current_locale fileutils.c 472
0x77ada4ab
12 _Py_DecodeLocaleEx fileutils.c 598
0x77adafce
13 unicode_decode_locale unicodeobject.c 3970
0x77a13b65
14 PyUnicode_DecodeLocale unicodeobject.c 4012
0x77a18c79
15 PyErr_SetFromErrnoWithFilenameObjects errors.c 772
0x77a94178
16 PyErr_SetFromErrnoWithFilenameObject errors.c 750
0x77a942ce
17 posix_path_object_errorposixmodule.c1709
0x77ae4201
18 path_object_error posixmodule.c1719
0x77ae420f
19 path_error posixmodule.c1737
0x77ae4221
20 posix_do_stat posixmodule.c2565
0x77af42f7
21 os_stat_impl posixmodule.c2897
0x77af4491
22 os_statposixmodule.c.h 71
0x77af45e5
23 cfunction_vectorcall_FASTCALL_KEYWORDS methodobject.c 443
0x779d2ece
24 _PyObject_VectorcallTstate pycore_call.h92
0x7798b579
25 PyObject_Vectorcallcall.c 299
0x7798b65e
26 _PyEval_EvalFrameDefault ceval.c 4772
0x77a75453
27 _PyEval_EvalFrame pycore_ceval.h 73
0x77a79887
28 _PyEval_Vector ceval.c 6435
0x77a799b1
29 _PyFunction_Vectorcall call.c 393
0x7798b215
30 _PyObject_VectorcallTstate pycore_call.h92
0x7798b579
31 PyObject_CallOneArgcall.c 376
0x7798b6cd
32 call_show_warning _warnings.c 609
0x77a31d24
33 warn_explicit _warnings.c 746
0x77a3223b
34 do_warn_warnings.c 947
0x77a3236f
35 warn_unicode _warnings.c 1106
0x77a32414
36 _PyErr_WarnFormatV _warnings.c 1126
0x77a32481
37 PyErr_ResourceWarning _warnings.c 1177
0x77a32d38
38 sock_finalize socketmodule.c 5073
0x76840cf3
39 PyObject_CallFinalizer object.c 208
0x779d6449
40 PyObject_CallFinalizerFromDealloc object.c 226
0x779d75ac
41 subtype_dealloctypeobject.c 1382
0x779ecf81
42 _Py_Deallocobject.c 2390
0x779d6c38
43 Py_DECREF object.h 527
0x779c116e
44 Py_XDECREF object.h 602
0x779c118d
45 free_keys_object dictobject.c 664
0x779c120a
46 dictkeys_decrefdictobject.c 324
0x779c130f
47 dict_dealloc dictobject.c 2378
0x779c2eeb
48 _Py_Deallocobject.c 2390
0x779d6c38
4
Re: PyGILState_Release called twice in embedded application
On 23-03-2023 13:33, Barry Scott wrote:
On 23 Mar 2023, at 08:46, Arnaud Loonstra wrote:
Hi all,
I'm running in a crash due to a ResourceWarning (some socket is not closed in a
used module) after calling PyGILState_Release.
I'm running Python in a native thread (so a thread created by C not Python).
I'm acquiring the GIL through PyGILState_Ensure before doing any CPYthon calls
and releasing when finished using PyGILState_Release.
This has worked fine. But now I'm using a python module (openai) which doesn't
close its socket correctly which results in a ResourceWarning which triggers an
assert.
In the backtrace (below) I can see PyGILState_Release is called again. (7)
while I've already called it (126).
I can make the crash go away by adding
import warnings
warnings.simplefilter("ignore", ResourceWarning)
to my python code. But I'd rather prevent this from happening in the first
place.
Any suggestion very welcomed cause I'm puzzled.
What 3rd party C extension are you running with?
I'd be surprised if the cpython code was the issue.
Barry
I'm not using any 3rd party extension myself. But the issue is caused by
the openai module using the requests module which is not closing sockets:
https://github.com/openai/openai-python/issues/140
I'm not aware what C extensions they might use.
Btw, I've tested this with python 3.8 and 3.11.2.
Rg,
Arnaud
--
https://mail.python.org/mailman/listinfo/python-list
Embedding Python crash on PyTuple_New
Hi,
I've got Python embedded successfully in a program up until now as I'm
now running into weird GC related segfaults. I'm currently trying to
debug this but my understanding of CPython limits me here.
I'm creating a Tuple in C but it crashes on creating it after a while.
It doesn't make sense which makes me wonder something else must be
happening? Could be it just crashes here because the GC is cleaning up
stuff completely unrelated to the allocation of the new tuple? How can I
troubleshoot this?
I've got CPython compiled with --with-valgrind --without-pymalloc
--with-pydebug
In C I'm creating a tuple with the following method:
static PyObject *
s_py_zosc_tuple(pythonactor_t *self, zosc_t *oscmsg)
{
assert(self);
assert(oscmsg);
char *format = zosc_format(oscmsg);
PyObject *rettuple = PyTuple_New((Py_ssize_t) strlen(format) );
It segfaults here (frame 16) after 320 times (consistently)
1 __GI_raise raise.c 49 0x772c4e71
2 __GI_abort abort.c 79 0x772ae536
3 fatal_errorpylifecycle.c2183 0x77d84b4f
4 Py_FatalError pylifecycle.c2193 0x77d878b2
5 _PyObject_AssertFailed object.c 2200 0x77c93cf2
6 visit_decref gcmodule.c 378 0x77dadfd5
7 tupletraverse tupleobject.c623 0x77ca3e81
8 subtract_refs gcmodule.c 406 0x77dad340
9 collectgcmodule.c 1054 0x77dae838
10 collect_with_callback gcmodule.c 1240 0x77daf17b
11 collect_generationsgcmodule.c 1262 0x77daf3f6
12 _PyObject_GC_Alloc gcmodule.c 1977 0x77daf4f2
13 _PyObject_GC_Mallocgcmodule.c 1987 0x77dafebc
14 _PyObject_GC_NewVargcmodule.c 2016 0x77daffa5
15 PyTuple_Newtupleobject.c118 0x77ca4da7
16 s_py_zosc_tuplepythonactor.c366 0x5568cc82
17 pythonactor_socket pythonactor.c664 0x5568dac7
18 pythonactor_handle_msg pythonactor.c862 0x5568e472
19 pythonactor_handlerpythonactor.c828 0x5568e2e2
20 sphactor_actor_run sphactor_actor.c 855 0x558cb268
...
Any pointer really appreciated.
Rg,
Arnaud
--
https://mail.python.org/mailman/listinfo/python-list
Re: Embedding Python crash on PyTuple_New
On 23-11-2021 13:07, Arnaud Loonstra wrote:
Hi,
I've got Python embedded successfully in a program up until now as I'm
now running into weird GC related segfaults. I'm currently trying to
debug this but my understanding of CPython limits me here.
I'm creating a Tuple in C but it crashes on creating it after a while.
It doesn't make sense which makes me wonder something else must be
happening? Could be it just crashes here because the GC is cleaning up
stuff completely unrelated to the allocation of the new tuple? How can I
troubleshoot this?
I've got CPython compiled with --with-valgrind --without-pymalloc
--with-pydebug
In C I'm creating a tuple with the following method:
static PyObject *
s_py_zosc_tuple(pythonactor_t *self, zosc_t *oscmsg)
{
assert(self);
assert(oscmsg);
char *format = zosc_format(oscmsg);
PyObject *rettuple = PyTuple_New((Py_ssize_t) strlen(format) );
It segfaults here (frame 16) after 320 times (consistently)
1 __GI_raise raise.c 49 0x772c4e71
2 __GI_abort abort.c 79 0x772ae536
3 fatal_error pylifecycle.c 2183 0x77d84b4f
4 Py_FatalError pylifecycle.c 2193 0x77d878b2
5 _PyObject_AssertFailed object.c 2200 0x77c93cf2
6 visit_decref gcmodule.c 378 0x77dadfd5
7 tupletraverse tupleobject.c 623 0x77ca3e81
8 subtract_refs gcmodule.c 406 0x77dad340
9 collect gcmodule.c 1054 0x77dae838
10 collect_with_callback gcmodule.c 1240 0x77daf17b
11 collect_generations gcmodule.c 1262 0x77daf3f6
12 _PyObject_GC_Alloc gcmodule.c 1977 0x77daf4f2
13 _PyObject_GC_Malloc gcmodule.c 1987 0x77dafebc
14 _PyObject_GC_NewVar gcmodule.c 2016 0x77daffa5
15 PyTuple_New tupleobject.c 118 0x77ca4da7
16 s_py_zosc_tuple pythonactor.c 366 0x5568cc82
17 pythonactor_socket pythonactor.c 664 0x5568dac7
18 pythonactor_handle_msg pythonactor.c 862 0x5568e472
19 pythonactor_handler pythonactor.c 828 0x5568e2e2
20 sphactor_actor_run sphactor_actor.c 855 0x558cb268
...
Any pointer really appreciated.
I've found enabling PYTHONTRACEMALLOC=1 in the environment gives me a
pointer to where to offending block was allocated.
I'm calling this method from C:
18def handleSocket(self, addr, data, type, name, uuid):
19if addr == "/pulse":
20self.lampval += 1
21return (addr, [0,0])
Modules/gcmodule.c:108: gc_decref: Assertion "gc_get_refs(g) > 0"
failed: refcount is too small
Memory block allocated at (most recent call first):
File "/home/arnaud/src/build-gazebosc-Desktop-Debug/bin/lampen.py",
line 21
object address : 0x7fffd81154c0
object refcount : 1
object type : 0x77f3df20
object type name: list
object repr : [117, 0]
Fatal Python error: _PyObject_AssertFailed
Python runtime state: initialized
Current thread 0x72481640 (most recent call first):
File "/home/arnaud/src/build-gazebosc-Desktop-Debug/bin/lampen.py",
line 21 in handleSocket
Thread 0x75d288c0 (most recent call first):
Now it clearly says the refcount is 1 so I'm puzzling what it means?
Rg,
Arnaud
--
https://mail.python.org/mailman/listinfo/python-list
Re: Embedding Python crash on PyTuple_New
On 23-11-2021 15:34, MRAB wrote:
On 2021-11-23 12:07, Arnaud Loonstra wrote:
Hi,
I've got Python embedded successfully in a program up until now as I'm
now running into weird GC related segfaults. I'm currently trying to
debug this but my understanding of CPython limits me here.
I'm creating a Tuple in C but it crashes on creating it after a while.
It doesn't make sense which makes me wonder something else must be
happening? Could be it just crashes here because the GC is cleaning up
stuff completely unrelated to the allocation of the new tuple? How can I
troubleshoot this?
I've got CPython compiled with --with-valgrind --without-pymalloc
--with-pydebug
In C I'm creating a tuple with the following method:
static PyObject *
s_py_zosc_tuple(pythonactor_t *self, zosc_t *oscmsg)
{
assert(self);
assert(oscmsg);
char *format = zosc_format(oscmsg);
PyObject *rettuple = PyTuple_New((Py_ssize_t) strlen(format) );
It segfaults here (frame 16) after 320 times (consistently)
1 __GI_raise raise.c 49 0x772c4e71
2 __GI_abort abort.c 79 0x772ae536
3 fatal_error pylifecycle.c 2183 0x77d84b4f
4 Py_FatalError pylifecycle.c 2193 0x77d878b2
5 _PyObject_AssertFailed object.c 2200 0x77c93cf2
6 visit_decref gcmodule.c 378 0x77dadfd5
7 tupletraverse tupleobject.c 623 0x77ca3e81
8 subtract_refs gcmodule.c 406 0x77dad340
9 collect gcmodule.c 1054 0x77dae838
10 collect_with_callback gcmodule.c 1240 0x77daf17b
11 collect_generations gcmodule.c 1262 0x77daf3f6
12 _PyObject_GC_Alloc gcmodule.c 1977 0x77daf4f2
13 _PyObject_GC_Malloc gcmodule.c 1987 0x77dafebc
14 _PyObject_GC_NewVar gcmodule.c 2016 0x77daffa5
15 PyTuple_New tupleobject.c 118 0x77ca4da7
16 s_py_zosc_tuple pythonactor.c 366 0x5568cc82
17 pythonactor_socket pythonactor.c 664 0x5568dac7
18 pythonactor_handle_msg pythonactor.c 862 0x5568e472
19 pythonactor_handler pythonactor.c 828 0x5568e2e2
20 sphactor_actor_run sphactor_actor.c 855 0x558cb268
...
Any pointer really appreciated.
You're creating a tuple that'll have the same number of members as the
length of a string? That looks strange to me.
How are you setting the tuple's members?
It's from a serialisation format called OSC. The string describes the
type of bytes, every character is a type.
I'm creating the tuple as follows:
PyObject *rettuple = PyTuple_New((Py_ssize_t) strlen(format) );
Then I iterate the OSC message using the format string, (just showing
handling an int (i))
char type = '0';
Py_ssize_t pos = 0;
const void *data = zosc_first(oscmsg, &type);
while(data)
{
switch (type)
{
case('i'):
{
int32_t val = 9;
int rc = zosc_pop_int32(oscmsg, &val);
assert(rc == 0);
PyObject *o = PyLong_FromLong((long)val);
assert( o );
rc = PyTuple_SetItem(rettuple, pos, o);
assert(rc == 0);
break;
}
Full code is here:
https://github.com/hku-ect/gazebosc/blob/822452dfa27201db274d37ce09e835d98fe500b2/Actors/pythonactor.c#L360
--
https://mail.python.org/mailman/listinfo/python-list
Re: Embedding Python crash on PyTuple_New
On 23-11-2021 16:37, MRAB wrote:
On 2021-11-23 15:17, MRAB wrote:
On 2021-11-23 14:44, Arnaud Loonstra wrote:
On 23-11-2021 15:34, MRAB wrote:
On 2021-11-23 12:07, Arnaud Loonstra wrote:
Hi,
I've got Python embedded successfully in a program up until now as I'm
now running into weird GC related segfaults. I'm currently trying to
debug this but my understanding of CPython limits me here.
I'm creating a Tuple in C but it crashes on creating it after a while.
It doesn't make sense which makes me wonder something else must be
happening? Could be it just crashes here because the GC is cleaning up
stuff completely unrelated to the allocation of the new tuple? How
can I
troubleshoot this?
I've got CPython compiled with --with-valgrind --without-pymalloc
--with-pydebug
In C I'm creating a tuple with the following method:
static PyObject *
s_py_zosc_tuple(pythonactor_t *self, zosc_t *oscmsg)
{
assert(self);
assert(oscmsg);
char *format = zosc_format(oscmsg);
PyObject *rettuple = PyTuple_New((Py_ssize_t) strlen(format) );
It segfaults here (frame 16) after 320 times (consistently)
1 __GI_raise raise.c 49 0x772c4e71
2 __GI_abort abort.c 79 0x772ae536
3 fatal_error pylifecycle.c 2183 0x77d84b4f
4 Py_FatalError pylifecycle.c 2193 0x77d878b2
5 _PyObject_AssertFailed object.c 2200 0x77c93cf2
6 visit_decref gcmodule.c 378 0x77dadfd5
7 tupletraverse tupleobject.c 623 0x77ca3e81
8 subtract_refs gcmodule.c 406 0x77dad340
9 collect gcmodule.c 1054 0x77dae838
10 collect_with_callback gcmodule.c 1240 0x77daf17b
11 collect_generations gcmodule.c 1262 0x77daf3f6
12 _PyObject_GC_Alloc gcmodule.c 1977 0x77daf4f2
13 _PyObject_GC_Malloc gcmodule.c 1987 0x77dafebc
14 _PyObject_GC_NewVar gcmodule.c 2016 0x77daffa5
15 PyTuple_New tupleobject.c 118 0x77ca4da7
16 s_py_zosc_tuple pythonactor.c 366 0x5568cc82
17 pythonactor_socket pythonactor.c 664 0x5568dac7
18 pythonactor_handle_msg pythonactor.c 862 0x5568e472
19 pythonactor_handler pythonactor.c 828 0x5568e2e2
20 sphactor_actor_run sphactor_actor.c 855 0x558cb268
...
Any pointer really appreciated.
You're creating a tuple that'll have the same number of members as
the length of a string? That looks strange to me.
How are you setting the tuple's members?
It's from a serialisation format called OSC. The string describes the
type of bytes, every character is a type.
I'm creating the tuple as follows:
PyObject *rettuple = PyTuple_New((Py_ssize_t) strlen(format) );
Then I iterate the OSC message using the format string, (just showing
handling an int (i))
char type = '0';
Py_ssize_t pos = 0;
const void *data = zosc_first(oscmsg, &type);
while(data)
{
switch (type)
{
case('i'):
{
int32_t val = 9;
int rc = zosc_pop_int32(oscmsg, &val);
assert(rc == 0);
PyObject *o = PyLong_FromLong((long)val);
assert( o );
rc = PyTuple_SetItem(rettuple, pos, o);
assert(rc == 0);
break;
}
Full code is here:
https://github.com/hku-ect/gazebosc/blob/822452dfa27201db274d37ce09e835d98fe500b2/Actors/pythonactor.c#L360
Looking at that code, you have:
PyObject *o = Py_BuildValue("s#", str, 1);
what I'd check is the type of the 1 that you're passing. Wouldn't the
compiler assume that it's an int?
The format string tells the function to expect a Py_ssize_t, but how
would the compiler know that?
Looking at https://www.mankier.com/3/zosc, it says for 'T' and 'F' "(no
value required)", but you're doing:
int rc = zosc_pop_bool(oscmsg, &bl);
If no value is required, is there a bool there to be popped?
The value is only required to set a user provided boolean to the value
in the message. There's no boolean value encoded in the message, only
the T and F in the format string.
With regards to the Py_BuildValue("s#", str, 1);, that's a valid point.
I'll fix that. However in the segfaults I'm testing that code is not
touched.
I'm now testing different parts of the code to see if it runs stable.
I've found it runs stable if I do not process the returned tuple.
PyObject *pReturn = PyObject_CallMethod(self->pyinstance,
"handleSocket", "sOsss",
oscaddress,
py_osctuple,
ev->type, ev->name, strdup(ev->u
Re: Embedding Python crash on PyTuple_New
On 23-11-2021 18:31, MRAB wrote:
On 2021-11-23 16:04, Arnaud Loonstra wrote:
On 23-11-2021 16:37, MRAB wrote:
On 2021-11-23 15:17, MRAB wrote:
On 2021-11-23 14:44, Arnaud Loonstra wrote:
On 23-11-2021 15:34, MRAB wrote:
On 2021-11-23 12:07, Arnaud Loonstra wrote:
Hi,
I've got Python embedded successfully in a program up until now
as I'm
now running into weird GC related segfaults. I'm currently trying to
debug this but my understanding of CPython limits me here.
I'm creating a Tuple in C but it crashes on creating it after a
while.
It doesn't make sense which makes me wonder something else must be
happening? Could be it just crashes here because the GC is
cleaning up
stuff completely unrelated to the allocation of the new tuple?
How can I
troubleshoot this?
I've got CPython compiled with --with-valgrind --without-pymalloc
--with-pydebug
In C I'm creating a tuple with the following method:
static PyObject *
s_py_zosc_tuple(pythonactor_t *self, zosc_t *oscmsg)
{
assert(self);
assert(oscmsg);
char *format = zosc_format(oscmsg);
PyObject *rettuple = PyTuple_New((Py_ssize_t)
strlen(format) );
It segfaults here (frame 16) after 320 times (consistently)
1 __GI_raise raise.c 49 0x772c4e71
2 __GI_abort abort.c 79 0x772ae536
3 fatal_error pylifecycle.c 2183 0x77d84b4f
4 Py_FatalError pylifecycle.c 2193 0x77d878b2
5 _PyObject_AssertFailed object.c 2200 0x77c93cf2
6 visit_decref gcmodule.c 378 0x77dadfd5
7 tupletraverse tupleobject.c 623 0x77ca3e81
8 subtract_refs gcmodule.c 406 0x77dad340
9 collect gcmodule.c 1054 0x77dae838
10 collect_with_callback gcmodule.c 1240 0x77daf17b
11 collect_generations gcmodule.c 1262 0x77daf3f6
12 _PyObject_GC_Alloc gcmodule.c 1977 0x77daf4f2
13 _PyObject_GC_Malloc gcmodule.c 1987 0x77dafebc
14 _PyObject_GC_NewVar gcmodule.c 2016 0x77daffa5
15 PyTuple_New tupleobject.c 118 0x77ca4da7
16 s_py_zosc_tuple pythonactor.c 366 0x5568cc82
17 pythonactor_socket pythonactor.c 664 0x5568dac7
18 pythonactor_handle_msg pythonactor.c 862 0x5568e472
19 pythonactor_handler pythonactor.c 828 0x5568e2e2
20 sphactor_actor_run sphactor_actor.c 855 0x558cb268
...
Any pointer really appreciated.
You're creating a tuple that'll have the same number of members as
the length of a string? That looks strange to me.
How are you setting the tuple's members?
It's from a serialisation format called OSC. The string describes the
type of bytes, every character is a type.
I'm creating the tuple as follows:
PyObject *rettuple = PyTuple_New((Py_ssize_t) strlen(format) );
Then I iterate the OSC message using the format string, (just showing
handling an int (i))
char type = '0';
Py_ssize_t pos = 0;
const void *data = zosc_first(oscmsg, &type);
while(data)
{
switch (type)
{
case('i'):
{
int32_t val = 9;
int rc = zosc_pop_int32(oscmsg, &val);
assert(rc == 0);
PyObject *o = PyLong_FromLong((long)val);
assert( o );
rc = PyTuple_SetItem(rettuple, pos, o);
assert(rc == 0);
break;
}
Full code is here:
https://github.com/hku-ect/gazebosc/blob/822452dfa27201db274d37ce09e835d98fe500b2/Actors/pythonactor.c#L360
Looking at that code, you have:
PyObject *o = Py_BuildValue("s#", str, 1);
what I'd check is the type of the 1 that you're passing. Wouldn't the
compiler assume that it's an int?
The format string tells the function to expect a Py_ssize_t, but how
would the compiler know that?
Looking at https://www.mankier.com/3/zosc, it says for 'T' and 'F'
"(no value required)", but you're doing:
int rc = zosc_pop_bool(oscmsg, &bl);
If no value is required, is there a bool there to be popped?
The value is only required to set a user provided boolean to the value
in the message. There's no boolean value encoded in the message, only
the T and F in the format string.
With regards to the Py_BuildValue("s#", str, 1);, that's a valid point.
I'll fix that. However in the segfaults I'm testing that code is not
touched.
You can use "C" as a format string for Py_BuildValue to convert a C int
representing a character to a Python string.
I'm now testing different parts of the code to see if it runs stable.
I've found it runs stable if I do not process the returned tuple.
PyObject *pReturn = PyObject_CallMethod(self-&
Re: Embedding Python crash on PyTuple_New
On 24-11-2021 01:46, MRAB wrote:
On 2021-11-23 20:25, Arnaud Loonstra wrote:
On 23-11-2021 18:31, MRAB wrote:
On 2021-11-23 16:04, Arnaud Loonstra wrote:
On 23-11-2021 16:37, MRAB wrote:
On 2021-11-23 15:17, MRAB wrote:
On 2021-11-23 14:44, Arnaud Loonstra wrote:
On 23-11-2021 15:34, MRAB wrote:
On 2021-11-23 12:07, Arnaud Loonstra wrote:
Hi,
I've got Python embedded successfully in a program up until now
as I'm
now running into weird GC related segfaults. I'm currently
trying to
debug this but my understanding of CPython limits me here.
I'm creating a Tuple in C but it crashes on creating it after a
while.
It doesn't make sense which makes me wonder something else must be
happening? Could be it just crashes here because the GC is
cleaning up
stuff completely unrelated to the allocation of the new tuple?
How can I
troubleshoot this?
I've got CPython compiled with --with-valgrind --without-pymalloc
--with-pydebug
In C I'm creating a tuple with the following method:
static PyObject *
s_py_zosc_tuple(pythonactor_t *self, zosc_t *oscmsg)
{
assert(self);
assert(oscmsg);
char *format = zosc_format(oscmsg);
PyObject *rettuple = PyTuple_New((Py_ssize_t)
strlen(format) );
It segfaults here (frame 16) after 320 times (consistently)
1 __GI_raise raise.c 49 0x772c4e71
2 __GI_abort abort.c 79 0x772ae536
3 fatal_error pylifecycle.c 2183 0x77d84b4f
4 Py_FatalError pylifecycle.c 2193 0x77d878b2
5 _PyObject_AssertFailed object.c 2200 0x77c93cf2
6 visit_decref gcmodule.c 378 0x77dadfd5
7 tupletraverse tupleobject.c 623 0x77ca3e81
8 subtract_refs gcmodule.c 406 0x77dad340
9 collect gcmodule.c 1054 0x77dae838
10 collect_with_callback gcmodule.c 1240 0x77daf17b
11 collect_generations gcmodule.c 1262 0x77daf3f6
12 _PyObject_GC_Alloc gcmodule.c 1977 0x77daf4f2
13 _PyObject_GC_Malloc gcmodule.c 1987 0x77dafebc
14 _PyObject_GC_NewVar gcmodule.c 2016 0x77daffa5
15 PyTuple_New tupleobject.c 118 0x77ca4da7
16 s_py_zosc_tuple pythonactor.c 366 0x5568cc82
17 pythonactor_socket pythonactor.c 664 0x5568dac7
18 pythonactor_handle_msg pythonactor.c 862 0x5568e472
19 pythonactor_handler pythonactor.c 828 0x5568e2e2
20 sphactor_actor_run sphactor_actor.c 855 0x558cb268
...
Any pointer really appreciated.
[snip]
Basically, yes, but I won't be surprised if it was due to too few
INCREFs or too many DECREFs somewhere.
https://github.com/hku-ect/gazebosc/blob/505b30c46bf3f78d188c3f575c80e294d3db7e5d/Actors/pythonactor.c#L286
Incidentally, in s_py_zosc_tuple, you're not doing "assert(rc == 0);"
after "after zosc_pop_float" or "zosc_pop_double".
Thanks for those pointers! I think your intuition is right. I might have
found the bugger. In s_py_zosc I call Py_DECREF on pAddress and pData.
However they are acquired by PyTuple_GetItem which returns a borrowed
reference. I think pAddress and pData are then also 'decrefed' when the
pReturn tuple which contains pAddress and pData is 'decrefed'?
Yes, members of a container are DECREFed when the container is destroyed.
It's bad practice for a function to DECREF its arguments unless the
function's sole purpose is cleanup because the function won't know where
the arguments came from.
I'm finding it out now. What strikes me was how hard it was to debug
this. I think it was caused because I INCREFed the return object. I
guess I did that to workaround the wrong DECREF data in the return
object. However that caused a hell to debug. I'm really curious what the
best practices are for debugging embedded CPython.
Thanks big time for your feedback!
--
https://mail.python.org/mailman/listinfo/python-list
Embedding Python (3.7) on OSX crash on Py_Initialize when run as bundle
Hi, I'm trying to wrap my head around an OSX app which has python embedded. This works fine if I run the build on the same machine as it is build on. However when using Travis to build the app I can only get it to run from the terminal. A double click on the appdir (or using open) results in a crash. trace: 0 libsystem_kernel.dylib 0x7fffb9d6cd42 __pthread_kill + 10 1 libsystem_pthread.dylib 0x7fffb9e5a457 pthread_kill + 90 2 libsystem_c.dylib 0x7fffb9cd24bb __abort + 140 3 libsystem_c.dylib 0x7fffb9cd242f abort + 144 4 com.your-company-name.www 0x000108c6674e fatal_error + 606 5 com.your-company-name.www 0x000108c65b3a _Py_FatalInitError + 26 6 com.your-company-name.www 0x000108c65bbe Py_Initialize + 126 I can't figure out what is going wrong. I can only guess that's there's something preventing it to access the python files? The build log on travis is here: https://travis-ci.org/hku-ect/NatNet2OSCbridge/jobs/499348594 It's artefact is here: https://pong.hku.nl/~buildbot/nightly/NatNet2OSCbridge_osx_7ceb.zip Python is build with the following flags: ./configure --prefix $HOME/openFrameworks/apps/devApps/$APPNAME/bin/python --disable-shared --with-openssl=$(brew --prefix openssl); Anybody any pointers or advice? Rg, Arnaud -- https://mail.python.org/mailman/listinfo/python-list
return a ctypes object to C
Hi all,
I'm trying to wrap my head around the ctypes API. I have a C structure I
wish to create in Python and then return from python to C.
So a python method is called from C and needs to return an object which
we then process in C again.
I have a binding to access and create the C methods and structures so in
Python I can call the Zmsg() constructor. I now need to return this.
My python test method is simply:
def actor_test( *args, **kwargs):
print("test")
msg = Zmsg()
frame = Zframe(b"Hello", 5)
msg.prepend(frame)
return msg
the method is called from C as follows:
PyObject *pReturn = PyObject_CallObject(pFunc, NULL);
This correctly calls the method. However the returned object is of
course a PyObject*. The debugger says it's
"" PyObject
[class] ""
[super class] ""
[meta type] ""
ob_refcnt 1 Py_ssize_t
However how I can I get it back to the original C type (zmsg_t *)
Any help really appreciated.
Rg,
Arnaud
--
https://mail.python.org/mailman/listinfo/python-list
Re: return a ctypes object to C
On 30-10-2019 09:32, Arnaud Loonstra wrote:
Hi all,
I'm trying to wrap my head around the ctypes API. I have a C structure I
wish to create in Python and then return from python to C.
So a python method is called from C and needs to return an object which
we then process in C again.
I have a binding to access and create the C methods and structures so in
Python I can call the Zmsg() constructor. I now need to return this.
My python test method is simply:
def actor_test( *args, **kwargs):
print("test")
msg = Zmsg()
frame = Zframe(b"Hello", 5)
msg.prepend(frame)
return msg
the method is called from C as follows:
PyObject *pReturn = PyObject_CallObject(pFunc, NULL);
This correctly calls the method. However the returned object is of
course a PyObject*. The debugger says it's
"" PyObject
[class] ""
[super class] ""
[meta type] ""
ob_refcnt 1 Py_ssize_t
However how I can I get it back to the original C type (zmsg_t *)
Any help really appreciated.
What I've found so far is that I can return the address of the ctypes
object.
msg = Zmsg()
frame = Zframe(b"Hello", 5)
msg.prepend(frame)
return addressof(msg._as_parameter_.contents)
In C I can then cast it back to the original type.
PyObject *pReturn = PyObject_CallObject(pFunc, NULL);
assert(pReturn);
long bla = PyLong_AsLong(pReturn);
zmsg_t* test = (zmsg_t *)bla;
assert(test);
char *hello = zmsg_popstr(test);
assert(hello);
assert(streq(hello, "Hello"));
This works, I'm not sure if this is the right way. It also creates a
complicated setup with the garbage collector.
Anybody better ideas?
Rg,
Arnaud
--
https://mail.python.org/mailman/listinfo/python-list
